Run a two-person conversation through Whisper and you get one continuous block of prose. A change of speaker looks identical to a change of sentence. For a solo recording that is irrelevant; for an interview, a meeting or a panel, it is the main thing standing between the transcript and something usable.
This is not a gap in one tool. It is a different task.
Transcription and diarization are two problems
Transcription answers “what words were said.” Diarization answers “how many people are speaking, and which segments belong to each.” They use different models and different signals — diarization works from voice characteristics (pitch, timbre, speaking rate), not from the words at all.
Whisper does one of these. Services showing “Speaker 1 / Speaker 2” are running a diarization model alongside it and merging the outputs by timestamp.
Two consequences worth internalising:
- The two can disagree. A speaker change mid-sentence produces a segment boundary that does not line up with a sentence boundary, and the merge has to guess.
- Diarization does not know names. It produces “Speaker 1,” “Speaker 2.” Mapping those to people is always manual, or comes from voice enrollment you set up in advance.
What diarization gets wrong even when it is there
Worth knowing before you go looking for a tool that has it, because the error modes are annoying:
- Speaker count is inferred. Three people can come out as two if two voices are similar, or as four if one person’s voice changes across a long recording.
- Short interjections get absorbed. A one-word “right” or “exactly” from the other person is often attached to whoever was already talking.
- Crosstalk defeats it entirely. Overlapping speech produces one blended segment, and it gets assigned to one speaker or the other.
- Labels can swap. Over a long file, Speaker 1 in the first half is occasionally Speaker 2 in the second.
So diarization moves you from “no attribution” to “attribution that needs checking.” That is a real improvement, not a solved problem.
Three ways to get attribution without a diarization model
1. Record separate tracks (best by a wide margin)
If each participant is recorded to their own file — standard in most remote-recording and conferencing tools — transcribe each file independently and merge by timestamp.
Each transcript then contains exactly one person. Attribution is exact, not inferred, because there is nothing to infer. No diarization model achieves this, since they are all working backwards from a mixed signal.
If your setup can do this, stop reading here. It is the whole answer.
2. Reconstruct from context
For a two-person interview this is faster than it sounds. You know what you asked; questions and answers alternate; the vocabulary differs. Reading the transcript with your question list beside it, most boundaries are obvious.
It degrades quickly past two speakers, and it is unreliable for anything you plan to quote with attribution.
3. Mark turns during recording
If you control the recording, a deliberate half-second pause before each new speaker leaves a gap that shows up as a segment boundary in the timestamps. Crude, free, and it makes the manual pass much faster.
When you actually need speaker labels
Being honest about this saves effort:
| Situation | Need labels? |
|---|---|
| Solo recording, voice memo, lecture | No |
| Searching your own meeting later | Rarely — you want keywords, not attribution |
| Interview you will quote from | Yes, and it must be verified |
| Legal or compliance record | Yes, and a human has to attest to it |
| Podcast show notes | Usually yes, but a manual pass is fine |
| Subtitles | No — captions do not name speakers |
A lot of transcription work does not need this at all, which is why browser tools that skip it are still useful for most of what people do.
What this site does
Our transcription runs Whisper in your browser and produces one text stream — no speaker labels. That is the honest state of it.
For attribution, record separate tracks and transcribe each file; it is more accurate than any automatic method and costs nothing extra. For a mixed single-track recording where attribution matters legally, use a service with diarization and verify the result, or a human transcriber.
Common questions
Can Whisper be made to do this? Not on its own. Prompting it for speaker labels produces invented ones — it will happily write “Speaker 1:” in plausible places based on the text, which is worse than no labels because it looks authoritative.
How accurate is automatic diarization? On clean two-speaker audio with little overlap, good. Accuracy falls with more speakers, similar voices, crosstalk and background noise — the same conditions that hurt transcription, compounded.
Does a bigger Whisper model help? No. Model size affects word accuracy. Speaker attribution is not something any size of Whisper produces.
Is there a browser-based diarization option? The models exist but are considerably heavier than the transcription model, and the merge step is fiddly. For now, separate tracks remain the practical answer for local workflows.