A broken transcript is easy to deal with. Garbled characters, empty output, an error message — you notice, and you try something else.
The failure that costs you is the other one: a clean, grammatical, confident sentence describing something nobody said. Speech models are trained to produce likely text, and when the audio does not support any likely text, they produce likely text anyway.
Here are the four ways it happens, roughly in order of how much damage each one does.
1. The language was never actually detected
The worst case, because it corrupts everything rather than a phrase here and there.
Whisper needs to be told what language it is listening to. Some implementations detect it; several widely used ones do not, and silently assume English. The most common browser runtime for Whisper has this in the function that builds the decoder’s opening tokens:
if (!language) {
// TODO: Implement language detection
language = 'en';
}With no language supplied, the model is instructed to emit English — regardless of what it hears. Mandarin audio produces this:
Spoken: 快速快放。今天的会议记录需要整理成文字，请把重点整理出来。 Returned: “The fast-food delivery of the product is now available for the customer. Please get the order out of the package.”
Not a translation. A translation would carry the meaning across. This is the model doing exactly what it was told — produce English — with audio that contains none.
How to catch it: transcribe fifteen seconds of a language you speak that is not English. If the tool never asked you what language the file is in, this is the first thing to test.
How to avoid it: use a tool with an explicit language selector, and set it. If there is no selector, assume English is being forced.
2. Silence gets filled
Whisper processes audio in 30-second windows. Hand it a window containing no speech — a pause, a long gap before the recording starts, room tone — and it still produces its best guess at what belongs there.
The classic artifacts are subtitle-style boilerplate that appears in no known audio: “Thank you for watching,” “Subtitles by the Amara.org community,” a channel sign-off. These come from the training data, which included a great deal of subtitle text, and they surface when there is nothing else to condition on.
How to catch it: look at the timestamps. Invented text over silence usually carries a suspiciously long single segment, or repeats verbatim in several places.
How to avoid it: trim leading and trailing silence before transcribing. Tools that chunk with overlap and use voice-activity detection produce far less of this.
3. Words vanish at chunk boundaries
The mirror image of invention — and easier to miss, because nothing wrong appears on the page. Something is simply absent.
Long audio has to be cut into windows. If the tool cuts at exactly 30 seconds with no overlap, any sentence spanning the cut loses its middle. The result is grammatical, because two half-sentences often join into a plausible whole.
This is not hypothetical for hosted services either. Testing a cloud Whisper endpoint on a 94-second file, the output was 191 words where the source had about 222 — a 14% loss concentrated at the boundaries. “The radiologist noted bilateral pleural effusions” came back as “…renewals. effusions and mild cardiomegaly.” The first half of the sentence was gone, and what remained still parsed.
How to catch it: count. If you know roughly how long the recording is, a rough word count tells you a lot — ordinary speech runs 120–150 words per minute.
How to avoid it: use a tool that overlaps its windows (30 seconds of audio, stepping forward 25, reconciling the repeat). This is the default in good implementations and absent in quick ones.
4. Proper nouns get replaced with plausible neighbours
The everyday version, and the one you will meet most often.
Whisper does not leave gaps. Given a name, a drug, a product or a technical term it has not learned, it substitutes the nearest thing it does know. Kubernetes becomes “Cubernites.” apixaban becomes something adjacent. In one comparison a model returned hypothyroidism where the audio said hyperthyroidism — the opposite condition, one letter apart, in a sentence that reads perfectly.
That last example is the whole problem in miniature. A garbled word makes you check the audio. A real word that means the opposite does not.
How to catch it: proofread proper nouns specifically, against the audio, every time. They are where the errors concentrate.
How to avoid it: you cannot, entirely. A larger model reduces the rate without eliminating it. For high-stakes vocabulary, plan on a verification pass regardless of the tool.
What this means for how you use a transcript
None of this makes AI transcription unusable. It makes it a draft — which is still an enormous saving over typing from scratch.
The distinction that matters: AI transcription fails silently, and human transcription fails loudly. A human transcriber who cannot make out a word writes [inaudible]. A model writes its best guess with no flag at all. So the review pass is not optional overhead you can skip when you are busy — it is the part of the process the model does not do.
A workable rule:
- Casual use — searching your own recording, remembering what was said: use it raw.
- Anything published — subtitles, quotes, articles: read it against the audio once.
- Anything with consequences — medical, legal, financial, journalistic: verify every proper noun and number, and keep the original recording.
Common questions
Do the paid services have this fixed? No. The boundary-loss example above came from a hosted API, not a browser tool. Another hosted model silently truncated a 7.9-minute file, processing four of five sections and reporting no error. Paying moves the failure modes around rather than removing them.
Does a bigger model hallucinate less? Generally yes, and not uniformly. Bigger models are more confident, which occasionally makes their inventions harder to spot.
Is there a confidence score I can check? Some implementations expose per-segment log probabilities, and low confidence does correlate loosely with errors. It is not reliable enough to skip reading, particularly for the failure in section 1, where the model is entirely confident and entirely wrong.
Which of these can I rule out before starting? The first one, in about a minute — set the language explicitly and check that the tool honoured it. That single check removes the failure mode that ruins whole files.
Transcribe a file with the language set explicitly — it runs in your browser, so the audio stays on your machine.