AI speech-to-text tools like OpenAI Whisper, Otter.ai, and Google Speech-to-Text are genuinely impressive - in the right conditions. Claim a clean recording, one speaker, no background noise, and these models can hit word error rates below 5%. That is near-human accuracy.
The problem is that most professionally relevant audio is nothing like that. Focus groups, field interviews, remote meetings, and real-world recordings are noisy, overlapping, and acoustically messy. In these conditions, AI transcription does not gradually degrade - it collapses. And it does so in ways that are both predictable and poorly communicated by vendors.
Here are the four core failure modes that practitioners encounter most often, and why they happen.
1. Background Noise Destroys Accuracy Fast
Modern ASR models process audio as mel-spectrograms - visual representations of sound frequencies over time. They learn to associate these patterns with words during training. The fundamental issue: training data is overwhelmingly clean, studio-quality audio. Real-world noise introduces spectral patterns the model has never learned to separate from speech.
The result is a steep accuracy cliff tied to Signal-to-Noise Ratio (SNR). At 20 dB SNR - a reasonably quiet office - most leading models still perform well. Drop to 10 dB (an open-plan office with HVAC) and accuracy falls to 75–88%. In a busy café at 5 dB SNR, you are looking at 50–70% accuracy on a good day.
⚠ Hallucination risk: Below a certain SNR threshold, transformer-based models do not produce [inaudible] markers - they generate plausible-looking but entirely fabricated text. Whisper is specifically documented to hallucinate repetitive phrases or unrelated sentences when processing low-SNR segments. A transcript that looks complete may contain invented content.
Common culprits in professional recordings include HVAC systems, room echo (hard surfaces reflect and smear the speech signal), Bluetooth audio compression, and VoIP codec artifacts from tools like Zoom or Teams - especially under network congestion.
2. Overlapping Speech Breaks Speaker Attribution Entirely
Focus groups are the stress test that exposes every weakness in an AI transcription pipeline simultaneously. When participants talk over each other - which happens constantly in group discussion - the diarization system (the component responsible for 'who said what') faces an impossible task.
Speaker diarization works by embedding short audio segments into a vector space and clustering them by speaker identity. This works tolerably for two people taking clear turns. It fails badly when:
• Three or more speakers are present
• Participants interrupt or respond simultaneously
• Speakers vary significantly in volume or distance from the microphone
• Background noise distorts the speaker embeddings
During overlap, the model typically picks the loudest speaker and treats the others as noise. Quieter or more distant participants - often including introverted group members whose contributions may be analytically important - are systematically underrepresented or lost entirely.
📊 Data point: Published research on the DIHARD diarization benchmarks shows Diarization Error Rate (DER) climbing from under 5% in clean two-speaker audio to 20–40%+ in multi-speaker spontaneous conversation with background noise. In qualitative research contexts, that means you often cannot reliably determine who said what - even if the words themselves were transcribed correctly.
3. Accents and Spontaneous Speech Expose Training Data Bias
Every ASR model's performance ceiling is set by its training data distribution. For English-language models, that distribution skews heavily toward American English, prepared speech, and studio recording conditions. The practical consequences are well-documented:
• Non-native English speakers see WER increases of 30–80% relative to native speakers, depending on accent strength
• Regional and minority language varieties (AAE, Scottish English, Irish English) show consistent performance gaps across all major commercial systems
• For Dutch-language transcription - including Flemish dialects and Belgian Dutch with French code-switching - most models are trained on Standaardnederlands and perform significantly worse on regional speech
Spontaneous conversational speech adds another layer of difficulty: filled pauses, false starts, reduced phonetic forms ('gonna', 'kinda'), and emotional prosody are systematically underrepresented in training corpora. These are not edge cases - they are the normal texture of natural human conversation.
4. Post-Correction Is More Expensive Than It Looks
A common response to AI transcription errors is 'just have someone fix it afterwards.' This underestimates the cognitive cost of error correction. Fixing a transcript requires the reviewer to simultaneously monitor the audio, read the incorrect text, identify discrepancies, and retype corrections. Research in cognitive ergonomics suggests that correcting a 15% WER transcript takes roughly 60–70% as long as transcribing from scratch.
For use cases where accuracy genuinely matters - qualitative research data, legal or compliance documentation, HR investigations, medical records - the efficiency case for AI-only transcription weakens considerably once post-correction time is included.
The Practical Takeaway
AI transcription is fast and cost-effective for clean, single-speaker recordings in standard conditions. It is the wrong tool - or at minimum, an insufficient tool without substantial human review - for:
• Focus groups and multi-speaker discussions
• Field recordings or interviews in non-controlled environments
• Participants with strong accents or non-standard speech patterns
• Any recording made over VoIP or with consumer-grade equipment
• Documents where attribution, precision, or legal weight matters
For these scenarios, human-led or hybrid transcription workflows remain the reliable standard. Specialist services like Outspoken.be are specifically built for the difficult cases - focus groups, noisy field interviews, dialect-heavy recordings, and multi-speaker meetings - where AI output alone consistently falls short. The acoustic physics of real-world audio have not changed; what matters is choosing a workflow that accounts for them.
This article is a condensed version of a longer technical deep-dive covering WER measurement, diarization architectures, SNR physics, and codec artifacts. The full version is available at outspoken.be.
Top comments (0)