DEV Community

hua sun
hua sun

Posted on

When “No Speech Found” Actually Means “Wrong Language”

A user uploads a video with clearly audible speech.
The transcription API returns nothing.
The obvious error message is:

No speech found.
But that message may be wrong. The audio is not silent—the transcription model may simply be listening for the wrong language.
We ran into this while building SubtitleGenerator, and it led to an interesting product decision: should we keep adding automatic detection, or just ask the user what language they are speaking?

How the problem happens

A speech-to-text request usually contains three important pieces of information:

  1. The audio
  2. The transcription model
  3. The expected language A simplified request might look like this:

ts
const formData = new FormData();
formData.append("audio", audioBlob);
formData.append("language", "en");
await fetch("/api/transcribe", {
  method: "POST",
  body: formData,
});
This works when the speaker is using English.

But if the video contains Chinese, Spanish or Arabic speech, forcing en can produce:

An empty transcript
Very low-confidence words
English-looking nonsense
Text written in an unexpected alphabet
A misleading “no speech found” error
The audio exists. The model just cannot map it confidently to the expected language.

Why not always use automatic detection?
The obvious solution is automatic language detection.

In ideal conditions, that works well. The problem appears with real creator content:

Five-second clips
Music before the first spoken sentence
Strong accents
Noisy phone recordings
Multiple speakers
Mixed-language speech
Hindi and English code-switching
One or two words surrounded by silence
Language detection needs evidence. A short clip may not provide enough of it.

This creates a dangerous failure mode: the application looks fully automatic, but when detection fails, the user has no idea why.

The fallback we considered
We initially considered a pipeline like this:

Run normal transcription
        ↓
Result is empty or uncertain
        ↓
Send audio to another model for language detection
        ↓
Run transcription again with detected language
It sounds smart, but it has hidden costs.

If a ten-minute video is processed twice, the application may pay for twenty minutes of model usage. The user also waits for two requests while believing they submitted only one job.

Worse, the second model can still guess incorrectly.

Adding another AI call does not automatically make a workflow more reliable. Sometimes it just makes the failure slower and more expensive.

The decision: ask before processing
We chose a simpler workflow:

Show a Spoken language selector before upload
Default to English for our current audience
Send the selected language explicitly
Remember the choice locally
Ask the user to check it when the result looks suspicious
The client sends the language with the audio:

const formData = new FormData();
formData.append("audio", audioBlob);
formData.append("language", spokenLanguage);
const response = await fetch("/api/transcribe", {
  method: "POST",
  body: formData,
});
The server validates the value and applies a predictable fallback:

const supportedLanguages = new Set([
  "en",
  "es",
  "fr",
  "de",
  "hi",
  "ar",
  "zh",
]);
const requestedLanguage = formData.get("language");
const language =
  typeof requestedLanguage === "string" &&
  supportedLanguages.has(requestedLanguage)
    ? requestedLanguage
    : "en";
The selected value is then passed to the transcription provider.

This approach is not as magical as “we detect everything automatically,” but its behavior is easier to explain and debug.

Detecting a suspicious result
Users can still choose the wrong language, so we need a recovery path.

After transcription, we inspect the returned words:

type TranscriptWord = {
  text: string;
  confidence?: number;
};
function needsLanguageReview(words: TranscriptWord[]): boolean {
  if (words.length === 0) {
    return true;
  }
  const wordsWithConfidence = words.filter(
    (word) => typeof word.confidence === "number",
  );
  if (wordsWithConfidence.length === 0) {
    return false;
  }
  return wordsWithConfidence.every(
    (word) => (word.confidence ?? 1) < 0.75,
  );
}
An empty result or consistently low confidence does not prove that the language is wrong. The recording might simply be poor.

So we do not display:

Wrong language detected.

Instead, we say something closer to:

The result looks uncertain. Check the spoken language and try again.

That distinction matters. Confidence is a signal, not proof.

Spoken language is not translation language
This also forced us to make the interface clearer.

Consider a Spanish video that needs English subtitles:

Spoken language: Spanish
Translation target: English
The correct pipeline is:

Spanish audio
    ↓ Transcription
Spanish transcript
    ↓ Translation
English subtitles
Trying to transcribe Spanish audio as English is not translation. It is a transcription request with incorrect context.

Keeping these two settings separate prevents a surprising number of errors.

Make the retry cost visible
Changing the language and running transcription again means making another API request.

That may consume additional processing time, so the interface should say so before the retry.

The alternative—silently retrying until something works—creates unpredictable costs for both the user and the application.

Our rule became:

Never hide a second paid model call behind an innocent-looking loading state.

If the retry costs time or credits, tell the user.

What about mixed-language videos?
A single language selector does not solve everything.

A speaker may switch between Hindi and English, Spanish and English, or Arabic and French. Some transcription providers support multilingual modes, but support and accuracy vary by model.

A practical language configuration might therefore look like this:

type LanguageMode =
  | { type: "single"; language: string }
  | { type: "multilingual" };
The important part is not promising multilingual accuracy unless the selected provider and model genuinely support it.

For unsupported combinations, a clear limitation is better than a confident-looking bad transcript.

What we learned
1. Ask users for information they already know
The speaker usually knows which language is in the video. Asking takes seconds and can prevent an expensive failed request.

2. Automatic does not always mean simpler
Automatic detection removes one control from the interface, but it can add retries, cost and confusing failure states behind the scenes.

3. Error messages should describe evidence
“No words were returned” is evidence.

“The video contains no speech” is an interpretation—and it may be wrong.

4. Confidence is not correctness
A low-confidence word needs review. It should not automatically be presented as an error.

5. AI costs are part of UX design
Fallback calls, retries and detection requests affect both the bill and the waiting experience. They are product decisions, not merely backend details.

Final thought
Developers often try to make AI products feel magical by removing every question.

But when the user already knows something the model must guess, one small question can make the product faster, cheaper and more trustworthy.

Sometimes the best AI fallback is not another model.

It is a dropdown.

I'm building [SubtitleGenerator](https://www.subtitlegenerator.net/), an AI subtitle editor that highlights uncertain words so creators can review what matters instead of checking every line.
Enter fullscreen mode Exit fullscreen mode

Top comments (0)