DEV Community

Nikita Iakovlev
Nikita Iakovlev

Posted on

Why Your YouTube Transcript Scraper Says 'No Captions Found' on a Video That Has Captions

#python #api #webdev #data

Spend an hour in the issue trackers of the popular YouTube transcript tools and one complaint drowns out the rest. Someone pastes a URL, gets No caption was found!, opens the video in a browser, turns captions on, and they are right there.

I went through 30 issues on the most-used transcript Actor on Apify — it carries a 3.7 rating across 49 reviews, which is what that class of bug does to a listing — and then hit every one of them myself while building a competitor. Four distinct causes, none of them obvious, all of them cheap to fix once you know what you are looking at.

1. The language code is not the language code

This is the big one.

YouTube's caption map does not use bare language codes. A video with English subtitles written by its uploader has en. A video with English subtitles contributed by a viewer, or auto-translated, has something like:

en-eEY6OEpapPo
Enter fullscreen mode Exit fullscreen mode

That suffix is a track identifier, not a locale. Despacito — 9.1 billion views, captions in dozens of languages — serves its English track under exactly that key.

So this, which is what most implementations do:

track = subtitles.get(language)   # language = "en"
Enter fullscreen mode Exit fullscreen mode

returns None on a video that plainly has English captions, and the user gets told there are none.

Match by prefix, and prefer the plainest label when several qualify:

def find_track(source: dict, wanted: str):
    keys = sorted(source.keys(), key=lambda k: (len(k), k))
    exact    = [k for k in keys if k.lower() == wanted]
    prefixed = [k for k in keys if k.lower().startswith((wanted + "-", wanted + "_"))]
    for key in exact + prefixed:
        for fmt in source.get(key) or []:
            if fmt.get("ext") == "json3" and fmt.get("url"):
                return fmt["url"], key
    return None, None
Enter fullscreen mode Exit fullscreen mode

Sorting by (len(k), k) puts en ahead of en-eEY6OEpapPo, so a hand-written track wins over a contributed one when both exist. Report the key you actually used back to the caller — the difference between a human translation and a machine one matters to whoever is reading the text.

2. "Just use the video's own language" is not reproducible

The obvious fix for "user asked for English, video is Spanish" is a fallback: take the language the video says it is in. yt-dlp exposes it as info["language"], so:

order = [info.get("language"), "en"] + everything_else   # looks right
Enter fullscreen mode Exit fullscreen mode

I shipped that, ran it twice against the same video, and got two different transcripts. First run: es, the Spanish original. Second run, same code, same video, minutes apart: the field came back empty and the fallback moved on to English.

The field is derived from whatever player response the extraction happened to get. It varies with the client YouTube answers, and therefore with your proxy exit, the time of day, and nothing you control. It is a hint, not a fact.

If your fallback is going to be documented — and it should be, because the caller is paying for rows — it has to be reproducible. Mine ended up as a fixed order with the unreliable field demoted:

order = ["en", (info.get("language") or "").split("-")[0], *manual_codes, *auto_codes]
Enter fullscreen mode Exit fullscreen mode

English first because that is what most callers want and it is a stable choice; the video's declared language second, where it helps when present and costs nothing when absent.

3. An alphabetical fallback returns German

Before I settled on that order, the fallback was "any manual track, shortest label first" — which sorts alphabetically among equals.

Run it on jNQXAC9IVRw, the first video ever uploaded to YouTube. It has community subtitles in a long list of languages. The transcript that came back:

Also hier sind wir vor den Elefanten. Das Coole an den Typen ist dass sie sehr, sehr, sehr, lange Rüssel haben.

Correct German. Completely unexpected output for an English video, and the user cannot predict it, because the reason is that de sorts before en.

Any implicit ordering becomes a product decision the moment a fallback exists. Sorting is implicit ordering.

4. Markup survives into your text

Caption tracks carry styling. Community-written ones especially:

<i>Ooh, oh, no</i>
<font color="#E5E5E5">hello</font>
Tom &amp; Jerry
Enter fullscreen mode Exit fullscreen mode

For a subtitle renderer that is formatting. In a transcript headed for summarisation, keyword search or a RAG index, it is noise that costs tokens and pollutes matching. Strip tags, decode entities.

One trap in doing it. The obvious pattern:

TAG = re.compile(r"<[^>]{1,120}>")
Enter fullscreen mode Exit fullscreen mode

eats arithmetic. A transcript containing a < b and c > d comes out as a d, because everything between the first < and the next > looked like a tag. Require a tag to open with a letter or a slash:

TAG = re.compile(r"</?[a-zA-Z][^>]{0,120}>")

def clean(text: str) -> str:
    return TAG.sub("", html.unescape(text))
Enter fullscreen mode Exit fullscreen mode

Unescape before stripping, or &lt;i&gt; survives as literal text after the real tags are gone.

The fifth one, which is not about captions at all

Half the "stopped working" reports in that tracker are not caption bugs. They are one unhandled string.

Fetch a batch of videos concurrently and YouTube will drop connections. Most transient-error lists cover the usual suspects — SSL, EOF, Connection reset, timed out — and miss this one:

Remote end closed connection without response
Enter fullscreen mode Exit fullscreen mode

It arrived as an ordinary exception, fell through the retry branch, and turned into a failed row. It cost me one video out of six on two separate batches before I read the message properly; with the string added, the same batch came back six for six.

A retry list is a list of literals someone wrote down once. It is worth re-reading it against what your logs actually contain, rather than what you assumed they would.

What to test before you ship

None of these show up on a happy-path video. The set that finds them:

  • A video whose only captions are contributed or auto-translated, so the key carries a suffix.
  • A video with captions in many languages and none declared — tests your fallback's determinism.
  • A non-English video with no English track at all.
  • A video with styled captions, and one whose transcript contains < or > as text.
  • The same video twice, minutes apart, compared byte for byte. If two runs disagree, something in your path is reading a field that is not stable.
  • A batch large enough that the host starts dropping connections.

The last two are the ones that get skipped, and they are the two that produce the reviews nobody wants.

The scraper these came out of is YouTube Transcript Scraper — transcripts with timecodes from videos, playlists and whole channels, SRT and WebVTT, and chunks that keep their timecodes for RAG. A video with no captions returns a free error row that lists the languages it does have, which is the honest answer to the complaint this whole article is about.

Top comments (0)