<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: Aquila</title>
    <description>The latest articles on DEV Community by Aquila (@massarico182).</description>
    <link>https://dev.to/massarico182</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F4189208%2F46e181a6-3035-414a-a8a4-1027309e4784.png</url>
      <title>DEV Community: Aquila</title>
      <link>https://dev.to/massarico182</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/massarico182"/>
    <language>en</language>
    <item>
      <title>Why your YouTube transcript scraper returns the wrong language (and how to fix it)</title>
      <dc:creator>Aquila</dc:creator>
      <pubDate>Fri, 16 Oct 2026 16:00:04 +0000</pubDate>
      <link>https://dev.to/massarico182/why-your-youtube-transcript-scraper-returns-the-wrong-language-and-how-to-fix-it-8fj</link>
      <guid>https://dev.to/massarico182/why-your-youtube-transcript-scraper-returns-the-wrong-language-and-how-to-fix-it-8fj</guid>
      <description>&lt;p&gt;I was building a YouTube transcript scraper for an AI pipeline and ran a simple test: 50 random videos (podcasts, tutorials, TED talks, news, music, Hindi speeches, a Japanese vlog).&lt;/p&gt;

&lt;p&gt;37 of them had captions. And &lt;strong&gt;9 of those 37 came back in the wrong language&lt;/strong&gt;:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;8 English videos returned captions in &lt;strong&gt;Arabic, Chinese or Albanian&lt;/strong&gt;
&lt;/li&gt;
&lt;li&gt;1 Hindi video returned the &lt;strong&gt;English dub&lt;/strong&gt; instead of the original Hindi&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;That's roughly 1 in 4 transcripts. If you're feeding transcripts to an LLM, a RAG index or a summarizer, that's a silent bug: nothing crashes, you just get garbage.&lt;/p&gt;

&lt;h2&gt;
  
  
  Why it happens
&lt;/h2&gt;

&lt;p&gt;Most scrapers (and a lot of tutorials) do something like this:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="n"&gt;tracks&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;get_caption_tracks&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;video_id&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="n"&gt;transcript&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;download&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;tracks&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;])&lt;/span&gt;  &lt;span class="c1"&gt;# first track in the list
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The problem: &lt;strong&gt;YouTube's caption list is not ordered by the original language.&lt;/strong&gt; A popular English video can have 40+ community translations, and the first one in the list can be anything.&lt;/p&gt;

&lt;p&gt;Two other traps:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;The "default" track changes with the viewer.&lt;/strong&gt; The caption YouTube marks as default depends on the language of whoever is asking, so a server in another country can get a different default than you do in your browser.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Dubbed videos.&lt;/strong&gt; Videos with multi-language audio have several audio tracks. If you pick the wrong one, you transcribe the dub, not the original.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  How to find the original language
&lt;/h2&gt;

&lt;p&gt;What worked for me, in order:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;Dubbed videos:&lt;/strong&gt; the original audio track has its own flag in the player data. Read it.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Everything else:&lt;/strong&gt; use the language of the &lt;strong&gt;auto-generated (speech recognition) track&lt;/strong&gt;. YouTube runs speech recognition in the language that is actually spoken, so that track's language is a very reliable signal.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Then pick the best captions in that language:&lt;/strong&gt; human-made captions first, auto-generated only if there are no human ones.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;If you're using the open-source &lt;code&gt;youtube-transcript-api&lt;/code&gt; library, step 2 looks roughly like this:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="n"&gt;youtube_transcript_api&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;YouTubeTranscriptApi&lt;/span&gt;

&lt;span class="n"&gt;ytt&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;YouTubeTranscriptApi&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
&lt;span class="n"&gt;tracks&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;ytt&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;list&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;VIDEO_ID&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

&lt;span class="c1"&gt;# The auto-generated track is in the spoken language
&lt;/span&gt;&lt;span class="n"&gt;asr&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="n"&gt;t&lt;/span&gt; &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;t&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="n"&gt;tracks&lt;/span&gt; &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;t&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;is_generated&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;
&lt;span class="n"&gt;original&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;asr&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;].&lt;/span&gt;&lt;span class="n"&gt;language_code&lt;/span&gt; &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;asr&lt;/span&gt; &lt;span class="k"&gt;else&lt;/span&gt; &lt;span class="bp"&gt;None&lt;/span&gt;

&lt;span class="c1"&gt;# Prefer human captions in that language, fall back to auto-generated
&lt;/span&gt;&lt;span class="k"&gt;try&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="n"&gt;transcript&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;tracks&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;find_manually_created_transcript&lt;/span&gt;&lt;span class="p"&gt;([&lt;/span&gt;&lt;span class="n"&gt;original&lt;/span&gt;&lt;span class="p"&gt;])&lt;/span&gt;
&lt;span class="k"&gt;except&lt;/span&gt; &lt;span class="nb"&gt;Exception&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="n"&gt;transcript&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;tracks&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;find_generated_transcript&lt;/span&gt;&lt;span class="p"&gt;([&lt;/span&gt;&lt;span class="n"&gt;original&lt;/span&gt;&lt;span class="p"&gt;])&lt;/span&gt;

&lt;span class="n"&gt;text&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt; &lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;join&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;s&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;text&lt;/span&gt; &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;s&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="n"&gt;transcript&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;fetch&lt;/span&gt;&lt;span class="p"&gt;())&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;One more thing that will bite you in production: &lt;strong&gt;YouTube blocks most cloud IPs.&lt;/strong&gt; In my test, requests from datacenter proxies succeeded 2% of the time; residential proxies, 100%. Code that works on your laptop can fail on a server with "Sign in to confirm you're not a bot".&lt;/p&gt;

&lt;h2&gt;
  
  
  If you don't want to maintain this yourself
&lt;/h2&gt;

&lt;p&gt;I packaged all of the above into an Apify Actor: &lt;a href="https://apify.com/massa182/youtube-transcript" rel="noopener noreferrer"&gt;YouTube Transcript API&lt;/a&gt;.&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Original spoken language by default (or any language you ask for)&lt;/li&gt;
&lt;li&gt;Input: video URL, playlist, channel or search term&lt;/li&gt;
&lt;li&gt;Output: plain text, timestamped text, SRT and Markdown, plus title, channel, duration and language&lt;/li&gt;
&lt;li&gt;Residential proxies and retries included&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;$3 per 1,000 transcripts, charged only when a transcript is delivered.&lt;/strong&gt; Videos without captions are free.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;From Python:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="n"&gt;apify_client&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;ApifyClient&lt;/span&gt;

&lt;span class="n"&gt;client&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;ApifyClient&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;&amp;lt;YOUR_APIFY_TOKEN&amp;gt;&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="n"&gt;run&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;client&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;actor&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;massa182/youtube-transcript&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;).&lt;/span&gt;&lt;span class="nf"&gt;call&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
    &lt;span class="n"&gt;run_input&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;sources&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;https://www.youtube.com/@TED&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;maxVideosPerSource&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;10&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;
&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;item&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="n"&gt;client&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;dataset&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;run&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;defaultDatasetId&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;]).&lt;/span&gt;&lt;span class="nf"&gt;iterate_items&lt;/span&gt;&lt;span class="p"&gt;():&lt;/span&gt;
    &lt;span class="nf"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;item&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;title&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt; &lt;span class="n"&gt;item&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;language&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt; &lt;span class="n"&gt;item&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;wordCount&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;])&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;It also works from n8n, Make, Zapier and as an MCP tool for AI agents.&lt;/p&gt;




&lt;p&gt;&lt;strong&gt;Have you run into the wrong-language problem?&lt;/strong&gt; I'm curious how others are handling dubbed videos and channels with lots of translations. Let me know in the comments.&lt;/p&gt;

</description>
      <category>youtube</category>
      <category>python</category>
      <category>ai</category>
      <category>webscraping</category>
    </item>
  </channel>
</rss>
