DEV Community

TranscriptFetch
TranscriptFetch

Posted on Originally published at transcriptfetch.com

youtube_transcript_api works on my laptop but fails on AWS: a checklist

#python #youtube #webdev #devops

If youtube_transcript_api runs fine locally and starts failing the moment you deploy, it is tempting to blame IP blocking straight away. Often that is the answer, but not always. Here is the order I check things in, cheapest first.

1. Rule out the video itself

Before touching infrastructure, run the exact failing video ID locally.

  • Captions disabled by the uploader raise TranscriptsDisabled.
  • Captions that exist only in another language raise NoTranscriptFound. List what is there first: 
from youtube_transcript_api import YouTubeTranscriptApi

ytt = YouTubeTranscriptApi()
for t in ytt.list("VIDEO_ID"):
    print(t.language_code, t.is_generated)
Enter fullscreen mode Exit fullscreen mode
  • Private, deleted, age restricted or region locked videos fail everywhere, not only in production.

If the video fails on your laptop too, it is not your server.

2. Same video, two networks

Run the identical request, same library version, from your laptop and from the production host. Local works and the server fails on the same ID? The request origin is the difference.

3. Read the exception, not just the stack trace

  • RequestBlocked or IpBlocked: YouTube is refusing your server's IP. Cloud provider ranges (AWS, GCP, Azure, most VPS hosts) are the usual trigger, and the library's own README says so.
  • HTTP 429: rate limiting. Concurrency that was fine locally may not be fine from one egress IP.
  • Timeouts and empty results that get worse with volume: also throttling, just quieter.

4. Check what changed

  • Did you upgrade the library? The API moved from static get_transcript calls to an instance with fetch and list in 1.x, so old snippets break.
  • Did your platform start rotating or sharing egress IPs (serverless, NAT gateways)?
  • Did you add workers or async fan out?

5. Pick a fix

If it really is the IP, there are three honest options:

  1. Stay local or residential. For a script you run now and then, run it from a home connection and move on.
  2. Proxies. The library supports them directly: 
from youtube_transcript_api import YouTubeTranscriptApi
from youtube_transcript_api.proxies import GenericProxyConfig

ytt = YouTubeTranscriptApi(
    proxy_config=GenericProxyConfig(
        http_url="http://user:pass@proxy.example:8080",
        https_url="http://user:pass@proxy.example:8080",
    )
)
print(ytt.fetch("VIDEO_ID").to_raw_data()[:3])
Enter fullscreen mode Exit fullscreen mode

Residential proxies work far better than datacenter ones here. The cost is per GB, and you own rotation, retries and the monitoring for silent failures.

  1. A hosted transcript API. Someone else runs the proxy layer and you call an HTTP endpoint. Disclosure: I build one of these, TranscriptFetch, so weigh that. The trade is a per request price against your own time on proxy upkeep.

The short version

Situation Fit
Occasional local scripts Keep the library, run it locally
You already run a proxy fleet Library plus proxies
A production app, no wish to run proxies A hosted API

The longer write up, with the full error list and code for the hosted route, is on our blog.

If you have hit a failure mode that is not on this list, I would like to hear it in the comments.

Top comments (0)