DEV Community

Tyr Xargaar
Tyr Xargaar

Posted on

Turn a YouTube playlist into RAG-ready chunks with timestamp links

#ai #rag #youtube #python

If you build a RAG app, chatbot or search over video content, the first problem is getting clean text with timestamps so answers can link back to the exact moment. Here is a small recipe using the YouTube Transcript Extractor Actor on Apify. It costs $0.004 per transcript, and videos without captions are not charged.

Actor: https://apify.com/xargaar_tools/youtube-transcript-extractor-batch

What comes back

For each video you get the title, channel, duration, view count, language, whether the captions are auto-generated, the full text, and optionally timestamped segments. With chunkWords set, you also get chunks: pieces of roughly that many words, each with a start time. That is the part you want for embedding.

Fetch a playlist and write chunks with deep links 

import json
from apify_client import ApifyClient

client = ApifyClient("YOUR_APIFY_TOKEN")
run = client.actor("xargaar_tools/youtube-transcript-extractor-batch").call(run_input={
    "startUrls": ["https://www.youtube.com/playlist?list=YOUR_PLAYLIST_ID"],
    "maxVideosPerUrl": 20,
    "chunkWords": 200,
    "includeAutoGenerated": True,
})

with open("chunks.jsonl", "w") as f:
    for v in client.dataset(run["defaultDatasetId"]).iterate_items():
        if v.get("status") != "OK":
            continue
        for c in v["chunks"]:
            f.write(json.dumps({
                "text": c["text"],
                "title": v["title"],
                "link": f'{v["url"]}&t={int(c["start"])}s',
            }) + "\n")
Enter fullscreen mode Exit fullscreen mode

Each line of chunks.jsonl is ready to embed, and the link field opens the video at the start of that chunk, so your app can cite the exact spot.

Things worth knowing

  • status tells you why a video has no transcript (private video, captions disabled), so you can skip or retry those without the whole run failing.
  • languages lets you set preferred caption languages, and availableLanguages shows what else exists.
  • YouTube blocks datacenter IPs, so the Actor uses a residential proxy by default. My cost in tests was about $0.0009 per video, which is why it can be priced this low.
  • In my test batch, 11 of 12 videos returned a transcript. The one that did not was reported with a clear status.

If a video you care about fails, open an issue on the Actor page with the URL.

Xargaar Tools builds small, pay-per-use data tools.

Top comments (0)