YouTube transcripts for RAG: fetch, chunk and embed captions in 30 lines of Python

Disclosure: I built the Apify Actor used in this tutorial. This post was written with AI assistance.

A talk, a lecture series or a product channel is a knowledge base that happens to be stored as video. To search it or feed it to a language model you need the captions as text, split into pieces, each piece turned into a vector. This tutorial does that with three pieces: an Apify Actor that returns timestamped transcripts, a small chunker that keeps the start time of every chunk so results link back to the exact second in the video, and a local open-source embedding model so nothing except the transcript download leaves your machine.

What you need:

• Python 3.11 or newer
• An Apify API token
• A few hundred megabytes of disk for the embedding model

The Actor is YouTube Transcript Scraper. It costs $0.001 per transcript delivered, so $1.00 per 1,000 videos. Videos with no captions, private or unavailable videos, bad links, and the listing of a playlist or channel are all free: you pay only for transcripts that arrive in your dataset. The free Apify plan's monthly usage credit covers the 20-video example below many times over.

pip install apify-client sentence-transformers numpy
export APIFY_TOKEN=apify_api_xxxxxxxx

pip install sentence-transformers downloads PyTorch if you do not have it, which is the bulk of that disk space.

1. What a transcript record looks like

The Actor's only required input is videos: a list of video links, youtu.be links, Shorts, playlist links, channel links (@handle or /channel/…) or bare video IDs, mixed freely. For a playlist or channel, maxVideosPerSource (default 50, up to 1,000) says how many of its videos to take; a channel yields its newest uploads first. language (default en) picks the caption track, preferring a human-written track over an auto-generated one in that language.

Each video comes back as one record:

{
  "videoId": "aircAruvnKk",
  "url": "https://www.youtube.com/watch?v=aircAruvnKk",
  "title": "But what is a neural network? | Deep learning chapter 1",
  "channel": "3Blue1Brown",
  "status": "OK",
  "language": "en",
  "isAutoGenerated": false,
  "languageMatched": true,
  "lengthSeconds": 1132,
  "segmentCount": 286,
  "segments": [
    { "start": 4.22, "duration": 1.18, "text": "This is a 3." },
    { "start": 6.06, "duration": 4.653, "text": "It's sloppily written and rendered at an extremely low resolution of 28x28 pixels," }
  ],
  "text": "This is a 3. It's sloppily written and rendered ..."
}

Three fields matter for RAG:

• segments: every caption line with start and duration in seconds. This is what lets a search result point at a timestamp.
• text: the whole transcript joined into one string, for when you do not need timestamps.
• status: OK, NO_CAPTIONS or FAILED, with error explaining the last two. Nothing comes back silently empty, and the two failure cases cost nothing.

isAutoGenerated: true means YouTube's automatic captions: fine for search, but expect missing punctuation and the occasional misheard term. languageMatched: false means the video had no track in the language you asked for and you got its best available one instead; availableLanguages lists what the video has.

2. Fetch, chunk, embed

Here is the whole pipeline. Save it as index.py.

import os, json
import numpy as np
from apify_client import ApifyClient
from sentence_transformers import SentenceTransformer
client = ApifyClient(os.environ["APIFY_TOKEN"])
run = client.actor("highbrow_fame/youtube-transcript-fast").call(run_input={
    "videos": ["https://www.youtube.com/@3blue1brown"],
    "language": "en",
    "maxVideosPerSource": 20,
})
WINDOW = 60  # seconds of speech per chunk
chunks = []
for v in client.dataset(run.default_dataset_id).iterate_items():
    if v.get("status") != "OK":
        print("skipped", v.get("url"), v.get("status"), v.get("error"))
        continue
    buf, start = [], None
    for seg in v["segments"]:
        start = seg["start"] if start is None else start
        buf.append(seg["text"])
        if seg["start"] + seg["duration"] - start >= WINDOW:
            chunks.append({"video": v["videoId"], "title": v["title"], "start": int(start), "text": " ".join(buf)})
            buf, start = [], None
    if buf:
        chunks.append({"video": v["videoId"], "title": v["title"], "start": int(start), "text": " ".join(buf)})
model = SentenceTransformer("all-MiniLM-L6-v2")
vectors = model.encode([c["text"] for c in chunks], normalize_embeddings=True, show_progress_bar=True)
np.save("chunks.npy", vectors)
json.dump(chunks, open("chunks.json", "w"))
print(len(chunks), "chunks from", len({c["video"] for c in chunks}), "videos")

That is the 30 lines. What each part does:

Fetch. call() starts the Actor run and waits for it. The Actor page reports fifteen videos processed in nine seconds in its own test run. Twenty videos cost at most $0.02 (fewer if some videos have no captions).

Chunk by time, not by characters. The chunker accumulates caption lines until the window covers 60 seconds of video, then emits a chunk carrying the start of its first line. Sixty seconds of speech is roughly 120 to 180 words, a comfortable size for a small embedding model and a readable size for a human looking at a result. The last partial window of each video is kept too. Time-based chunking has a property character-based chunking lacks: every chunk maps to a deep link, https://www.youtube.com/watch?v=<id>&t=<start>s.

Embed locally. all-MiniLM-L6-v2 is a 384-dimension sentence-transformers model, under 100 MB, fast on a CPU, and good enough for "find the part of the talk about X". normalize_embeddings=True makes every vector unit length, so cosine similarity is a plain dot product later. If you want better retrieval quality and can spare the time, swap in BAAI/bge-small-en-v1.5 or intfloat/e5-small-v2; the code does not change (the e5 family expects a query: / passage: prefix, check its model card).

Store. A NumPy array and a JSON file are enough for a few thousand chunks. Replace the two saves with a call to Chroma, FAISS, pgvector or whatever you already run once you outgrow it.

3. Search it

import sys, json
import numpy as np
from sentence_transformers import SentenceTransformer

chunks = json.load(open("chunks.json"))
vectors = np.load("chunks.npy")
model = SentenceTransformer("all-MiniLM-L6-v2")

q = model.encode([sys.argv[1]], normalize_embeddings=True)[0]
scores = vectors @ q
for i in np.argsort(-scores)[:5]:
    c = chunks[i]
    print(f"{scores[i]:.2f}  {c['title']}")
    print(f"      https://www.youtube.com/watch?v={c['video']}&t={c['start']}s")
    print(f"      {c['text'][:160]}...\n")

python search.py "how does backpropagation update the weights"

Each hit prints its similarity score, the video title, a link that opens the video at the right second, and the first line of the chunk. That is already a usable "search inside this channel" tool.

4. Handing the chunks to a language model

Retrieval-augmented generation is this search plus a prompt. Take the top few chunks, put them in the prompt with their source, and ask the model to answer only from them:

top = [chunks[i] for i in np.argsort(-scores)[:5]]
context = "\n\n".join(
    f"[{c['title']} @ {c['start']}s] {c['text']}" for c in top
)
prompt = (
    "Answer the question using only the transcript excerpts below. "
    "Cite the video title and timestamp you used. If the excerpts do not "
    "contain the answer, say so.\n\n"
    f"{context}\n\nQuestion: {sys.argv[1]}"
)

Send prompt to whichever model and SDK you use. Because every chunk carries a title and a start time, the model can cite [title @ 412s] and you can turn that back into a link.

Choices you may want to make differently

• Window size and overlap. 60 seconds with no overlap is simple. If answers often straddle a boundary, add a 10-second overlap by starting the next chunk from the last few segments of the previous one, or shrink the window to 30 seconds and retrieve more hits.
• Use text instead of segments when you do not care about timestamps; then any text splitter will do.
• Re-indexing. Keep the run's dataset ID. Transcripts of published videos rarely change, so for a growing channel fetch only new videos (a channel link with a small maxVideosPerSource returns the newest uploads) and append to your store instead of re-embedding everything.
• Subtitles for a player. The Actor can add srt and vtt fields (includeSrt, includeVtt) if you also want to show captions alongside the search result.

Limits

• Videos without captions return NO_CAPTIONS and are free, but they are also absent from your index. There is no speech-to-text step here; add one yourself if you need those videos.
• Machine translation to your language (translate: true) is best effort and YouTube restricts it heavily; in my own test it came through for 1 of 8 videos. The original transcript is returned either way.
• Up to 1,000 videos per playlist or channel per run.
• Transcripts are the creator's content. Indexing them for your own search or research is a different thing from republishing them; respect YouTube's terms and the creators' rights.

The complete input schema, output fields and pricing are on the Actor page.

Comments

[AgentSearch]

Keeping the start time on every chunk is the detail that makes this usable, since a result that opens the video at the right second gets clicked. One thing I'd tweak: a fixed 60-second window often cuts mid-sentence, and auto-generated captions have almost no punctuation to fix that afterwards. Closing a chunk at the first gap between segments after the window (where the speaker pauses) and carrying the last segment over into the next chunk keeps each piece readable on its own, and retrieval gets noticeably less choppy.