Summarise any YouTube video with an LLM: transcript chunking that keeps timestamps

Pasting a YouTube transcript into a chat window works for a five-minute video. It stops working at about the time the video gets interesting. A two-hour lecture is roughly 20,000 words, which is more than you want to stuff into one prompt, and even when it fits, a single giant prompt tends to produce a summary that favours the start and the end and skips the middle.

The other thing a plain-text transcript loses is where things were said. A summary that says "the speaker explains attention" is much less useful than one that says "attention is explained at 25:10" with a link that opens the video there. This post builds that: a small Python pipeline that fetches a transcript with timestamps, splits it into chunks that respect caption boundaries, summarises each chunk, combines the notes, and rewrites every cited time into a clickable link.

We ran the transcript and chunking parts against a real video on 23 October 2026: Andrej Karpathy's "Let's build GPT: from scratch, in code, spelled out", 6,980 seconds long. The LLM calls are the one piece we could not run for you, and we say exactly where below.

Step 1: get segments, not just text

Most "transcript to text" tools return one big string. For summarisation you want the timestamped segments, because the timestamps are what let you cite. The Siftwright YouTube Transcript API returns them when you ask for segments:

curl -s -X POST https://siftwright.com/v1/youtube-transcript \
  -H "Authorization: Bearer $SIFTWRIGHT_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"urls": ["https://www.youtube.com/watch?v=jNQXAC9IVRw"], "outputFormats": ["segments"]}'

Each result carries the video's metadata plus a segments array. These are the first three segments of the first YouTube video ever uploaded, exactly as returned:

[
  {"start": 1.2,   "duration": 2.16,  "text": "All right, so here we are, in front of the elephants"},
  {"start": 5.318, "duration": 2.656, "text": "the cool thing about these guys is that they have really..."},
  {"start": 7.974, "duration": 4.642, "text": "really really long trunks"}
]

start and duration are in seconds. Two properties of caption data matter for what follows:

• Segments are short. The 6,980-second lecture came back as 2,955 segments, about 7 words each. A segment is a caption line, not a sentence.
• Segments can overlap, or leave gaps. In the lecture, the first segment starts at 0.199 s and lasts 4.7 s, but the second starts at 2.52 s, before the first has finished. In the zoo video above, there is a gap instead: the first segment ends at 3.4 s and the next starts at 5.3 s. Auto-generated captions are drawn with a rolling window, so never assume start + duration\ equals the next start\.

If a video has no captions, the result comes back with a non-ok status and is not billed. Check status before you use segments; the code below does.

Step 2: chunk by segment, keep the time range

The naive way to chunk is to split the text every N characters. That cuts sentences in half and throws away the timestamps. Chunking by segments keeps both: you add whole segments to a chunk until it reaches a word budget, then start the next one. The start of the first segment and the end of the last give you the chunk's time range for free.

Two small details improve the result. First, a few segments of overlap between chunks, so a thought that straddles a boundary appears in both summaries instead of being cut. Second, a time marker like [7:49] inserted roughly every 30 seconds inside the chunk text. Without markers the model has no idea when anything happened; with a marker on every segment you waste tokens. Every 30 seconds is a reasonable middle.

import json, sys

def fmt(t):
    t = int(t)
    h, m, s = t // 3600, t % 3600 // 60, t % 60
    return f"{h}:{m:02d}:{s:02d}" if h else f"{m}:{s:02d}"

def chunk_segments(segments, max_words=1500, overlap_segments=5):
    """Group caption segments into chunks of ~max_words, never splitting a segment.
    Each chunk keeps its start/end time so a summary can point back to the video."""
    chunks, cur, words = [], [], 0
    for seg in segments:
        n = len(seg["text"].split())
        if cur and words + n > max_words:
            chunks.append(cur)
            cur = cur[-overlap_segments:]          # small overlap so a sentence cut at the edge appears twice
            words = sum(len(s["text"].split()) for s in cur)
        cur.append(seg)
        words += n
    if cur:
        chunks.append(cur)
    out = []
    for c in chunks:
        out.append({
            "start": c[0]["start"],
            "end": c[-1]["start"] + c[-1]["duration"],
            # one [m:ss] marker per ~30 s keeps the prompt small but lets the model cite times
            "text": render(c),
        })
    return out

def render(seg_list, every=30):
    lines, last = [], -every
    for s in seg_list:
        if s["start"] - last >= every:
            lines.append(f"\n[{fmt(s['start'])}] ")
            last = s["start"]
        lines.append(s["text"] + " ")
    return "".join(lines).strip()

We ran this on the lecture. 2,955 segments became 15 chunks of about 1,500 words each. The time ranges are what you would hope for: the first chunk covers 0:00 to 8:01, the second 7:49 to 15:52 (the overlap is the 12-second overlap between them), the third 15:38 to 24:07, and the last 1:53:07 to 1:56:21 with 598 words. The text of a chunk starts like this:

[7:49] we're going to focus on the Transformer neural network itself okay so I created a new Google collab uh jup notebook here and this will allow me to later easily share this code ...

Note the uh and the missing punctuation. That is what auto-generated captions look like. Models cope with it fine, and the summaries they write are cleaner than the input, but do not be surprised when the raw text looks rough.

Choosing the chunk size

1,500 words is a default, not a law. Smaller chunks (500 to 800 words) give more detailed notes and more calls. Larger chunks (3,000+) are cheaper but start losing the middle of each chunk. For lectures and tutorials, 1,000 to 2,000 words is a sensible range. For a fast podcast with lots of topic changes, go smaller.

Step 3: map, then reduce

This is the classic map-reduce summarisation pattern:

1. Map: ask the model for short notes on each chunk, keeping the [m:ss] markers of the moments it mentions.
2. Reduce: give the model all the notes (not the full transcript) and ask for the final summary, again citing times.

The reduce step only sees the notes, which are a small fraction of the transcript, so it fits in a single prompt even for long videos. Here is the whole pipeline, including the fetch and the time-to-link rewrite:

import json, os, re, requests
from chunker import chunk_segments, fmt   # the chunker above, saved as chunker.py

API = "https://siftwright.com/v1/youtube-transcript"

def get_video(url):
    r = requests.post(API, headers={"Authorization": f"Bearer {os.environ['SIFTWRIGHT_API_KEY']}"},
                      json={"urls": [url], "outputFormats": ["segments"]}, timeout=300)
    r.raise_for_status()
    item = r.json()["results"][0]
    if item["status"] != "ok":
        raise LookupError(item.get("error", item["status"]))
    return item

def summarize_video(url, llm):
    """llm(prompt: str) -> str is any chat-model call you already have."""
    v = get_video(url)
    chunks = chunk_segments(v["segments"])
    notes = []
    for c in chunks:                                   # map: one call per chunk
        notes.append(llm(
            "Summarize this part of a video transcript in 3-5 bullets. "
            "Keep the [m:ss] markers of the moments you mention.\n\n" + c["text"]))
    joined = "\n\n".join(f"## {fmt(c['start'])}-{fmt(c['end'])}\n{n}" for c, n in zip(chunks, notes))
    final = llm("Combine these section notes into a summary with a short overview and "
                "5-8 key points. Cite times like [12:34].\n\n" + joined)       # reduce
    # turn [h:mm:ss] / [m:ss] into links that open the video at that moment
    def link(m):
        parts = [int(p) for p in m.group(1).split(":")]
        secs = sum(p * 60 ** i for i, p in enumerate(reversed(parts)))
        return f"[{m.group(1)}](https://youtu.be/{v['videoId']}?t={secs})"
    return v["title"], re.sub(r"\[(\d+(?::\d{2}){1,2})\]", link, final)

llm is deliberately a plain function that takes a prompt string and returns a string, so you can plug in whichever model and SDK you already use.

The last step is the one readers will actually notice. The regular expression finds every [12:34] or [1:02:03] in the final summary, converts it to seconds, and rewrites it as a youtu.be/<id>?t=<seconds> link. YouTube opens the video at that second when it sees the t parameter.

What we tested, and what we did not

We ran the full pipeline against the real lecture with a stand-in llm function that returns a canned bullet quoting a timestamp from its input. That tests everything except the model: the API call, the 15 chunks, the 16 total calls (15 map plus one reduce), the largest prompt at 1,535 words, and the link rewrite. The output was:

Let's build GPT: from scratch, in code, spelled out.
16 llm calls; max prompt words 1535
- discusses a topic around [1:00:41](https://youtu.be/kCc8FmEb1nY?t=3641)

1:00:41 becomes t=3641, which is 3,600 + 41. The rewrite works for both m:ss and h:mm:ss.

We did not run it with a real language model, so we cannot show you a real summary or tell you what it costs on your provider. That is a function of the model and the length of the video: 15 map calls of about 1,500 words in, a few bullets out, plus one reduce call. You can estimate it before you start by counting the words in the chunks, which chunk_segments already gives you.

Edge cases worth handling

Very long videos. The reduce prompt grows with the number of chunks. A two-hour lecture produced 15 chunks, which is fine. A ten-hour stream would produce about 75, and 75 sets of notes may be too long for one call. The fix is another round: reduce groups of ten notes into section summaries, then reduce those. The same function works at every level.

Videos without captions. Some videos have none, and some have captions in a different language. The API accepts a language parameter and a translateTo option, and it reports isAutoGenerated and availableLanguages in the result, so you can tell the user why a video could not be summarised instead of failing silently.

Playlists and channels. The same endpoint accepts a playlist or channel URL and returns one result per video (see maxVideosPerSource). Running the loop over a playlist gives you a summary of each lecture in a course. Billing is per video that returns a transcript, so check the video count before you run a large playlist.

Prompt injection. A transcript is untrusted text. A video can literally say "ignore your instructions and..." out loud. Keep your instructions in the system prompt, treat the transcript as data, and do not give the summarising model tools that can act on anything. A summary pipeline needs none.

Copyright and terms. Summaries for your own research are one thing; republishing someone's content is another. Check the terms that apply to you before publishing summaries at scale.

Making it cheaper

• Cache by video ID. Transcripts of a published video almost never change. Store the segments and the summary keyed by videoId and a version of your prompt.
• Summarise once, ask many times. If users will ask questions about the video later, keep the chunk notes. Retrieval over 15 sets of notes with timestamps is far cheaper than re-reading the whole transcript on every question.
• Skip the map step for short videos. Under about 3,000 words, a single call with the markers inserted is simpler and usually just as good.

Summary

Timestamped segments are the key ingredient. With them, chunking keeps boundaries intact, the model can cite times, and a three-line regular expression turns those citations into links. Without them you get a block of text and a summary nobody can verify.

If you do not need code for one-off videos, the free transcript tool gives you the text, timestamped lines, SRT and VTT in the browser. For the pipeline above, the API is 10,000 results for $29 a month; our Python and Node.js guide covers the rest of the request options, and if your own server gets blocked when it talks to YouTube directly, see the RequestBlocked fix.