roberttomko

Posted on Aug 1

Building a podcast summarizer in 20 lines of Python

#webdev #python #llm #rag

A few months ago I tried to build a "podcast summarizer for my reading list." The pitch sounded trivial: take an episode URL, return a five-bullet summary attributed to the right speakers. A weekend project.

It wasn't.

The pipeline you don't want to build

If you've gone down this road, the steps are familiar:

Locate the audio file. Podcasts aren't on YouTube alone — RSS feeds, Spotify-locked content, Apple-exclusive series. You end up writing a per-platform resolver.
Download and normalize. 50–100 MB per episode. Different sample rates, different containers.
Transcribe. Whisper API ($0.006/min) or self-host whisper.cpp. Either way, you're now managing audio jobs.
Diarize. Pyannote, or pay for a transcription service that bundles it. The output is "Speaker 0", "Speaker 1" — not useful.
Identify speakers. A separate LLM pass over the transcript with a "given these dialogue cues, who is each speaker?" prompt. Wrong about 10% of the time.
Align timing. Whisper segments and Pyannote turns aren't perfectly aligned. Edge cases everywhere.
Cache. Or you re-transcribe the same episode every time you tweak the prompt.

That's the bones of every "podcast RAG cookbook" I've read from Haystack, LangChain, and others. It's also why nobody actually ships a working podcast summarizer as a side project — by the time the plumbing is done, you've lost interest.

What I actually wanted

transcript = get_podcast("huberman sleep")
summary = llm.summarize(transcript)

Two lines. The plumbing is the wrong layer for a side project.

What's actually online

Here's the thing: most popular podcasts publish transcripts. Apple's podcast app shows them; YouTube transcribes them; the shows themselves often have them on their websites. The audio-transcription pipeline I was building was re-doing work that had already been done by the show or the platform.

So I built a thin retrieval API over published transcripts, added LLM-driven speaker name detection on top (so the output has "Andrew Huberman" instead of "Speaker 0"), and started using it.

It's at spoken.md. There's a free demo key pt_demo so you don't have to sign up to try it.

The 20-line summarizer

import requests
from anthropic import Anthropic

client = Anthropic()

def summarize(query):
    # Find episode
    results = requests.get(
        f"https://spoken.md/search?q={query}",
        headers={"x-api-key": "pt_demo"},
    ).json()["results"]
    if not results: return "no match"
    ep = results[0]

    # Fetch transcript (Markdown, with real speaker names)
    transcript = requests.get(
        f"https://spoken.md/transcripts/{ep['id']}",
        headers={"x-api-key": "pt_demo"},
    ).text

    # Summarize
    msg = client.messages.create(
        model="claude-haiku-4-5-20251001",
        max_tokens=1024,
        messages=[{"role": "user", "content":
            f"Summarize in 5 bullets, attributing to speakers:\n\n{transcript}"}],
    )
    return f"# {ep['title']}\n\n{msg.content[0].text}"

print(summarize("acquired nvidia"))

A one-hour podcast is 8,000–15,000 tokens — fits comfortably in Claude Haiku's context window with room for the summary prompt.

RAG over podcasts is the same pattern

For a back-catalogue indexer, the only difference is the splitter:

from langchain_text_splitters import MarkdownTextSplitter
splitter = MarkdownTextSplitter(chunk_size=1200, chunk_overlap=180)

# Real speaker names survive chunking
for episode_id in episode_ids:
    md = get_transcript(episode_id)
    chunks = splitter.split_text(md)
    # embed and store as usual

Because each speaker turn starts with **Name** (0:45), the speaker attribution lands in every chunk by default — no extra metadata layer.

What this isn't

This is not a Whisper replacement. If you have your own audio (meetings, calls, recordings), you still need a speech-to-text service. Deepgram, AssemblyAI, Whisper — those are the right tools. The point is that transcribing already-published podcasts from scratch is reinventing the wheel.

Cost

For published podcasts at moderate volume:

Whisper + Pyannote DIY: ~$0.36/hr Whisper + diarization compute + naming-LLM pass = real cost $0.50–$1.00+ per episode
AssemblyAI Universal + naming pass: ~$0.20–$0.30 per episode all-in
spoken.md: $0.08–$0.15 flat, names included

It's not a 2x difference, it's roughly 5–10x once you account for everything the all-in pipeline does that a simple fetch doesn't.

Try it

curl -H "x-api-key: pt_demo" https://spoken.md/transcripts/1000651996090

That's the demo episode. The format is Markdown with bold speaker names and per-turn timestamps. If it'd be useful for what you're building, the docs are at spoken.md.

I built spoken.md and am posting this myself — happy to answer questions about the API or the tradeoffs above in the comments.

DEV Community