Two-host AI dialogue specs: how I structure YouTube longform scripts with A/B speaker JSON

The video pipeline I've been building for two YouTube channels running off this monorepo started with short-form vertical clips — a single narrator, a single slide, done. Longform is different. A ten-minute explainer with one voice and no conversational variation is hard to watch even when the content is good. I wanted something that felt like two people talking through a problem, not a text-to-speech audiobook.

The solution was a two-host dialogue spec: a JSON file where each line of audio is tagged with a speaker (A or B), and the build script renders it as a full-length video alternating between two neural voices.

What the spec looks like

The simplest possible spec:

{
  "title": "How Turso libSQL compares to Cloudflare D1",
  "description": "...",
  "tags": ["database", "cloudflare", "turso"],
  "privacy": "public",
  "segments": [
    {
      "speaker": "A",
      "text": "Turso and D1 look similar from the outside — both are SQLite-compatible edge databases.",
      "slide": { "kind": "title", "title": "Turso vs D1" }
    },
    {
      "speaker": "B",
      "text": "Right. But they differ significantly on branching, replication topology, and cost model once you scale past the free tier."
    },
    {
      "speaker": "A",
      "text": "Let's go through each. First, branching.",
      "slide": { "kind": "bullets", "heading": "Branching", "items": ["Turso: git-style branches", "D1: no branching"] }
    }
  ]
}

A few things to notice. The slide field is optional — when omitted, the build script holds the previous slide while the audio plays. This means you don't need a new visual for every sentence, which would be exhausting to maintain and would produce a choppy video. A new slide appears only when there's something worth showing.

The speaker field maps to a voice. In build_longform.py:

VOICE = {
    "A": os.environ.get("LF_VOICE_A", "en-US-GuyNeural"),
    "B": os.environ.get("LF_VOICE_B", "en-US-AvaNeural"),
}

Both are edge-tts neural voices — a Python wrapper around Microsoft Edge's text-to-speech API that gives access to the same neural voices as the browser without requiring an Azure subscription. The A/B assignment came from testing: one lower/measured voice for exposition, one that sounds more conversational for follow-up and counterpoint. You can override both with environment variables, which matters if the default voices aren't available in a given edge-tts catalog version.

How the build works

build_longform.py processes the spec linearly:

1. For each segment with a slide, render the slide to PNG via slides.py
2. Synthesize the segment's text with edge-tts for the assigned speaker voice, writing to an mp3
3. Build a silent video clip from the PNG, then mux it with the audio
4. After all segments: concatenate all clips with ffmpeg

The result is a single output.mp4 where each visual change happens exactly when a new slide is specified in the spec — usually at section transitions, not on every sentence.

If a segment has no slide key, the previous slide's PNG is reused. The timing automatically matches the audio duration because each clip is built from its own audio file. No manual timestamp editing.

The build script prints the output path on its last line of stdout:

YT_OUTPUT_PATH=/tmp/lf/output.mp4

The CI step tees the build log, greps that line out of it, and writes the path to $GITHUB_OUTPUT as mp4. The downstream YouTube publish step reads that step output and uploads. Same pattern as the short-form video pipeline I wrote about.

What the spec generator produces

The specs aren't written by hand. A Claude call takes a topic and an outline and produces the full segment list, deciding where slides should appear, which speaker handles which part of the argument, and what heading text goes on each slide.

The prompt instructs the model to split responsibilities clearly: speaker A leads and introduces, speaker B challenges, adds nuance, or extends with examples. This produces a conversational dynamic that's more engaging than a single narrator even though neither voice is a real person.

One thing that took adjustment: Claude tends to generate very even A/B splits — roughly alternating every sentence. Real dialogue isn't that regular. I added an instruction to vary the run lengths: sometimes A speaks three sentences before B responds, sometimes B only adds a single sentence. That small change makes the output feel less mechanical.

What I haven't solved yet

The PNGtuber-style character art is asset-gated behind the build script's _host_assets() function: it returns None if either host is missing a closed-mouth PNG in scripts/yt-longform/assets/hosts/, and in that case the video falls back to slides-only. Both hosts do have art now — a closed and an open frame each, plus a few emotion frames — so both are on screen for every clip, with the speaker showing the open-mouth or emotion frame and the listener the closed one. What's still missing is actual animation: the overlay is statically placed, there's no idle motion, and expression changes only happen when a segment names an emotion.

The slide renderer (slides.py) registers eight layouts: title cards, bullet decks, comparison tables, tool cards, outros, Mermaid diagrams, matplotlib charts, and Openverse images. The Mermaid + matplotlib diagram pipeline I wrote about earlier this week lives in this same longform renderer (slides.py plus visuals.py) — it wasn't ported over from the short-form pipeline, which has no diagram or chart renderer at all. What I still don't have is a code-block layout with syntax highlighting, which would need either real font work in Pillow or a headless browser — I'm deferring that.

The two-voice format is working for the content I'm producing. Whether it affects watch time versus a single-voice format — I don't have enough data yet to say anything reliable. I'll publish numbers once there are 30+ videos in the channel.

---

Part of an ongoing 6-month experiment running three AI-curated directory sites. The technical claims here are real; this article was AI-assisted.