What Broke Automating Film Production with Claude Code and Google Flow

Razi Kallayi — Wed, 02 Sep 2026 23:59:23 +0000

What Broke Automating Film Production with Claude Code and Google Flow

I made my four-year-old and my two-year-old into superheroes. Here is every single thing that broke.

Lia is four. For about a year now she has been directing rescue missions from the top bunk — she has watched enough PAW Patrol to have absorbed its story grammar whole, and she narrates her own episodes out loud at bedtime. Airik is two. He follows her around shouting the parts he can pronounce.

One night I was doing the voices for the fourth time and thought: the tools are supposedly here now. Are they actually? Could one parent, one GPU and one month's subscription put those two children on screen, in a real film, with their own faces and a proper score and a narrator?

This is the answer. It is called Star Patrol: The Moon in the Tree. Twenty shots. Three minutes and twenty seconds. Comet can fly but is not strong; Boing is strong but cannot fly; the moon is stuck in a tree and neither of them can get it down alone.

It works. It also broke in about nine different ways, several of which I have not seen written down anywhere, and that is the useful half of this post.

One thing before you start: this is one path, not the path. At every decision below there was a cheaper route, a local route, or a route I looked at and walked away from, and I have written those in alongside what I actually did — tagged tried, tried and failed, or not tried, so you can tell the difference. The subscription in the table below bought me speed, not the film. If you have no subscription, no GPU, or a different language to work in, there is a way through and it is marked.

The line-up image, Together.jpg. This one file fixed two separate production disasters at once — and it was the third character image I generated, when it should have been the first. More on that below.

Watch it first

What it cost and how long it took


Runtime	3:20 (200.04 s), 1920×1080, 24 fps, 20 shots × 10 s
Direct cost	One month of Google AI Pro. Nothing else — and this could equally have been made for nothing, on the free plan's 50 daily Google Flow credits.
Was the subscription necessary?	No. It bought wall-clock, not capability — Flow's free tier makes the same film in about five days. See Making this with no subscription at all.
Flow credits spent	200 of the 1,000 in the monthly allowance
Marginal cost of everything else	Zero — score, voice cloning, transcription and assembly all ran locally or on free tiers
The real cost	Time. Reference-image iteration, one long debugging session on Malayalam durations, and the narration edit. Video generation was the cheap part.
Deliverables	Six finished cuts — two Malayalam, three English, one music-only. Picture and score identical across all six; only the narration track differs.

The stack, end to end: Veo 3.1 Lite via Google Flow for picture and native audio, Nano Banana (Gemini image generation) for reference art, MusicGen Medium for the score, F5-TTS and IndicF5 for voice cloning, ElevenLabs for one stock-voice cut and for the Malayalam transcription, and FFmpeg holding the whole thing together. All of it driven from Claude Code running Opus, which wrote every script named in this post — more on that below.

Everything below is what actually happened.

What was automated, and what was not

Before the story starts, here is who actually did what.

Almost every mechanical step below was performed by an agent. Not assisted by — performed. I described what I wanted in a terminal, argued with the first draft, and kept the version that survived contact with real output. The scripts in 05_TOOLS/ are the result, and I did not type them.

Stage	Who did it
Story, shot list, prompt assembly	Agent drafted, I edited and approved
Reference art direction	Me — every taste call, every re-roll kept or killed
Generating the twenty shots	Me, by hand. Flow has no API
Score generation and arrangement	Agent wrote the pipeline; I picked between three options
Voice cloning pipeline	Agent — and I threw the output away
Cutting the recordings into shot slots	Agent
Transcription and alignment	Agent
Mixing, ducking, loudness	Agent
Subtitle timing and .srt files, two languages	Agent
Six deliverable builds from one master	Agent
Drifting watermark	Agent
The narration you actually hear	Two humans, one phone

No video editor was ever opened. No Premiere Pro, no Final Cut Pro, no DaVinci Resolve, no CapCut, no InShot — no timeline at any point. The shots were generated by hand in Google Flow, and everything after that — the assembly, the mix, the subtitles, the watermark and six separate deliverables — was FFmpeg — and every one of those filter graphs was generated by an LLM in Claude Code rather than typed by me.

The interesting rows are the bold ones, and they disagree with each other on purpose.

The agent did far more than I expected — a sidechain chain I would not have found, a watermark that moves on two incommensurate sine periods, subtitle timing I never asked for. It also absorbed a day of environment damage that would otherwise have eaten a weekend.

And it could not do the two things that mattered most: it could not click through Google Flow, and it could not make a voice sound like it loved anybody.

If you only take one thing from this post, take that shape. The boring, expensive, repeatable middle of a creative pipeline is now genuinely automatable. The ends — taste at one end, feeling at the other — are still yours. There is a longer account of how this ran, and what became of it, in The pipeline had a fourth author.

Twenty shots, because the model can only count to eight

The constraint came before the story.

Veo 3.1 in Google Flow generates 8-second clips, and credits are consumed per clip regardless of what is in it. So the story had to be told in a whole number of short, self-contained beats — and if I wanted a round runtime with a bit of breathing room, each 8-second generation would sit in a 10-second slot on the timeline.

Twenty shots. 200 seconds. That is the entire structural argument.

Twenty beats turns out to be almost exactly the PAW Patrol episode skeleton, which is convenient, because it is what a four-year-old already has installed:

Beats	Function	Shots
Meet the squad	Establish the two heroes and their catchphrases	1–2
The alert	A problem arrives from outside	3–4
Roll out	Vehicles, motion, joy	5–7
The reveal	See the problem at full scale	8–10
The complication	Each hero fails alone	11–14
The idea	The two abilities combine	15
The rescue	Payoff	16–19
Home	Descend to sleep	20

The whole film as a contact sheet. Read left to right, top to bottom, and the structure above is visible without a single word of dialogue: bedroom, alert, roll-out, the tree, the failures, the light, the moon going home, bed.

Two things were written specifically for a 2- and 4-year-old audience.

Every emotional beat is physical. Comet can fly but is not strong. Boing is strong but cannot fly. That is the whole theme — nobody can do it alone — and it is expressed entirely as two children failing to reach a thing in a tree. No dialogue is required to understand it. A two-year-old gets it from the picture.

Every line of dialogue is a shout. "Comet — bright and ready!", "Star Patrol — on the go!", "SUPER SIBLINGS — GO!", "I SEE IT!", "Gotcha." Eleven dialogue lines across twenty shots, none longer than a breath. That was partly about attention span and partly a hedge against a problem I will come back to: Veo generates a fresh voice on every clip, and short loud lines vary far less perceptibly than sustained conversation.

There isn't enough tone in a two-word shout for the ear to lock onto and then notice changing.

The final shot list lives in a single production document (01_SCRIPT/star-patrol-production-pack.md) holding the prose story, the credit budget, all twenty prompts, the reference-image prompts and the working rules. Everything downstream reads from it.

Other ways to do this

Not tried — longer beats. Twenty ten-second slots is a shape the tool chose. Flow has an extend feature, and some other services generate ten seconds or more natively; either would let you write a real two-hander scene instead of eleven shouts. If your story needs conversation rather than physical comedy, don't inherit my structure — it was built around an eight-second ceiling.

Not tried — no narrator at all. One of the six cuts is music-only and it holds up. If the story is expressed physically, the narration track is a nice-to-have, and skipping it removes the entire voice-cloning half of this post from your pipeline.

Not tried — a different skeleton. PAW Patrol's grammar was chosen because my four-year-old already had it installed. A bedtime poem — twenty rhyming couplets, one per shot — would fit the same constraint and is arguably easier to narrate.

Turning two children into ingredients

The third image I made should have been the first.

That sentence is most of what this chapter cost me, and I am putting it at the top so you get it for nothing. What follows is how two real children became something a video model would accept, and the two disasters that one boring group shot fixed at a stroke.

The kids had to be recognisably themselves. That meant starting from photographs and ending at something stylised enough that a video model would accept it.

A note on the photographs. The source photos of Lia and Airik are not in this post and never will be. They are family pictures of small children and they stay off the internet. Everything you see here is the output — the stylised 3D characters — which is also the only thing that ever entered the video pipeline. That was a deliberate rule from the start, and it turned out to have a technical payoff too, discussed in the safety section below.

Four candidate photos per child. The selection criteria were unglamorous: front-facing, sharp, neutral colour temperature. One of Airik's photos had the better grin but a heavy orange cast that would have propagated into the render as skin tone, so it lost to a duller, brighter, more neutral frame. For Lia the best face was a video still, which meant cropping off the phone UI — status bar, scrubber, share strip — before uploading, at the cost of some softness.

Gemini's image generation — Nano Banana — accepts multiple reference images, and attaching two photos per child with the instruction "use both as reference for the same child" produced a visibly better likeness than either alone.

These were run in the Gemini app, deliberately, because the app's free image quota is a separate pool from Flow's video credits. Six images cost nothing.

Comet and Boing as isolated ingredients. The source photographs are withheld deliberately — these renders are the only version of these children that this project ever published, or uploaded anywhere.

Here is the prompt that produced Comet:

Using the uploaded photo(s) as reference for the same child, create a 3D animated Pixar/DreamWorks-style character of this girl. Preserve her exact facial structure, features, proportions and expression from the reference photo — the only change is a fair, light skin tone. Keep her clearly recognizable: fine dark brown-black wavy hair pulled back into a small ponytail with soft wispy curly tendrils escaping around her face and temples and a light wispy fringe across her forehead, large round bright dark brown eyes with dark lashes, softly arched dark eyebrows, a round face with full cheeks and dimples, a small nose, and a wide joyful open smile showing small child's teeth. Stylize into soft appealing 3D animation with slightly larger expressive eyes while keeping her real facial structure. Full body head to toe, standing centered, facing camera, confident happy hero pose, arms relaxed at her sides. She wears bright orange pajamas with a gold star on the chest, a flowing golden-yellow cape, and a glowing gold five-pointed star badge, with a soft warm golden aura around her. Plain light-grey studio background, soft even studio lighting, sharp focus. No text, no watermark, no other characters.

Three structural choices in there are worth isolating:

"Preserve her exact facial structure… the only change is X." Naming exactly one permitted deviation stops the model treating the photo as loose inspiration.
The features are enumerated in prose anyway. Hair, eyes, brows, face shape, nose, mouth. The reference image is not trusted to carry them on its own — and the spacesuit incident, later, shows exactly why that instinct was right.
Full body, centered, plain light-grey background, studio lighting. This is a character sheet, not a picture. It exists to be cut out and re-lit inside twenty different scenes.

Boing's prompt is the same skeleton with his own features and a deliberate note that his skin tone is "a touch warmer and very slightly deeper than his sister's." Sibling colour relationships have to be stated relatively or the two renders drift apart.

The same treatment produced four props:

GlideStar, BounceBuggy, Moon and Twink — isolated, centred, plain background, no text, no other characters, every time. An ingredient image with scenery in it drags that scenery into your shot.

The image that saved the film

The first pass produced Comet.jpg and Boing.jpg. Both head-to-toe. Both filling the frame.

Which meant that neither image contained any height information. Framed identically, a four-year-old and a two-year-old are exactly the same size.

Veo drew them exactly the same size. In the eleven shots where both children appear, the entire visual joke of the film — the big sister and the small round brother — evaporated. In some renders the inversion went the other way, and the two-year-old came out taller than his sister.

The fix was a third reference, the Together.jpg line-up at the top of this post: one frame, both children, correct relative heights baked in, and — the part I did not anticipate — both rendered in the same pass, so their styles could not diverge. Generating them separately had already allowed small drift in eye size and skin shading that a single joint generation eliminates by construction.

That gave three character ingredients plus four props, and one rule that follows immediately from having all three: never attach Comet + Boing + Together at once. Three references of the same two children dilute each other. Use the two solos or the line-up.

The line-up also solves an arithmetic problem. Flow allows a maximum of three ingredients per prompt, and shots 6 and 7 need both children and a vehicle. Together covers both kids in one slot, leaving room for @GlideStar and @BounceBuggy.

Other ways to do this

Not tried — generate the whole cast in one sheet and crop the solos out of it. This is the same insight as Together.jpg, applied from the first image instead of the third. One joint generation cannot drift in style, and every solo you cut out of it inherits the correct scale for free. If you take one thing from this section, take this.

Not tried — local image models instead of Nano Banana. SDXL or Flux with IP-Adapter or InstantID for face conditioning runs on the same GPU that made the score, costs nothing, and never sends a photograph anywhere. Expect to work harder for likeness: the Gemini app's "use both photos as reference for the same child" is doing a lot of quiet lifting, and the sibling-scale problem gets worse, not better, when you have finer control and more knobs to get wrong.

Not tried, and deliberately so — training a LoRA on each child's face. This is the technically strongest route to consistency, and I did not take it. A LoRA is a portable, redistributable model of a real child's face, which is a categorically different artefact from four stylised PNGs sitting in a folder on my machine. The consistency was not worth creating that file. If your subjects are adults who have agreed to it, the calculus is yours to make.

Not tried — hand-drawn or commissioned character sheets. The pipeline does not care where the reference image came from. An illustrator friend and a scanner produce a better ingredient than any of this, and the twenty prompts downstream are unchanged.

Eighty re-rolls beat one pretty render

This is where the project's real budget lives, and it is worth being precise, because the difference between the right and wrong setting is a factor of ten.

Flow's free tier gives 50 credits a day. Google AI Pro gives 1,000 credits a month with no daily cap, which is what I used — it collapses the shoot from a two-week drip into one or two sittings.

Model	Credits per 8 s clip	All 20 shots
Veo 3.1 Lite ← baseline	10	200
Veo 3.1 Fast	20	400
Veo 3.1 Quality	100	2,000

The plan was Lite for everything. 200 credits buys the film and leaves 800 for re-rolls — eighty of them.

That trade is the single most important production decision in the project.

On generative video, more takes beats higher resolution. A Quality render of a shot where the model misunderstood the action is worth nothing; a Lite render on the fourth attempt, where it finally got the timing right, is worth everything.

Quality on the three money shots (9, 16, 19) would have cost 300 more and still fit comfortably inside the budget. It was not needed. And 1080p upscaling is free on Pro, so every finished clip was upscaled anyway.

I shot in story order, 1 through 20. Re-rolls being cheap, there was no reason not to, and continuity problems surfaced while there was still budget to fix them.

The setting that quietly costs you a month

Before generating anything, four agent settings need checking. One of them is a real trap:

Setting	Must be	Why
Confirm before generating	Always	On Never, the agent burns the monthly allocation without asking.
Video model	Veo 3.1 – Lite	The default was Omni 1.1 Flash — outside the Veo family, and outside what the free video credits cover. Materially more per clip.
Video aspect	16:9
Video count	×1	×4 generates four variants per click. Re-roll deliberately.

The model default is the expensive one. Flow's interface offers Gemini Omni alongside Veo, and it was pre-selected. Nothing in the UI flags that the credit-eligible Veo tiers are a different menu entry. Check it before your first generation, not after your twentieth.

Prompt anatomy

Every shot prompt is four stacked blocks. The style block is pasted first, verbatim, never retyped — small wording drifts cause visible style drift across shots:

Modern 3D animated family film, Pixar-quality rendering, soft warm cinematic lighting, shallow depth of field, rich saturated colors, cozy storybook mood, 16:9. No subtitles, no on-screen text, no captions.

Then the scene, which is where the words should actually go. Shot 8, the one where Comet becomes the light source:

They stop at the edge of a pitch-black forest. @boing looks up nervously. Then @comet rises into the air and BLAZES with golden light, and the whole forest path illuminates gold — every leaf and puddle shining like daytime. Wide cinematic reveal. Audio: a swelling magical chime, @boing whispering "whoa."

Note that audio direction is part of the prompt. Veo generates the soundtrack with the picture — spring boings, hiccups, orchestral swells and dialogue all come out of the same generation.

Shot 9, the reveal. The whole film exists to earn this frame: the moon is not in the sky, it is stuck in a tree, and it is enormous, and they are very small. Chaining shot 8's final frame into shot 9's start image is what made the forest match across the cut.

Then, on the eleven two-shots, the sibling-scale line:

Comet is noticeably taller than Boing — he is about a head shorter, rounder and chubbier, with shorter limbs.

And on any shot with dialogue, the voice block for each speaking character, copied verbatim and never reworded between shots:

COMET speaks with a bright, clear little girl's voice, around five years old — warm, confident and cheerful, mid-high pitch, with an excited upward lilt.

THE MOON speaks with an enormous, deep, slow, sleepy male voice — gentle and rumbling, like a kind old giant.

Veo invents a fresh voice on every generation. There is no voice-lock at any Flow tier. The repeated verbatim description is the only lever available — and it does work. It turns "two different children" into "the same child on a different day."

Re-roll budget was concentrated on shots 4 and 15. Shot 4 establishes both catchphrases and both voices and sets the reference the ear anchors to; shot 15 is "SUPER SIBLINGS — GO!", the emotional turn, and it pays off shot 4. Everything else got what it got. The Moon, incidentally, is the most reliable voice in the film — a distinctive deep slow rumble is an easy target for a model to hit repeatedly.

Video: Animated GIF: the heroes launching out of the driveway, glider and spring buggy trailing gold sparks

Shot 6, the roll-out. Three ingredients in one prompt — Together for both children, plus @GlideStar and @BounceBuggy — which is exactly the maximum Flow allows, and only possible because the line-up image collapses two characters into one slot.

One free continuity trick worth stealing: chain the shots. Take the final frame of a shot you like and feed it as the start image for the next. It costs nothing and substantially improves the match across cuts.

Other ways to do this

Tried — the free tier. 50 credits a day is 5 clips a day at Lite rates. The same twenty shots take five sittings, or about ten days if you want a real re-roll budget. Nothing about the film changes; only the calendar does. The Pro month bought me two evenings instead of two weeks, and that is all it bought.

Not tried — Quality on the money shots. Shots 9, 16 and 19 at Veo 3.1 Quality would have cost 300 more credits and still fitted inside the allowance comfortably. I skipped it on judgment, not on budget, because the Lite renders were already the ones I wanted. If your film has three frames that carry it, that is where the credits belong.

Not tried — ×4 variants instead of ×1. Generating four variants per click trades credits for choice, and on a shot you already know is hard (my shots 4 and 15) it may well beat four sequential deliberate re-rolls. I turned it off to stop the allowance evaporating by accident, which is a different concern from whether it is a good idea.

Not tried — first-frame and last-frame conditioning. Chaining fixes the seam going forward. Pinning both ends of a shot, where the tool supports it, would have removed most of my continuity re-rolls outright.

Not tried — leaving Google entirely. Open-weight video models — Wan, LTX-Video, HunyuanVideo, CogVideoX — run locally on a consumer GPU for nothing but electricity, and the hosted services (Kling, Hailuo, Pika, Luma) all have free daily allowances. The catch is specific and large: none of them generate the soundtrack with the picture. Veo's native audio gave me spring boings, hiccups, orchestral swells and every line of dialogue in the same generation. Take that away and eleven dialogue lines and twenty shots of foley become your problem. Check each service's terms on children's likenesses before uploading anything, too — they differ, and they change.

Why my son turned into an astronaut

This is the section I would have wanted to read before starting.

The reference image was right there

Shot 8's first prompt referenced @Boing and described what he does, but not what he looks like. The reasoning seemed sound: the ingredient image already carries the costume, so why retype it?

It came back with Airik in a spacesuit and helmet. Cape gone.

The model had kept the coil springs and the star badge, correctly identified the scene as nocturnal and sky-adjacent, and free-associated its way to an astronaut.

Ingredients bias the look; they do not lock it. A reference image shifts the distribution — it does not constrain it.

Where the prompt text is silent, the model fills the silence from scene context, and "dark sky, glowing, flying" is an extremely strong pull toward space.

The fix is redundancy. Every shot a character appears in, describe them inline as well as attaching the ingredient:

@comet — a girl in bright orange pajamas with a gold star on the chest and a flowing golden-yellow cape, bare head, dark hair in a small ponytail

@boing — a much smaller little boy in royal-blue pajamas with a sky-blue cape worn backwards over his chest, bare head with thick dark hair, and blue boots with coil springs

Plus an explicit negative on every shot with a child in it:

Both children have bare heads with visible hair. No helmets, no space suits, no antennae, no goggles, no masks.

Naming the specific failure mode is far more effective than a generic "no costume changes." The model needs the token.

Refused. Twice.

Shot 14 is the emotional low point of the film. Boing has bounced into a branch three times, sits down in the grass, and his lip wobbles. The original prompt read, in part:

a small three-year-old boy … lower lip wobbling, right on the edge of tears

It was refused. Twice. With no indication of what had tripped it.

The classifier is not objecting to the shot. It is objecting to the conjunction of an explicit child age with distress vocabulary — a pattern that legitimately warrants scrutiny in the general case, and which also happens to describe a completely ordinary beat in every children's film ever made. Because the refusal message names no token, the mitigations have to be derived by reasoning about the classifier rather than by reading the error.

Four fixes, in descending order of effectiveness:

Remove every age word. "The smaller character", "the taller character". The height relationship is already carried by the sibling-scale line and the Together reference, so nothing is lost. This alone cleared most refusals.
Soften distress vocabulary. "Glum", "deflated", "thoughtful". Never "tears", "crying", "upset". The performance you get back is nearly identical — a deflated two-year-old and a tearful two-year-old look the same on screen.
Soften physical contact. "An encouraging pat on the shoulder" rather than "puts an arm around him."
Open with fiction context. Begin the prompt "Two animated cartoon characters…" so the classifier has established this is animation before it reaches the action.

Video: Animated GIF: Boing bouncing up toward the moon in the tree and bonking a branch

Shot 13 — the third failed bounce, which is the beat that leads into the refused shot 14. Rewritten to drop every age word, it generated first time.

And one rule adopted before the first generation and never broken: never upload the original photographs of the children to Flow. Veo flags real faces, and routing the likeness through a stylised Gemini render keeps the actual photos out of the video pipeline entirely. It is the right call on privacy grounds first, and it happens to also be the smoother technical path. If Flow rejects a reference as too photorealistic, push it further into cartoon territory in Gemini ("more stylized, softer, more like a Pixar character, less photographic") and re-upload.

The one that was just my own fault

Minor but persistent: retyping the style block instead of pasting it produced visible tonal drift between shots. Lighting warmth and colour saturation are sensitive to remarkably small wording changes. Paste. Never retype.

Other ways to do this

Tried, failed — trusting the ingredient image to carry the costume. That is the astronaut. Describe the character inline on every single shot, even when the reference is attached.

Tried, failed — naming the children's ages in a prompt with a sad beat in it. Refused twice, with no indication why. Every age word came out and it generated first time.

Not tried — shooting the sad beat without the child in frame. Shot 14 could have been the moon looking down, or a cape on the grass, or the buggy stopped and still. A refusal is sometimes telling you to find a better shot rather than a better prompt, and "the thing he dropped" is a more grown-up piece of filmmaking than "his lip wobbles" anyway.

Not tried — a different service when a shot is refused. I rewrote until Veo accepted. Moving one stubborn shot to another model is legitimate and I never needed to; the cost is that the style block does not travel, so that one shot will not match the other nineteen without grading.

A caution on the workarounds above. Removing age words and softening the vocabulary worked because the shot itself was an ordinary beat from a children's film. These are techniques for describing an innocuous scene in language a classifier can recognise as innocuous. They are not a method for getting something past a filter that is right to stop you, and if you find yourself reaching for them to do that, the filter is working.

A score that follows the story

Every film you have ever loved lied to you about sound.

You remember the pictures. What you actually felt was the score underneath them, doing quiet work you never noticed. I did not understand that properly until I had twenty finished shots that were technically correct and completely dead.

Two of the three jobs here went to plan. The third taught me something I have not seen written down anywhere, and it is the only part of this project that briefly frightened me.

Clip audio from Veo covers the diegetic sound — springs, wind, dialogue — but there is no continuous score, and twenty independently generated 8-second beds would not join.

The score was generated locally with MusicGen Medium (1.5 B parameters) through HuggingFace Transformers, rather than via AudioCraft directly:

import torch, scipy.io.wavfile as wav
from transformers import MusicgenForConditionalGeneration, AutoProcessor

MODEL = "facebook/musicgen-medium"
proc = AutoProcessor.from_pretrained(MODEL)
model = MusicgenForConditionalGeneration.from_pretrained(MODEL, dtype=torch.float16).to("cuda")
sr = model.config.audio_encoder.sampling_rate

PROMPTS = [
 ("a", "gentle warm orchestral lullaby for a children's animated bedtime film, soft music box, harp and warm strings, tender magical and hopeful, slow calm tempo"),
 ("b", "whimsical light orchestral score for a children's animated adventure, playful pizzicato strings, soft woodwinds, warm gentle and storybook, moderate tempo"),
 ("c", "dreamy cinematic lullaby, celesta and soft strings under a night sky, magical wonder, calm slow and emotional, no drums"),
]
TOKENS = 30 * 50   # 50 audio tokens per second -> 30 seconds

for tag, prompt in PROMPTS:
    inputs = proc(text=[prompt], padding=True, return_tensors="pt").to("cuda")
    with torch.no_grad():
        audio = model.generate(**inputs, do_sample=True, guidance_scale=3.0,
                               max_new_tokens=TOKENS)
    a = audio[0, 0].float().cpu().numpy()
    a = a / max(abs(a).max(), 1e-6) * 0.9
    wav.write(f"music-option-{tag}.wav", rate=sr, data=(a * 32767).astype("int16"))

Three 30-second cues, at 50 audio tokens per second. MusicGen's practical ceiling for coherent output is around 30 seconds; asking for 200 produces something that wanders.

The naive next step — loop one 30-second cue twenty times — produces three minutes of unchanging texture that actively fights the film. The moon reveal and the goodnight need different music.

So the bed is arranged, using the two cues that survived audition (b, the adventure spine; c, wonder and emotion) laid out against the shot map:

Section	Shots	Cue	Length
Adventure	1–8	b	82 s
Wonder + the quiet beat	9–14	c	62 s
Triumph	15–18	b	42 s
Home	19–20	c	20 s

Each section is built by crossfading a cue into itself — a hard loop point is audible, a 2-second triangular crossfade is not — and then the four sections are crossfaded together:

X=2   # crossfade seconds

# seamless_loop <src> <copies> <trim_len> <out>
seamless_loop () {
  local src=$1 copies=$2 len=$3 out=$4
  local inputs="" fc="" cur="[0:a]"
  for i in $(seq 0 $((copies-1))); do inputs="$inputs -i $src"; done
  for i in $(seq 1 $((copies-1))); do
    fc="${fc}${cur}[${i}:a]acrossfade=d=${X}:c1=tri:c2=tri[x${i}];"
    cur="[x${i}]"
  done
  "$FF" -y $inputs -filter_complex \
    "${fc}${cur}atrim=0:${len},asetpts=PTS-STARTPTS,aresample=48000[o]" \
    -map "[o]" -c:a pcm_s16le "$out"
}

seamless_loop music-option-b.mp3 3 82 _s1.wav   # shots 1-8   adventure
seamless_loop music-option-c.mp3 3 62 _s2.wav   # shots 9-14  wonder + the quiet beat
seamless_loop music-option-b.mp3 2 42 _s3.wav   # shots 15-18 triumph
seamless_loop music-option-c.mp3 1 20 _s4.wav   # shots 19-20 home

"$FF" -y -i _s1.wav -i _s2.wav -i _s3.wav -i _s4.wav \
  -filter_complex "[0][1]acrossfade=d=${X}:c1=tri:c2=tri[a];\
[a][2]acrossfade=d=${X}:c1=tri:c2=tri[b];\
[b][3]acrossfade=d=${X}:c1=tri:c2=tri[c];\
[c]loudnorm=I=-22:TP=-3,atrim=0:200.05[out]" \
  -map "[out]" -c:a libmp3lame -b:a 192k music.mp3

The bed is normalised to −22 LUFS — deliberately quiet, because it is going to sit under narration and clip audio in the final mix and get normalised again there.

Other ways to do this

Not tried — no score at all. Veo's native audio already carries the film. The music-only cut exists because the score turned out well, not because the picture needed rescuing. If you are on a laptop with no GPU, cutting this entire section is a legitimate choice and costs you less than you would think.

Not tried — library music. The YouTube Audio Library, the Free Music Archive and Incompetech are free, cleared, and vastly better recorded than anything MusicGen produces. What you give up is exactly the thing this section is about: a library track does not know where your reveal is. You would be crossfading someone else's cues against your beat map instead of your own, which is the same arrangement work with less control.

Not tried — MusicGen's melody-conditioned and continuation modes. Those are the routes to genuine through-composition instead of four crossfaded loops. My arrangement is the cheap eighty per cent: it changes when the story changes and it never repeats audibly, and that was enough. It is not the same thing as a score that develops.

Not tried — Stable Audio Open, or MusicGen Small on CPU. The Small model runs without a GPU at maybe a minute of compute per thirty seconds of audio. Three cues is a coffee break, not an obstacle.

Not tried — scoring it properly. A person with a keyboard and a free DAW will beat all of the above in an afternoon, and I mention it because it is easy to forget that the non-AI option is still sitting there.

Shot 16, where cue b comes back for the triumph section. The tree is lit by Comet, not by the moon — the payoff of a power established eight shots earlier.

The voice you are not allowed to clone

The film works on Veo's own dialogue and the score alone. But a single constant narrator across all twenty shots gives the ear something stable to hold, and the drifting character voices become colour rather than the spine of the film. So I recorded a narration, and then cloned it.

English: F5-TTS

F5-TTS is a flow-matching TTS with zero-shot voice cloning: give it a reference clip and its transcript, and it generates arbitrary new text in that voice. The reference is one sentence from my own recording:

REF_TEXT = ("At the top of the puddle hills stood the biggest oak tree in the world, "
            "and in it was the moon.")

tts = F5TTS()
for n, text in lines:
    tts.infer(ref_file="../04_AUDIO/vo_raw/_ref.wav", ref_text=REF_TEXT,
              gen_text=text, file_wave=f"narr-{n}.wav", remove_silence=True)

Narration is stored as a machine-readable NN|text file keyed to shot number, so the same lines drive every language and every voice:

01|One night, the sky went dark. Very, very dark.
02|This is Liya. She is four. And this is Eye-rick. He is two. Every night, they turn into superheroes!
...
19|Up and up went the moon, all the way home. Click!
20|The moon said thank you, and tucked them into bed. Goodnight, Liya. Goodnight, Eye-rick.

Note the phonetic respellings — "Liya", "Eye-rick". The models mispronounce the real spellings, and this file is read by machines, not humans; the human reading scripts are separate files with modulation marks.

Only fifteen of the twenty slots carry narration. Shots 3, 4, 5, 10 and 15 are left alone: those are the catchphrase and dialogue shots, and talking over them would be vandalism.

Each generated line then gets fitted to its 10-second slot, with the time-stretch capped so it stays natural:

d=$(ffprobe -v error -show_entries format=duration -of csv=p=0 "$f")
t=$(python -c "d=$d; print(round(min(max(d/9.2,1.0),1.15),4))")
[ "$t" != "1.0" ] && ffmpeg -y -i "$f" -filter:a "atempo=$t" _t.wav && mv -f _t.wav "$f"

A cap of 1.15× is the point past which a bedtime read starts sounding hurried. If a line will not fit at 1.15×, the correct fix is to shorten the line, not to speed it up further.

Lines are then placed at their shot's start time — shot N begins at (N−1)×10 s — with a 0.4 s lead-in so the picture establishes before the voice starts, and mixed:

FILTER="$FILTER[$IDX:a]aresample=48000,adelay=${off}|${off}[d$IDX];"

The ElevenLabs finding

I evaluated ElevenLabs as an alternative, and two things came back that shaped the project.

Voice cloning is not on the free tier. Instant Voice Cloning starts at the paid Starter plan; Professional Voice Cloning starts at Creator. The free tier gets text-to-speech with stock voices only. That is why one of the six cuts is narrated by "Sarah", a stock voice — it was the free-tier option, and it is genuinely good, but it is not anyone's parent.

Cloning a minor's voice is prohibited outright. The ElevenLabs use policy forbids replicating a person's voice "without consent or legal right", forbids material designed to impersonate a minor, and restricts service availability to minors. Parental consent does not open a door here; the prohibition is on the output.

The children's voices in this film are entirely Veo's invention. Only the adults' voices — mine and their mother's — were ever cloned.

It is the right line, and I would have drawn it there myself, but it is worth knowing before you plan a pipeline around it. If you want a child's voice in a film: generate a fictional one, or record the child directly. Do not clone.

Other ways to do this

Not tried — free local cloning stacks. F5-TTS is not the only one: XTTS-v2, OpenVoice and Chatterbox all do zero-shot cloning on consumer hardware. Read the licence before anything commercial — some of these ship permissive code with non-commercial weights, and the two are separately licensed. None of them change the rule above: do not clone a child's voice, whichever tool makes it easy, and get an adult's explicit agreement before cloning theirs.

Not tried — no cloning at all. Piper runs on a CPU, has good stock voices and is genuinely instant; Kokoro is small and unusually natural for its size. Neither will sound like you. For a narrator, that may not matter — one of my six cuts is narrated by an ElevenLabs stock voice and it is perfectly good, it is just not anybody's parent.

Tried, and it won — recording it yourself. The cheapest option in this entire pipeline is a phone and a quiet-ish room, and it produced the best voice in the film. See the grandmother section below; the numbers are not close. If you have any way at all to record a real person reading it, do that first and treat cloning as the fallback for when you cannot.

Malayalam: IndicF5

Their mother's family speaks Malayalam, so two of the six cuts are Malayalam — one read by their grandmother, one in my cloned voice.

IndicF5 from AI4Bharat is an F5-architecture model trained on 1,400+ hours across 11 Indian languages, Malayalam included, MIT-licensed. It ships as a HuggingFace repo with a bundled f5_tts package.

Getting it running produced the most interesting failure in the entire project.

Three times too long, and full of things I never wrote

Every Malayalam line came out roughly three times longer than it should have been. And the surplus was not silence. It was not noise either.

It was fluent, confident, entirely invented Malayalam — the model calmly narrating sentences that appeared nowhere in the input, in my own cloned voice, to my children.

Where it comes from

F5-TTS estimates how much audio to generate from the length of the target text. The estimate is:

len(text.encode("utf-8"))

For English, UTF-8 is 1 byte per character and this is a perfectly fine proxy for duration.

Malayalam lives in the U+0D00 block. Three bytes per character.

So a 60-character Malayalam sentence measures as 180 "characters". Scaled against an English reference clip, the model is instructed to produce roughly 3× the audio the text actually needs — and a flow-matching TTS asked to fill a duration will fill it. It does not stop early. It generates plausible speech until the buffer is full.

This is a byte-length-as-proxy-for-text-length bug — the same family as truncating a UTF-8 string at a byte offset. It is completely invisible until the script changes.

The fix

Estimate from akshara count instead — Malayalam's orthographic units. An akshara is a base character optionally followed by combining marks (vowel signs, virama), and the combining marks do not add duration. So: normalise to NFC, count characters that are neither combining marks nor whitespace.

UNITS_PER_SEC = 6.6   # Malayalam aksharas per second; lower = slower/longer

def akshara_count(text):
    return sum(1 for ch in unicodedata.normalize("NFC", text)
               if not unicodedata.combining(ch) and not ch.isspace())

def est_seconds(text):
    return akshara_count(text) / UNITS_PER_SEC + 0.35

6.6 aksharas per second was tuned by ear against a test line; the +0.35 s is trailing room so the final word is not clipped. The estimate is then passed explicitly as fix_duration, computed as the reference clip's duration plus the target's:

audio, sr, _ = infer_batch_process(
    (ref_t, sr_in), ref_text, [chunk], model, vocoder,
    mel_spec_type="vocos", speed=speed, device=device,
    fix_duration=ref_dur + gen_sec)

And then it came back

Calling infer_process() — the obvious high-level entry point — reintroduces the whole problem from a different direction. It re-chunks the input on a byte-based max_chars and then applies fix_duration to each resulting sub-batch, so a line that got split in two comes out at twice the intended length.

The fix is to chunk it yourself, on sentence boundaries, sized by your own duration estimate, and call infer_batch_process directly with exactly one batch:

def split_sentences(text, max_sec=10.0):
    """Split on Malayalam/Latin sentence enders, regrouping into <= max_sec chunks."""
    parts = [p for p in re.split(r"(?<=[.!?।])\s+", text.strip()) if p]
    chunks, cur = [], ""
    for p in parts:
        cand = (cur + " " + p).strip()
        if cur and est_seconds(cand) > max_sec:
            chunks.append(cur); cur = p
        else:
            cur = cand
    if cur: chunks.append(cur)
    return chunks or [text]

Chunks are concatenated with a 0.18 s gap and per-chunk silence trimming.

Two more IndicF5 mechanics

Namespace-package shadowing. The IndicF5 mirror bundles its own f5_tts package, which must take precedence over the pip-installed F5-TTS. Because f5_tts is a namespace package, Python merrily merges both directories on sys.path and you get a hybrid that imports without error and behaves wrongly — the worst possible failure mode. Prepending to sys.path is not enough; you have to pin __path__ and assert it:

sys.path.insert(0, REPO)
import f5_tts
f5_tts.__path__ = [os.path.join(REPO, "f5_tts")]   # namespace pkg: pin to bundled copy
assert os.path.abspath(f5_tts.__path__[0]).startswith(REPO), f"wrong f5_tts: {list(f5_tts.__path__)}"

Checkpoint key prefixes. The published model.safetensors is a saved torch.compiled wrapper. Its keys are prefixed ema_model._orig_mod. and it also bundles the vocoder weights, while the F5-TTS loader expects bare EMA keys. Strip and filter once, cache the result:

from safetensors.torch import load_file, save_file
pre = "ema_model._orig_mod."
sd = load_file(src)
out = {k[len(pre):]: v.contiguous() for k, v in sd.items() if k.startswith(pre)}
assert out, "no ema_model._orig_mod.* keys found"
save_file(out, dst)

And a platform note. PyTorch on a Blackwell GPU (sm_120) needs the CUDA 12.8 wheels:

python -m venv .venv-tts
.venv-tts/Scripts/python.exe -m pip install torch torchaudio \
    --index-url https://download.pytorch.org/whl/cu128
.venv-tts/Scripts/python.exe -m pip install f5-tts

Also, torchaudio's current I/O path goes through torchcodec, which wants FFmpeg ≤ 7 shared libraries; the FFmpeg 9 static build on this machine is incompatible. Rather than maintaining a second FFmpeg, a ten-line shim routes torchaudio.load/save through soundfile:

# route torchaudio I/O through soundfile: torchcodec needs ffmpeg<=7 shared libs, we have 9 static
import numpy as np, soundfile as sf, torch, torchaudio
def _load(path, *a, **k):
    data, sr = sf.read(str(path), dtype="float32", always_2d=True)
    return torch.from_numpy(np.ascontiguousarray(data.T)), sr
def _save(path, tensor, sample_rate, *a, **k):
    arr = tensor.detach().cpu().numpy()
    if arr.ndim == 2: arr = arr.T
    sf.write(str(path), arr, int(sample_rate))
torchaudio.load, torchaudio.save = _load, _save

Other ways to do this

Not tried — a forced aligner instead of a duration estimate. The whole bug exists because the model guesses how long the text should take. Montreal Forced Aligner and aeneas measure it against real audio instead. That does not help you generate speech, but it is the right tool for anything downstream that needs to know where words land, and it would have made the subtitle timing a solved problem rather than a clever one.

Not tried — other Indic TTS. AI4Bharat publish more than IndicF5, and the large cloud providers have Malayalam voices on free tiers. If your target language is not English, budget an evening for the possibility that the obvious model simply does not speak it well.

Not tried — skipping synthesis entirely for the second language. The Malayalam cut that people actually prefer is the one their grandmother read into a phone. Cloning my own voice into Malayalam was the technically interesting path and the second-best result.

The transferable rule, which cost me a day: synthesise one line in the target script and listen to it before you build anything around the model. Three bytes per character would have been obvious in five minutes and instead it surfaced across a full batch of fluent, invented, confidently-narrated Malayalam.

Fluent, and completely flat

I fixed the bug. The lines came back the right length, in my own voice, saying exactly what I had written. It worked.

It was also lifeless, and that is the part I want you to hear rather than take my word for.

Fluent is not the same as good. A model trained to fill a duration gives you the right phonemes and almost no performance: even stress, even pace, no lift into a question, no warmth landing on a child's name. Measured against the real read, the clone covers 5.1 semitones of pitch range to the recording's 10.0 — less than half the melody. It is a voice that is technically mine and expressively nobody's.

It was slow, too. Every re-render was minutes rather than seconds, so tuning fifteen lines was an evening's work, not a loop you could spin quickly. I spent that evening chasing warmth that the method could not produce.

Listen to the same line twice. This is shot 2, in both languages — the clone first, then the person.

Malayalam, shot 2

ഇത് ലിയ. അവൾക്ക് നാല് വയസ്സ്. ഇത് ഐറിക്. അവന് രണ്ട് വയസ്സ്. രാത്രിയായാൽ അവരുടെ ലോകം സൂപ്പർ ഹീറോകളുടെ ലോകമാണ്.

This is Lia. She is four. This is Airik. He is two. At night, their world becomes a world of superheroes.

IndicF5 clone — my voice, cloned into Malayalam: https://www.razi.pro/audio/blog/star-patrol/ml-02-ai-indicf5.mp3
Real recording — read into a phone, in a living room: https://www.razi.pro/audio/blog/star-patrol/ml-02-real-amma.mp3

▶ Play both takes side by side on razi.pro

English, shot 2

This is Liya. She is four. And this is Eye-rick. He is two. Every night, they turn into superheroes!

ElevenLabs stock voice — the best synthetic take of the three I tried: https://www.razi.pro/audio/blog/star-patrol/en-02-ai-elevenlabs.mp3
Real recording — one continuous read, cut up afterwards: https://www.razi.pro/audio/blog/star-patrol/en-02-real-dad.mp3

▶ Play both takes side by side on razi.pro

The synthetic takes are not bad. They are competent, clear, and completely uninterested in the story. The recordings have someone in them who knows these two children.

So the film ships human voices. Every clone I built became an alternate cut, and the two versions in the FINAL folder are both real reads — which quietly vindicated a working rule I had written on day one, before I understood why it mattered: record the narration yourself rather than relying on generated voices. I had meant it as a note about consistency. It turned out to be a note about feeling.

That is not an argument against voice cloning. It is an argument about what it is for. Cloning solved a problem I actually had — a Malayalam cut in my own voice, which I cannot perform — and it solved it well enough to ship as an option. What it could not do was care.

Six films from one master

A film is not finished when it looks finished.

Twenty clips existed. The score existed. What did not exist was a file — something with fades and a mix and captions that a person could open on a phone in another country and understand. That gap took longer than the shooting.

Six versions come out of this chapter, all frame-identical, differing only in narration. Almost everything below exists because six is not one.

Picture

The twenty clips are joined with FFmpeg's concat demuxer, which is stream-copy — no re-encode, no generation loss — producing _master-assembly-nofades.mp4 at 200.042667 s. Every deliverable is encoded from that one file, so all six cuts are frame-identical.

ffmpeg -f concat -safe 0 -i shots.txt -c copy _master-assembly-nofades.mp4

This works because all twenty clips came out of the same model at the same settings and therefore share codec parameters exactly. It is the reason to keep the pipeline uniform.

Video: Animated GIF: the huge combined bounce, Boing launched high toward the moon

Shot 17, the mega bounce. LRA=11 in the final loudness normalisation exists specifically so this still feels big after everything else has been levelled.

The mix

build-final.sh takes the master, a music bed, and optionally a narration track, and produces one deliverable. The whole mix is a single filter_complex.

Clip audio is attenuated by how much else is competing with it:

if   [ -n "$NARR" ];  then CLIPVOL=0.45     # narration present: clip audio is texture
elif [ -n "$MUSIC" ]; then CLIPVOL=0.85
else                       CLIPVOL=1.0; fi

The music bed is looped to length, faded, and mixed under the clip audio:

INPUTS+=(-stream_loop -1 -i "$MUSIC")
FC+="[${IDX}:a]volume=0.45,afade=t=in:st=0:d=3,afade=t=out:st=${FOUT}:d=1.5,atrim=0:${DUR},asetpts=PTS-STARTPTS[mus];"
FC+="[clip][mus]amix=inputs=2:duration=first:normalize=0[bed];"

normalize=0 on every amix is important: by default amix divides by the number of inputs, which would silently halve everything each time a layer is added. Levels here are set deliberately, so the automatic normalisation is switched off.

Narration is the interesting part. The bed is sidechain-compressed against the narration, so music and clip audio duck automatically whenever the narrator speaks and come back up when they stop:

INPUTS+=(-i "$NARR")
FC+="[${IDX}:a]volume=1.3,aresample=48000[narr];"
FC+="[${IDX}:a]volume=1.3,aresample=48000[narrsc];"
FC+="${BED}[narrsc]sidechaincompress=threshold=0.04:ratio=9:attack=15:release=350[duck];"
FC+="[duck][narr]amix=inputs=2:duration=first:normalize=0[mixed];"

The narration is split into two identical branches — [narr] goes into the mix, [narrsc] is the sidechain key — because sidechaincompress consumes its key input. The parameters: ratio=9 is aggressive, because the goal is intelligibility rather than subtlety; attack=15 ms ducks fast enough not to clip the narrator's first syllable; release=350 ms is slow enough that the music does not pump between words within a sentence.

Finally, loudness normalisation, a safety limiter, and matched audio/video fades:

FC+="${BED}loudnorm=I=-16:TP=-1.5:LRA=11,alimiter=limit=0.97,afade=t=in:st=0:d=1.5,afade=t=out:st=${FOUT}:d=1.5[aout]"

"$FFMPEG" -y "${INPUTS[@]}" \
  -filter_complex "$FC" -map 0:v -map "[aout]" \
  -vf "fade=t=in:st=0:d=1.5,fade=t=out:st=${FOUT}:d=1.5" \
  -c:v libx264 -crf 18 -preset medium -pix_fmt yuv420p \
  -c:a aac -b:a 192k -ar 48000 -movflags +faststart -t "$DUR" "$OUT"

−16 LUFS with a −1.5 dBTP ceiling is the streaming-delivery target and the right choice for something played back on a tablet or a TV at bedtime. Video fade in and out are 1.5 s, matched to the audio fades, starting at 198.5 s.

Building a deliverable is then one line:

NARR_FILE=narration-ml-amma.wav NARR_OUT="Star Patrol (Malayalam - Amma).mp4" ./build-final.sh
./build-final.sh    # music-only version

Six cuts, one script, one master. The script itself came out of a terminal session rather than my own hands — The pipeline had a fourth author.

Other ways to do this

Not tried — a timeline editor. DaVinci Resolve's free edition and Shotcut both do everything in this section with a mouse — as would Premiere Pro, Final Cut Pro, CapCut or InShot — and for a single deliverable they are unquestionably the faster route. The script pays for itself at cut number two: six frame-identical versions, in six languages and voices, from one command and no chance of a drifting edit between them. If you are only ever shipping one file, use the GUI and skip this whole section without guilt.

Not tried — manual ducking instead of a sidechain. Drawing volume automation under the narration by hand is what an editor would do, and it sounds better than any compressor because a human knows which word matters. It also has to be redone for every language. Sidechaining is the choice that scales; automation is the choice that sounds best.

Tried — leaving the clip audio alone. The music-only cut runs Veo's native audio at full level with the score under it, and it is the version I would show someone who has never seen the film. Not every deliverable needs the whole mix.

Subtitles nobody asked me to burn in

Every deliverable ships with a .srt beside it. None of them have the subtitles burned into the picture, and that was a decision rather than an omission.

I did not write the subtitle timing either. The cue splitting, the character-per-second limits and the two-language variants all came out of the same terminal session as the rest of 05_TOOLS/ — I specified the constraints and corrected the output against the film.

Burned-in subtitles are permanent. They cannot be turned off, they cannot be restyled by a player that knows more about the viewer's screen than I do, they are baked at one resolution and re-scaled badly at every other, and they make the file untranslatable — the Malayalam cut and the English cut share a picture master precisely so that all six deliverables are frame-identical, and burning text in would fork the picture six ways. A sidecar .srt costs two kilobytes and keeps every one of those options open.

The timing is the part worth stealing. The obvious approach is to estimate: count characters, divide by a reading rate, hope. That is the same class of mistake as the byte-length duration bug, and it fails the same way — silently, and worse on the language you can't proofread.

So nothing is estimated. Each narration line is anchored to its shot slot by the same arithmetic the audio uses — line for shot N starts at (N-1) * 10 + 0.4 — and then every word inside that line inherits a measured timestamp from the Scribe transcription of the actual read:

slot   = rd.SLOT[n]
offset = ((slot - 1) * rd.SHOT + rd.LEAD_IN) - s_src

s_src is where the line began in the raw take. The line was cut there and placed at its slot, so a word's position in the finished film is just its measured position in the source plus that constant offset. The subtitle and the voice cannot drift apart, because they are derived from the same measurement.

One refinement that matters more than it sounds: the cue text comes from the script, not from the transcriber. Scribe heard "Leah", "Iric" and "Quk"; the script says "Lia", "Airik" and "quick". Subtitling the ASR output would put the recogniser's spelling of my children's names on screen. Instead the script's tokens are aligned to the ASR's tokens with difflib.SequenceMatcher, inherit their times, and anything unmatched is interpolated between its known neighbours.

Take the timing from the machine and the words from the human. Never both from the machine.

Cues are then wrapped to 42 characters and two lines (38 for Malayalam — the glyphs are wider), split at sentence ends where possible, capped at 6 seconds, and nudged so no cue ever overlaps the next by less than 40 ms.

And the practical detail that makes all of it work: name the .srt to match the .mp4 byte for byte, basename included.

Star Patrol (English - Dad FINAL).mp4
Star Patrol (English - Dad FINAL).srt

VLC, mpv, Plex, Jellyfin and the Windows and macOS system players all load a sidecar subtitle automatically on that rule alone. There is no metadata, no muxing, no configuration. Star Patrol (English - Dad FINAL) subs.srt does not load. The space before subs is the entire difference between a subtitle that appears and one that does not.

The burn-in I tried anyway. For completeness I did attempt a hard-subbed Malayalam cut, and it failed twice for two unrelated reasons, both worth knowing before you spend an evening on them.

First, the filter path. FFmpeg's subtitles filter takes a fontsdir, and the filter-graph parser treats : as the option separator — so a Windows path detonates on its own drive letter:

-vf "subtitles=subs.srt:fontsdir=C:/Windows/Fonts"
                                  ^ parsed as the end of the fontsdir option

Escaping it (C\\:/Windows/Fonts) works sometimes and depends on how many layers of shell and filter-graph quoting the string has already survived, which on Git Bash under Windows is more than you would like. The reliable fix is the one the watermark script below already uses: stage the font into the working directory and pass a plain relative path, because a relative path is the one form that no parser in the stack tries to rewrite.

Second, and fatally, the glyphs. Malayalam is a complex script — conjuncts, reordered vowel signs, the lot — and the default font libass reaches for does not have it. You get boxes, or you get correctly-spaced garbage, which is worse because it looks like it worked. Malayalam on Windows needs Nirmala UI, and it needs a shaper that will actually reorder the marks.

Both problems are solvable. Neither is solvable quickly, and both of them are arguments for the sidecar file, which hands the rendering to a player that already has a font stack and already knows the viewer's screen. The .srt was not a compromise. It was the better answer that I only fully appreciated after trying the other one.

Other ways to do this

Not tried — soft subtitles muxed into the MP4. -c:s mov_text puts the captions inside the file as a track the viewer can switch off, which gets you one file to send someone without giving up any of the advantages of not burning in. It is the obvious middle road and I simply did not need it, because everything here is played from a folder.

Not tried — letting YouTube do it. Upload the .srt, or upload nothing and edit the auto-captions in YouTube's own editor. Zero tooling, and for a film that lives on YouTube anyway it is hard to argue with. You lose the sidecar file for local playback, which is the only reason I did it myself.

Not tried — an aligner instead of my trigram matching. See the note in the Malayalam section. difflib.SequenceMatcher against ASR tokens is a workaround for not having a forced aligner; it works, but a real aligner is the tool for this job.

Worth keeping whatever you choose: take the timing from the machine and the words from the human. That rule survives every one of these routes.

A watermark that will not sit still

Every deliverable carries a drifting razi.pro watermark, applied during the build. This is the least glamorous section in the post and the one I would most want if I were shipping something to the internet.

The whole mark is defined in exactly one place, 05_TOOLS/_watermark-vf.sh, which emits a filter chain on stdout and is called by both the builder and a standalone burner. Nothing else knows what the watermark looks like.

Three decisions in it are non-obvious.

It is drawtext, not an image overlay. The no-asset route: no PNG to keep in sync, no alpha channel to get wrong, and the size is an expression rather than a number, so the mark scales with the frame instead of being authored for 1080p and shrinking to nothing on a 4K export:

FONT="05_TOOLS/.wm-work/wm-font.ttf"   # staged: a relative path survives every parser
ffmpeg -i in.mp4 -vf \
  "drawtext=fontfile=${FONT}:text='razi.pro':fontsize=h/40:fontcolor=white\
:shadowcolor=black@0.35:shadowx=1:shadowy=1:alpha='0.060'\
:x=(W-tw)/2:y=(H-th)/2" \
  -c:v libx264 -crf 18 -preset medium -pix_fmt yuv420p -c:a copy out.mp4

fontsize=h/40 is 27 px at 1080p and 54 px at 4K. The shadow is not decoration — it is what keeps a 6% white mark readable when a bright frame comes up underneath it, and it is scaled to the mark, because at that alpha a heavier shadow reads as a dark smudge with nothing inside it.

The image-overlay equivalent is worth having in your pocket for when a logo is mandatory rather than a text mark. Roughly 8% of frame width, bottom-right with a margin, and the opacity applied to the asset rather than baked into the PNG:

ffmpeg -i in.mp4 -i logo.png -filter_complex \
  "[1:v][0:v]scale2ref=w=iw*0.08:h=ow/mdar[wm][base];\
   [wm]format=rgba,colorchannelmixer=aa=0.12[wmo];\
   [base][wmo]overlay=W-w-24:H-h-24" \
  -c:v libx264 -crf 18 -preset medium -pix_fmt yuv420p -c:a copy out.mp4

scale2ref rather than scale is not a stylistic choice: plain scale cannot see the main frame's dimensions, and reaching for main_w inside it fails with "Expressions with scale2ref variables are not valid in scale filter". scale2ref takes the video as a second input purely to measure it, so iw*0.08 means 8% of the frame, not 8% of the logo — and the mark stays proportional at every export resolution.

format=rgba,colorchannelmixer=aa=0.12 is the part people get wrong. Exporting a pre-faded PNG bakes the opacity into the asset and you cannot change your mind without going back to the image editor; doing it in the filter graph makes opacity a parameter of the build.

It moves, and it never repeats. A watermark in a fixed corner is one crop away from gone. Two copies of the text drift on independent paths, each axis a sum of two sines with mutually incommensurate periods:

AX="(W-tw)/2*(1+0.60*sin(2*PI*t/71.0)+0.36*sin(2*PI*t/29.3+2.1))"
AY="(H-th)/2*(1+0.58*sin(2*PI*t/53.7+0.7)+0.38*sin(2*PI*t/23.1+3.9))"
AA="0.060+0.012*sin(2*PI*t/41.0)"

Position is a continuous function of t, so it flows rather than teleporting between corners; the amplitudes sum to under 1.0, so it mathematically cannot clip out of frame; and because 71 and 29.3 share no common period inside 200 seconds, no fixed crop or blur region removes the mark for the whole runtime. The opacity breathes too, on a third period, so it never settles into something a static filter could subtract.

A watermark's job is not to be seen. It is to be expensive to remove. Motion on incommensurate periods is what turns a corner logo into a removal problem.

It is applied at the final encode, and never to the master. _master-assembly-nofades.mp4 is clean, and stays clean. The mark is appended to the video filter chain after the fades, inside the same encode that does the mix — so it costs nothing extra, and it exists only in the distribution copy:

VF="fade=t=in:st=0:d=1.5,fade=t=out:st=${FOUT}:d=1.5"
WM="$(bash "$HERE/_watermark-vf.sh")"
[ -n "$WM" ] && VF+=",$WM"

This is the rule that has no exceptions. A watermarked master is a master you can never re-grade, never re-cut, never hand to a festival, and never re-encode without stacking a second mark on top of the first. WATERMARK=0 builds a clean copy, and that is what a client or an archive gets. The archival copy is the one that has to survive decisions you have not made yet.

Finally: check it on a dark shot. Faint white over black is where a watermark disappears, and a value tuned against a daylight frame can vanish entirely at night. To confirm it survived the encode, lift the shadows rather than boosting contrast — eq=contrast pivots at mid-grey and crushes exactly the range the mark lives in:

ffmpeg -i out.mp4 -vf "format=gray,lut=y='min(255\,val*9)'" -frames:v 1 check.png

Other ways to do this

Not tried — no watermark. Genuinely an option, and for a film about your own children that mostly gets played on your own television, arguably the right one. Everything above is machinery for a problem you may not have. The rule that matters even then is the last one: never mark the master.

Not tried — Content Credentials (C2PA). Cryptographically signed provenance metadata attached to the file rather than painted onto the picture. It answers "where did this come from" properly, which a drifting text mark does not, and it survives none of the things a visible mark survives. The two solve different problems and are not alternatives so much as complements.

Not tried — invisible watermarking. Steganographic marks that survive re-encoding and cropping, recoverable by a detector rather than by eye. If the actual goal is proving ownership after the fact, this is the serious answer and drawing text on the frame is not.

Be honest about what the visible mark does. It raises the cost of a lazy repost. It does not stop a determined one. I built the drifting version because a corner logo is one crop away from gone, not because I think it is protection.

Shot 19. Cue c returns, the sidechain has nothing left to duck against, and the film starts its 1.5-second fade out.

Cutting a recording apart with a computer

The best performance in the film was recorded on a phone, in a living room, by someone who had never recorded anything.

That is the chapter. Everything below is the machinery for getting a real human read — retakes, fridge hum, corrupt frame and all — into fifteen shot-shaped pieces without flattening what made it worth keeping.

The recorded narrations arrived as one continuous take per language: fifteen lines read top to bottom, three seconds of silence between them, fumbles simply repeated after a pause. That is the right way to record — no syncing to picture, no per-line file management — but it means the take has to be cut up afterwards, retakes and all.

Whisper vs Scribe on Malayalam

First attempt used faster-whisper with large-v3, which is the obvious local choice:

from faster_whisper import WhisperModel
m = WhisperModel("large-v3", device="cpu", compute_type="int8", cpu_threads=20)
segs, _ = m.transcribe("voice-female-clean.wav", language="ml", vad_filter=True,
                       vad_parameters=dict(min_silence_duration_ms=350, speech_pad_ms=120),
                       word_timestamps=True, beam_size=5)

It failed on the Malayalam recording. Whisper's training distribution is heavily English-weighted, and its documented tendency to hallucinate text that was not spoken gets much worse in low-resource languages; the output was not usable for alignment. ElevenLabs Scribe transcribed the same file correctly — Malayalam is in its high-accuracy band, under 10% WER, across its 90+ supported languages.

Worth stating plainly: for English, faster-whisper is excellent and free and there was no reason to use anything else. For Malayalam it was simply the wrong tool, and there is no amount of parameter tuning that fixes a model that does not know the language well enough.

Nine of thirteen segments started mid-word

The English recording needed the same treatment, and my first attempt at it was bad: segments chopped at fixed durations to fit 10-second slots, with nine of thirteen starting or ending mid-word.

The rebuild — again, written in-session rather than by me — uses a deliberately energy-first, not ASR-first design:

Find utterance blocks by measured energy. Compute an RMS envelope, take the 10th percentile as the noise floor, gate at 6 dB above it, merge pauses shorter than 250 ms, discard blips under 350 ms.
Use word timestamps only to decide which block is which line. Whisper's word boundaries stretch across pauses; measured silence does not lie. Cutting on energy is immune to both that and to the flubbed retake sitting in the middle of the take.
Match blocks to script lines on character trigrams, not words. Word-level overlap is too brittle — the transcript says "good night" and "mid air" where the script has "Goodnight" and "midair", which zeroes a word-overlap score on precisely the lines that most need placing.
Detect false starts by asymmetric containment. A false start is a short fragment whose text all reappears in the retake that follows. Symmetric F1 sinks on that comparison because the lengths differ wildly; asymmetric containment — how much of the small block reappears in the following blocks — catches it cleanly. Blocks are then assigned a drop cost: free if they contain no words (breath, page turn) or are a detected false start, expensive otherwise, which is what prevents the optimiser cheerfully deleting real narration.

Every cut is then verified: if either edge of a cut sits more than 8 dB above the noise floor, it landed on speech, and the script exits non-zero.

The retained segments get a "storyteller warm" enhancement chain. The order matters, and the comments record a mistake worth reproducing — a naive bass boost cost 4 dB of SNR, because this room's tone is low-frequency and sits exactly where the warmth goes:

ENHANCE = ",".join([
    "highpass=f=85:poles=2",                # rumble out (fundamental is 120-140Hz)
    "afftdn=nr=20:nf=-38",                  # denoise at the measured floor
    "agate=threshold=0.03:ratio=3:attack=10:release=200:knee=6",
    "equalizer=f=125:t=q:w=0.9:g=4",        # chest weight at the fundamental
    "bass=g=3:f=110:w=0.6",                 # low shelf for body
    "equalizer=f=330:t=q:w=1.6:g=-2",       # de-box (narrow: a wide cut thins the voice)
    "equalizer=f=4000:t=q:w=1.2:g=3",       # presence / intimacy
    "treble=g=3:f=7000",                    # air
    "deesser=i=0.35",                       # tame the sibilance the HF lift adds
    "acompressor=threshold=-18dB:ratio=2.5:attack=15:release=250:makeup=1",
])

Clean the low end before adding weight to it; expand before compressing. After the fix, room tone sits 17.2 dB under the speech, better than the 14.2 dB of the untouched original.

Assembly onto the timeline asserts non-overlap rather than assuming it — the previous version had narr-16 running 1.14 s into narr-17 at 02:40 — and time-fits a line only when it genuinely does not fit its available room, capped at 1.12×.

Other ways to do this

Not tried — record line by line. Fifteen takes, fifteen files, narr-01.wav through narr-20.wav, and this entire section evaporates. There is no cut to make, no false start to detect, no energy gate to tune. The cost is studio discipline from someone who may not have any: stopping and starting fifteen times is a real imposition on a reader, and it flattens the performance because nobody builds momentum across a take they keep restarting. I chose one continuous read for the performance and paid for it in code. That was the right trade for me and might not be for you.

Not tried — cutting it by hand. Audacity, fifteen selections, fifteen exports. Twenty minutes for one language. My auto-cut is worth writing because there are two languages, six cuts and a near-certainty of re-recording — not because slicing a WAV is hard.

Tried, failed — chopping at fixed durations. The first version cut to fit the ten-second slots and left nine of thirteen segments starting or ending mid-word. Cut on measured silence, then fit; never fit first.

Tried, failed — Whisper large-v3 on Malayalam. No amount of parameter tuning fixes a model that does not know the language. For English it was excellent and free and I had no reason to use anything else.

A grandmother, a phone, and a room with a fridge in it

The Malayalam read did not come out of a booth. It came out of a phone, held at arm's length, in an ordinary living room, by someone who had never recorded anything before and was not going to be asked to do it twice.

That is the realistic case, and it is worth treating as the default rather than the exception. The performance in that file is the best thing in the project — the real read measures 10.0 semitones of pitch range against the clone's 5.1 — and every problem with it is fixable.

The first problem was not audio at all. The MP3 had a corrupt frame near the end of the file, and libsndfile refused it outright, which is exactly the wrong response to a recording that cannot be re-made. FFmpeg decodes straight through the damage with a warning and loses a few milliseconds:

def decode_source(dst):
    """ffmpeg, not soundfile: the mp3 has a corrupt frame near EOF that
    libsndfile chokes on but ffmpeg skips with a warning."""

Decode with the most forgiving tool you have, not the most correct one. A strict decoder on an irreplaceable file is a bug, not a feature.

Then the restoration chain, in this order:

highpass=f=85,
afftdn=nf=-32:tn=1,
adeclick,
equalizer=f=250:t=q:w=1.0:g=-2.5,
equalizer=f=3200:t=q:w=0.9:g=2.5,
deesser=i=0.35,
acompressor=threshold=-20dB:ratio=3:attack=8:release=120:makeup=2,
loudnorm=I=-18:TP=-2:LRA=9

Eight stages, and the order is the whole argument. Repair first, then subtract, then add, then control dynamics, then set level — because every stage amplifies whatever the stages before it left behind.

highpass=f=85. Nothing in a human voice lives below 85 Hz. Everything else does: the fridge compressor, traffic through a window, the low thud of a hand shifting on a phone case, and any DC offset the phone's ADC introduced. It is not audible on laptop speakers, which is why people leave it in, and it eats headroom that the compressor at stage seven will later spend gain amplifying. Cut it before anything downstream can act on it.

afftdn=nf=-32:tn=1. Spectral denoise, with nf set to the room's measured noise floor rather than a guess. −32 dBFS is what this room actually read between lines. Too low and the hiss survives; too high and you start subtracting the quiet tails of words, which is the underwater artefact everyone recognises and nobody can name. tn=1 turns on noise tracking so the filter follows the floor as it moves — a room is not stationary, and neither is someone holding a phone.

adeclick. Impulse repair: lip smacks, a fingernail on the phone case, the decoder's own stitching over that corrupt frame. It runs after the denoise deliberately. A click detector fed broadband hiss finds clicks everywhere; on a cleaned signal, a real transient stands out and gets repaired without the filter chewing on consonants.

equalizer=f=250 … g=-2.5. A phone held close to a face has proximity effect, and a small room piles its lowest modes into the same band. Both land around 200–300 Hz and both sound the same: boxy, muffled, close-but-dull. −2.5 dB is a small number on purpose. This band also carries the warmth that makes a grandmother's voice sound like a grandmother, and a 6 dB scoop takes the person out along with the room.

equalizer=f=3200 … g=+2.5. Consonant definition. This is where t, k and s live, where intelligibility actually comes from, and where a phone microphone is weakest. Note that it is the fifth stage, not the second: lifting presence before cutting mud means the compressor sees a signal with both problems still in it. Subtract, then add.

deesser=i=0.35. Payment for the previous stage. A presence lift is indiscriminate — it raises sibilance by exactly as much as it raises articulation. The de-esser gives back the harshness and keeps the clarity.

acompressor=threshold=-20dB:ratio=3. An untrained reader varies by 15 dB or more within a single line — leaning in on the emphatic words, dropping away at the end of sentences. A 3:1 ratio at −20 dB is enough to put every syllable in the same neighbourhood without flattening the performance. This one is not cosmetic: the final mix ducks music and clip audio via sidechaincompress keyed off the narration, and a sidechain key with 15 dB of internal variation ducks unevenly. Compressing here is what makes the ducking downstream predictable.

loudnorm=I=-18:TP=-2:LRA=9. Deliver at a known number. −18 LUFS and not the −16 of the finished film, because this track gets normalised again in the final mix and arriving pre-loud only means arriving pre-squashed. TP=-2 leaves true-peak headroom for the AAC encode, which can overshoot the sample peak. LRA=9 is the one to argue about: it is deliberately generous, because a bedtime story that has been levelled to a constant loudness is no longer a bedtime story.

Every stage in that chain exists to pay for the one before it. That is what makes the order load-bearing and not a matter of taste.

And then the finding that I did not expect and would put at the top of the post if it were not buried three thousand words in:

The phone beat the GPU. The best voice in this film cost nothing, needed no model, no CUDA wheels, no byte-length debugging and no licence review — 10.0 semitones of pitch range against the clone's 5.1, from someone who had never recorded anything before and did it once.

Every other section of this post is a machine standing in for a person who was not available. If the person is available, use the person. The restoration chain above exists to make an ordinary living room acceptable, and it is worth having precisely because it means you never have to say "we can't, we haven't got a studio."

Other ways to do this

Not tried — a $25 USB microphone. Most of that eight-stage chain is compensating for a phone held at arm's length. Any cheap cardioid on a desk removes the room before you have to remove it in software.

Not tried — cleaning it up by ear in Audacity. Noise reduction, a high-pass, a compressor, done in a GUI while listening. For one irreplaceable file that is the humane route; my chain exists because there were several files and I wanted the same treatment on all of them.

A rule worth taking regardless: decode an irreplaceable recording with the most forgiving tool you have, not the most correct one. libsndfile refused this file outright. FFmpeg walked through the damage and lost a few milliseconds.

The directory listing is the pipeline

You can tell how a project actually ran by looking at its folders.

Mine are numbered in pipeline order, and that is not tidiness — it is the working memory the agent read from and wrote back to for a month. This chapter is the shape of the thing, who really wrote it, and what the whole exercise cost.

Post-production folders are numbered in pipeline order:

00_DELIVERABLES/   six finished .mp4 + matching .srt
01_SCRIPT/         production pack, narration lines (machine), reading scripts (human)
02_ART/            characters/  items/  reference_photos/  contact-sheet.jpg
03_FOOTAGE/        shot-01..20.mp4 + _master-assembly-nofades.mp4
04_AUDIO/          music/  vo_raw/  vo_lines/  vo_mixed/
05_TOOLS/          the scripts
99_ARCHIVE/        superseded builds, scratch — safe to delete wholesale

Three conventions carry most of the value:

00 is what you hand someone; 99 is what you can delete. No judgement call about what is disposable six months from now.
Subtitles sit beside their video with a matching basename, so players load them automatically.
Zero-padded shot numbers everywhere — shot-01, narr-01 — so lexical sort equals story order in every tool, shell glob included. make-narration.sh iterates narr-*.wav and derives each line's timeline offset arithmetically from its filename: (N-1)*10 + 0.4. Filename discipline is what makes that a one-liner instead of a lookup table.

One more, learned the hard way and recorded in the README: do not move the virtualenv. Python virtualenvs store absolute paths.

The pipeline had a fourth author

Nothing in 05_TOOLS/ was typed by me.

The whole production ran out of a terminal: Claude Code, on Opus, sitting in the project directory with the numbered folders above as its working memory. No IDE, no notebook. The production pack in 01_SCRIPT/ was the shared document — I edited it, the model read it, and everything downstream derived from it rather than restated it.

What that actually bought me was the FFmpeg. Every filter graph in this post arrived as a first draft out of a conversation and was then argued into shape against real output:

make-music-bed.sh. I asked for a 200-second score and rejected the obvious answer, because looping one 30-second cue six times is audible. What came back instead was a seamless_loop function chaining copies through acrossfade=d=2:c1=tri:c2=tri, then four sections crossfaded into a bed that follows the story — 82 s adventure, 62 s wonder, 42 s triumph, 20 s home.
build-final.sh. The sidechain chain two sections up, including the detail I would not have found alone: that sidechaincompress consumes its key input, so the narration has to be split into two identical branches. The watermark got folded into the same -vf as the fades, so there is no second encode and no second generation of loss.
_watermark-vf.sh. Two drifting drawtext marks whose x and y are each a sum of two sines with incommensurate periods — 71.0 and 29.3 seconds on one axis, 61.3 and 19.7 on the other — so the marks never line up, with the amplitudes summing to under 1.0 so neither can wander out of frame.

And it absorbed the environment damage, which is the part that would otherwise have eaten a weekend. torchaudio wanted FFmpeg 7's shared libraries; this machine has FFmpeg 9, static — so a shim monkey-patches load and save onto soundfile and gets imported before anything else. IndicF5 ships a vendored f5_tts that collides with the pip-installed one, so its __path__ is pinned and then asserted at startup. A Windows console in cp1252 cannot print Malayalam, so stdout is reconfigured to UTF-8 on the first line of the script. None of that is interesting. All of it is load-bearing.

Where it was no help at all: Flow has no API, and pointing a browser automation at it was not something I was willing to do, so all twenty shots and all eighty re-rolls were clicked by hand. Every taste call was mine — which of the three MusicGen options, which re-roll to keep, whether the line-up actually looked like my children. And the arithmetic lies: I asked Veo for 8 seconds and got 10, which is why the master gets measured with ffprobe before the fades are placed instead of calculated as twenty times anything.

The honest sequence is that the pipeline was improvised, in-session, while the film was being made. It only became a system afterwards. Once the six cuts were rendered I turned the whole run into six Claude Code skills — film-00-new through film-05-post — sharing one film.json that holds the cast, the style block, the backend, and a state object recording which shots are done, which references exist, and what has shipped, so a run can be resumed rather than restarted. Two places it stops and waits for a human: after the script, and after the character line-up. Both are checkpoints this film taught me to want.

Everything in them is a mistake from this post, written down so the next film does not pay for it twice. Restate the costume in the text of every single shot, or you get an astronaut. Generate the group line-up first and cut the characters out of it, or the four-year-old comes back shorter than the two-year-old. Set Confirm before generating to Always. Shoot on Lite and spend the savings on re-rolls.

The film cost a month of Pro. The skills are what I got to keep.

Other ways to do this

Everything here is not tried — I used one agent for the whole film and cannot tell you how the others behave under this particular load. The category moved fast enough during production that the honest note is "check what these are today, not what a blog said in 2026."

Not tried — Codex CLI. OpenAI's terminal agent, Apache-2.0. The nearest like-for-like swap for what I did here.

Not tried — Antigravity CLI. Google's agent platform spans an IDE, a desktop app, a CLI and an SDK; the CLI is the part comparable to this workflow. Worth knowing if you are on Gemini: from 18 June 2026 Gemini CLI stopped serving Google AI Pro/Ultra and free individual Code Assist users, who are directed to Antigravity CLI instead — Code Assist Standard and Enterprise are unaffected. Unlike the rest of this list it is not published under an open licence.

Not tried — OpenCode. Open source (MIT), model-agnostic, terminal-first. Note the name collision: the archived opencode-ai/opencode repository is a different, discontinued project whose author moved on to Crush; the maintained one is sst/opencode.

Not tried — Aider. Apache-2.0, model-agnostic, git-native. It is built as an interactive pair-programmer rather than an unattended runner, so treat it as a different shape of tool rather than a drop-in.

Not tried — Kimi Code CLI. Moonshot's terminal agent, MIT. Its predecessor kimi-cli is being wound down. Separately, the Kimi Code API can be pointed at as the model backend for Claude Code, OpenCode and Codex, which is the cheaper experiment if you like your current agent and only want to change the model underneath it.

The thing that actually mattered was not the brand. It was that the agent lived in the project directory, could run FFmpeg and read its own output, and kept the numbered folders as working memory. Any tool with those three properties would have got most of the way here.

Costs

Item	Cost
Google AI Pro (1,000 Flow credits/month)	one month's subscription
Flow credits spent — 20 shots at Veo 3.1 Lite	200 of 1,000
Flow credits available for re-rolls	800 (≈80 re-rolls)
1080p upscaling, all 20 clips	free on Pro
Reference images — Gemini app free tier	free (separate pool from Flow credits)
MusicGen Medium score	free (local GPU)
F5-TTS English cloning	free (local GPU)
IndicF5 Malayalam cloning	free (local GPU, MIT)
ElevenLabs stock-voice narration	free tier
ElevenLabs Scribe transcription	free tier
FFmpeg assembly, mixing, encoding	free
Marginal cost beyond the subscription	zero

Everything except the video generation runs locally on consumer hardware. For anyone on the free Flow tier — 50 credits a day — the same film is five days of shooting at Lite rates with a handful of re-rolls, or about ten days if you want a real re-roll budget. The section above, Making this with no subscription at all, maps every stage onto a free substitute.

The genuine cost of this project was time: reference-image iteration, the Malayalam duration debugging, and the narration edit. Video generation was the cheap part.

Making this with no subscription at all

The one line in the cost table that deserves an asterisk is the first one. A month of Google AI Pro bought me wall-clock, not capability: 1,000 credits with no daily cap collapses the shoot from a two-week drip into two evenings. Nothing in the finished film requires it.

Here is the whole pipeline with the paid step removed. Everything marked not tried is exactly that — a route I believe works and did not walk, offered so you can start from a real map instead of from my one path.

Stage	What I used	Free substitute	What it costs you
Video	Veo 3.1 Lite, Google AI Pro	Flow free tier, 50 credits/day — same model (tried)	Five days of shooting, ten if you want re-rolls
Video, no Google	—	Open weights locally: Wan, LTX-Video, HunyuanVideo, CogVideoX (not tried)	No native audio. Twenty shots of foley and eleven lines of dialogue become separate jobs
Video, no GPU	—	Free daily credits on Kling, Hailuo, Pika, Luma (not tried)	Small daily quotas; check each service's terms on children's likenesses
Reference art	Nano Banana, Gemini app free tier	SDXL or Flux with IP-Adapter / InstantID, local (not tried)	Likeness is harder to hold; the line-up trick matters more, not less
Score	MusicGen Medium, local GPU	MusicGen Small on CPU; Stable Audio Open; YouTube Audio Library (not tried)	Slower, or cues that don't follow your beat map
Score, at all	—	Skip it — Veo's native audio carries the film (tried: the music-only cut)	Nothing, honestly
English narration	F5-TTS clone, local GPU	Piper or Kokoro, CPU, stock voices (not tried); or read it yourself into a phone (tried)	Stock voices aren't anyone's parent. Your own voice is better than both
Malayalam narration	IndicF5, local GPU	A family member and a phone (tried — and it won)	Nothing. This was the best audio in the project
Transcription	ElevenLabs Scribe, free tier	faster-whisper, local (tried, English only)	For Malayalam, nothing local worked for me
Cut, mix, encode, subtitle	FFmpeg	FFmpeg — or DaVinci Resolve free / Shotcut (not tried)	—
Hardware	RTX Blackwell, local	Google Colab / HuggingFace Spaces free tiers (not tried)	Session limits and queueing

Three honest caveats about the free route.

Leaving Veo is the expensive "free" decision. Every open video model on that list generates a silent clip. Veo handed me the springs, the hiccups, the orchestral swell on the reveal and every spoken line inside the same generation I was paying for anyway. Rebuilding that by hand is more work than everything else in this post put together, and it is the reason I would tell someone on a strict budget to use Flow's free tier slowly rather than a local model quickly.

The free tier is not a worse film. It is a longer calendar. Same model, same weights, same prompts, five credits a day fewer. If you are not in a hurry, there is no argument for the subscription at all.

The thing that is never free is time, and this post is mostly a list of the places I spent it. Reference-image iteration. A day on byte-length duration estimation. The narration edit. None of that gets cheaper on a paid plan.

What I'd do differently

Lia can now recite the narration before it plays.

That is the only metric that ever mattered here, and it is worth saying out loud before the corrections start. Everything below is what I would do differently — not because the film failed, but because most of these cost me a day each and they should cost you an afternoon between them.

Generate the two-character line-up first, not third. Together.jpg fixed the inverted sibling heights and the mismatched render styles in one image, and both problems only existed because the children were generated in separate passes. Generate the group shot first, then crop the solos out of it if you need them.

Write the costume description into the prompt from shot one. The spacesuit render cost credits and an afternoon. Ingredients bias the look; they do not lock it. Assume the reference contributes nothing the text does not also say, and you will never be surprised.

Check the model selector before the first generation. Flow defaulted to Omni 1.1 Flash, outside the credit-eligible Veo family and more expensive per clip. This is a thirty-second check that protects a month's allocation.

Prototype the non-English TTS on one line before building the pipeline. The 3-bytes-per-character duration bug would have surfaced in five minutes on a single test sentence. Instead it surfaced across a full batch of hallucinated narration. --test mode exists in make-narration-ml.py now; it should have existed first.

Cut the narration recording properly the first time. The initial version chopped at fixed durations to fit shot slots and left nine of thirteen segments starting or ending mid-word. The energy-first re-cut is objectively better and it was less work than the manual fixing it replaced. Automate the cut before you automate anything downstream of it.

Assert, don't assume, on the timeline. Two narration lines overlapped by 1.14 seconds for several builds before anyone noticed. Non-overlap is a one-line assertion. Add it.

Decide the watermark policy before the first deliverable, not after. The rule is one line — the mark goes on at the final encode and never touches the master — and it is trivially cheap to follow from the start and expensive to retrofit, because by then you have shipped files you can no longer prove are clean.

Test the transcriber on the target language before designing around it. Whisper large-v3 is the reflexive choice and it was the wrong one for Malayalam. Ten minutes of evaluation would have saved a day.

Cost the free path before buying the subscription. The Pro month bought two evenings instead of two weeks. That is a real thing to buy and I would buy it again — but I bought it before I knew that was all it was, which is a different and worse reason.

Record a human before you clone one. The best voice in the film came out of a phone held at arm's length in a living room with a fridge in it, and it beat the GPU on every measure I could think of to check. I built the cloning pipeline first and discovered that second.

Reference

Tool	Purpose	Link	Licence / cost
Google Flow	Video generation front-end, ingredients, upscaling	labs.google/flow	Free tier 50 credits/day; Google AI Pro 1,000/month
Veo 3.1	The video model itself (picture + native audio)	ai.google.dev/gemini-api/docs/video	Credit-metered; Lite = 10 credits / 8 s clip
Gemini API pricing	Credit and token rates	ai.google.dev/gemini-api/docs/pricing	—
Nano Banana (Gemini image generation)	Character sheets, prop ingredients	ai.google.dev/gemini-api/docs/image-generation	Free tier in the Gemini app; API metered
MusicGen Medium	Score generation, 1.5 B params	huggingface.co/facebook/musicgen-medium	Weights CC-BY-NC 4.0
AudioCraft	MusicGen's reference implementation	github.com/facebookresearch/audiocraft	Code MIT, weights CC-BY-NC 4.0
HuggingFace Transformers	How MusicGen was actually invoked	huggingface.co/docs/transformers/model_doc/musicgen	Apache 2.0
F5-TTS	English zero-shot voice cloning	github.com/SWivid/F5-TTS	Open source, free
IndicF5	Malayalam voice cloning, 11 Indian languages	huggingface.co/ai4bharat/IndicF5	MIT
ElevenLabs	Stock-voice narration; cloning evaluated	elevenlabs.io	Free tier excludes cloning; Starter $6/mo up
ElevenLabs use policy	Voice-cloning restrictions, minors	elevenlabs.io/use-policy	—
ElevenLabs Scribe	Malayalam transcription that worked	elevenlabs.io/speech-to-text	Free tier available
faster-whisper	English word-level timestamps for auto-editing	github.com/SYSTRAN/faster-whisper	MIT
Whisper large-v3	ASR weights (failed on Malayalam)	huggingface.co/openai/whisper-large-v3	Apache 2.0
PyTorch	GPU runtime; cu128 wheels for Blackwell	pytorch.org/get-started/locally	BSD-3
FFmpeg — filters	sidechaincompress, loudnorm, acrossfade, afftdn, adeclick, deesser, drawtext, overlay, scale2ref	ffmpeg.org/ffmpeg-filters.html	LGPL/GPL
FFmpeg — formats	concat demuxer, lossless assembly	ffmpeg.org/ffmpeg-formats.html	LGPL/GPL
libass	What burns an `.srt` into picture, when you insist	github.com/libass/libass	ISC
Nirmala UI	The Windows font with real Malayalam glyph coverage	learn.microsoft.com/typography/font-list/nirmala-ui	Ships with Windows

Alternatives named in this post that I did not use

Everything in this table is not tried — it is the map I wish I'd had, not a list of endorsements. Check the licence yourself before anything commercial; several of these ship permissive code with restricted weights, and the two are licensed separately.

Tool	Would replace	Link
Wan	Veo, locally, no native audio	github.com/Wan-Video
LTX-Video	Veo, locally, fast	github.com/Lightricks/LTX-Video
HunyuanVideo	Veo, locally	github.com/Tencent-Hunyuan/HunyuanVideo
CogVideoX	Veo, locally, modest VRAM	github.com/THUDM/CogVideo
FLUX	Nano Banana, locally	github.com/black-forest-labs/flux
IP-Adapter	Face/style conditioning for the character sheets	github.com/tencent-ailab/IP-Adapter
InstantID	Identity-preserving character generation	github.com/InstantID/InstantID
ComfyUI	The whole local image/video front end	github.com/comfyanonymous/ComfyUI
Stable Audio Open	MusicGen	huggingface.co/stabilityai/stable-audio-open-1.0
YouTube Audio Library	Generating a score at all	youtube.com/audiolibrary
Free Music Archive / Incompetech	Cleared library music	freemusicarchive.org · incompetech.com
Piper	F5-TTS, CPU-only, no cloning	github.com/rhasspy/piper
Kokoro	F5-TTS, small and natural	huggingface.co/hexgrad/Kokoro-82M
XTTS-v2	F5-TTS, multilingual cloning — licence needs reading	huggingface.co/coqui/XTTS-v2
OpenVoice	F5-TTS, tone-colour cloning	github.com/myshell-ai/OpenVoice
Chatterbox	F5-TTS, expressive cloning	github.com/resemble-ai/chatterbox
Montreal Forced Aligner	Duration estimation and subtitle alignment	montreal-forced-aligner.readthedocs.io
aeneas	Lighter-weight text↔audio alignment	github.com/readbeyond/aeneas
DaVinci Resolve (free)	The FFmpeg assembly and mix	blackmagicdesign.com/products/davinciresolve
Premiere Pro	The FFmpeg assembly and mix, with a timeline	adobe.com/products/premiere
Final Cut Pro	The FFmpeg assembly and mix, on a Mac	apple.com/final-cut-pro
CapCut	The whole edit, on a phone or the web	capcut.com
Codex CLI	Claude Code, in the terminal — Apache-2.0	github.com/openai/codex
Antigravity CLI	Claude Code — Google's agent platform, not open source	antigravity.google
OpenCode	Claude Code, open source and model-agnostic — MIT	opencode.ai
Aider	Claude Code, as an interactive pair-programmer — Apache-2.0	aider.chat
Kimi Code CLI	Claude Code, from Moonshot — MIT	github.com/MoonshotAI/kimi-code
Shotcut	The FFmpeg assembly, lighter	shotcut.org
Audacity	The narration cut and clean-up, by ear	audacityteam.org
C2PA Content Credentials	Provenance, where a visible watermark can't help	c2pa.org
Google Colab	The local GPU	colab.research.google.com

The acceptance test

Shot 20. The capes are on the bedpost. The moon is back where it belongs. The film is over in 200.04 seconds, which is roughly the length of a bedtime story.

Lia has now watched it enough times to recite the narration ahead of it.

Airik, who is two and does not yet care about ingredient drift or byte-length duration estimation, points at the screen and says his own name.

That was the acceptance test. It passed.

How this was made: the picture came from Veo 3.1 via Google Flow, the score from MusicGen, and the narration from two people reading into a phone. No video editor was ever opened — the film was assembled, mixed, subtitled and delivered entirely with FFmpeg scripts generated using Claude Code, running Opus. Every script named in this post lives in 05_TOOLS/, and the whole run has since been turned into six reusable Claude Code skills. The taste calls, the re-rolls and the voices are human.

DEV Community: Razi Kallayi

What Broke Automating Film Production with Claude Code and Google Flow

What Broke Automating Film Production with Claude Code and Google Flow

I made my four-year-old and my two-year-old into superheroes. Here is every single thing that broke.

Watch it first

What it cost and how long it took

What was automated, and what was not

Twenty shots, because the model can only count to eight

Turning two children into ingredients

The image that saved the film

Eighty re-rolls beat one pretty render

The setting that quietly costs you a month

Prompt anatomy

Why my son turned into an astronaut

The reference image was right there

Refused. Twice.

The one that was just my own fault

A score that follows the story

The voice you are not allowed to clone

English: F5-TTS

The ElevenLabs finding

Malayalam: IndicF5

Three times too long, and full of things I never wrote

Where it comes from

The fix

And then it came back

Two more IndicF5 mechanics

Fluent, and completely flat

Six films from one master

Picture

The mix

Subtitles nobody asked me to burn in

A watermark that will not sit still

Cutting a recording apart with a computer

Whisper vs Scribe on Malayalam

Nine of thirteen segments started mid-word

A grandmother, a phone, and a room with a fridge in it

The directory listing is the pipeline

The pipeline had a fourth author

Costs

Making this with no subscription at all

What I'd do differently

Reference

Alternatives named in this post that I did not use

The acceptance test