A "swipe" on the main video-generation service takes a reference ad and reproduces its structure — cuts, pacing, who's on camera when, where the product shows up — with new creative underneath. An editor's review of an early August run put it plainly: "the previous swipe pipeline beautifully replicated the scenes as per the original swiped video… now we are not even replicating the scenes." That one sentence opened a monthlong arc of thirty-odd PRs, because "doesn't replicate the reference" turned out to be four or five independent defects wearing the same coat, each one only visible once the others were peeled back.
A regression report that names one symptom can be several bugs at once. Fixing the first one you find just changes which of the others becomes visible next.
Who's allowed to be a presenter
The costliest bug in this arc took three separate PRs to actually kill, because each fix stopped short of the next layer underneath it. A short test on one reference derived fifteen presenters and zero narrated scenes from a reference that contained not one talking-head shot — turning a voiceover documentary-style ad into a wall-to-wall selfie monologue. PR #322 found the immediate cause: the code was reading "the reference has speech in this window" as "the person on screen is speaking," and in a voiceover ad every window has speech, so every window looked like a presenter shot.
Fixing that surfaced the second layer. PR #324 found that a cutaway window correctly returning "no signal" for its role was falling through to the planner's own guess — or a literal hardcoded string, "presenter" — because the fallback chain had nowhere sensible to land. A reference built from a mascot, seven live-action b-roll shots, four CGI shots, and an end-card, narrated throughout, with not one talking head, still came back with seven presenter scenes.
A plan-gate sweep of all six references — four dollars and change, no clips rendered — found the third layer, and it's the one that explains why the first two fixes hadn't visibly worked: the actual resolution order was refRole ?? p.role ?? planHumanRole. On a cutaway window, the reference-derived role is undefined by design — a cutaway says nothing about who's in the scene beneath it — so the planner's guess was winning the coin toss before the corrected default ever got consulted. Three PRs, three fixes, and only the third one actually changed the resolution order that mattered. The first two were both correct and both insufficient, which is a harder thing to notice than being simply wrong.
A word budget measured in someone else's voice
PR #332 traced the ad's pacing drift — consistently too slow, consistently running long against its reference — to a single parameter with an easy-to-miss error: the per-scene word budget was computed at the reference speaker's words-per-second, not the pipeline's own voice model's rate. A reference narrated at 3.6 words per second produced a script sized for 3.6 words per second, delivered by a voice that speaks at 2.75. That script cannot fit by construction — no downstream fix recovers it, because trimming copy is explicitly banned and the audio time-stretch is capped well below what the gap would require. The picture just gets sized to accommodate copy that was never going to fit its own clip.
The dead-air gate meant to catch under-length lines had the identical bug from the other direction. PR #336 found it demanding nine words for a three-second scene, when that scene's actual speakable budget — at the correct voice rate, accounting for a mandatory breath — was seven. A line written exactly to the correct budget was being flagged as a defect, while a line that passed the check couldn't actually fit its clip. Two mechanisms measuring the same quantity with two different, uncoordinated formulas, and each one invisible precisely because the failure only shows up as "the ad feels slow," which is exactly the vague complaint that started this whole arc.
The bundle the planner had been building and throwing away
This is the single most expensive bug in the batch, in the literal sense: it made every clip cost more than it needed to for two weeks before anyone found it. Kling's three-second minimum clip length applies per generation, not per shot — but every scene was rendered as its own generation, so every scene paid that floor individually, even when the reference cut it was matching lasted under a second. A 2.2-second reference cut became a 3-second scene. Stacked across an ad, that's the exact mechanism behind "pacing doesn't match the input video": a 7.9-second sequence in the reference rendering as 15 seconds of ad.
PR #318 found the fix already half-built and entirely unused: planMultiShotGroups had been computing which scenes could share one generation — and persisting that computation to the job record — on every single run since August 12th. It had zero readers. The bundling logic existed, ran, cost nothing extra, and its output sat in the database unconsumed for a fortnight while every scene kept paying the floor it was supposed to eliminate. PR #330 is the follow-up that actually wires the bundle into rendering, and the header number makes the stakes clear: every output had been overrunning its reference by 30 to 60 percent against a 5 percent target, entirely from this one mechanism.
Bundling correctly meant answering harder questions honestly rather than guessing. PR #413 ran six paid probes specifically to establish where a cut actually lands inside a multi-shot generation — a claim in the code's own comments turned out to be wrong once measured against real frames, with sub-three-second shots silently merging into one continuous take with no cut at all rather than the hard cut the code assumed. PR #433 found the operational cost of bundling on the back end: once a scene's picture comes from a shared bundle rather than its own generation, a "redo" request for just that scene has nowhere to send its edited prompt, and a same-vs-different comparison used to gate a re-render was reading a stale canonical clip and silently discarding paid takes that were, in fact, better than what shipped.
Color that drifts, and a channel that was never right
An ad assembled from N separate Kling generations is N separate video shoots as far as color is concerned — each clip drifts from its own starting frame independently, so a finished ad reads as clips from different cameras on different days. PR #316 built shot-to-shot color matching for exactly this reason, on the explicit principle "balance first, look second": an earlier attempt at a stylistic look — grain, vignette, contrast, gamma — had been rejected because applying a look on top of an unbalanced base just multiplies the differences that were already there instead of hiding them.
PR #343 found that the color-matching pipeline itself had a channel-order bug hiding underneath a plausible-looking filter chain: the source was converted to a planar pixel format ordered green-blue-red, and the color-lookup filter applied afterward indexed its lookup table assuming red-green-blue order. A correction authored in RGB was being read in GBR, so every color adjustment landed one channel over — measured directly on a known input color, halving the wrong channel entirely. Two independently correct-looking pieces of a pipeline, composed in the wrong order, producing a result that was confidently, specifically wrong rather than merely approximate.
The reference decides, not the script
A recurring theme across this whole arc is a planner making decisions from its own script when it should have been deferring to the reference it was supposed to be copying. PR #334 found that automatic cutaway insertion read only the script's own dialogue and authored brand-new spoken lines for each cutaway it inserted — after the plan-gate pacing check had already run, so the newly authored words were invisible to the very check meant to catch pacing drift. PR #344 names the rule this violated directly: nothing should author spoken words the editor didn't write, full stop — authoring copy is the planner's job at plan time, not something a downstream pass gets to do quietly on its own initiative afterward.
PR #345 closes the loop on where authority over length actually sits: the reference supplies structure — beat count, cut placement, who's on camera — but an ad-length ceiling constant had been silently bounding the rendered picture through the timing engine's own scale-and-fill logic, while nothing anywhere was shortening the words filling that picture. The fix is a statement of priority as much as a code change: the script is the ad's ground truth for length, and a numeric ceiling doesn't get to override that by squeezing the frame around copy it never touched.
A gate that measured a plan nobody shipped
PR #333 is the finding I'd flag as the sharpest in the batch, because of what it says about validating a check rather than just the thing it checks. The plan-gate pacing report was computed from the planner's raw output — correctly, at the moment it ran — but two more passes happened after that checkpoint: automatic cutaway insertion, and a product-reveal backstop that could rewrite other scenes. The gate's own persisted numbers described a draft that had already been discarded by the time the ad actually rendered. One reference's gate reported an 11 percent overrun; the shipped ad actually ran 40 percent over. A pacing gate that measures the wrong version of the plan is worse than no gate at all — it actively tells an editor the ad is fine while the ad that ships is not the ad it measured.
The pattern: fix what the ad actually became, not what it was supposed to be
Almost every defect in this arc has the same shape: something measured, gated, or authored an earlier version of the ad than the one that actually shipped. A gate that checked a discarded draft. A word budget computed in a voice nobody was using. A bundling optimization that ran and persisted and never got read. A color correction authored in the right color space and applied in the wrong one. None of these needed a better model or a smarter prompt. They needed someone to measure the artifact that actually left the building, at the moment it left, instead of trusting that an earlier, cheaper checkpoint upstream still described it.
Top comments (0)