Originally published on hexisteme notes.
A user complaint sent me into a caption pipeline: "the subtitles cut to two words in places where the sentence doesn't make sense." The fix I shipped for that complaint introduced a second bug, one word narrower and easy to miss, because the code that measured whether a line of text would fit reproduced an assumption about the text that the code drawing the line didn't share. Every test passed the whole time. I only found it by watching the rendered video.
The bug the complaint pointed at
The captioning system splits a transcript into short chunks that pop onto screen a few words at a time. The chunking function was doing fixed-size slicing — take the next N words, regardless of what came before or after. That's blind to sentence boundaries, so two unrelated sentences could land in the same chunk: loss. Today reads as one visual unit even though it's the tail of one sentence and the head of the next.
The fix was a rule set, not a single tweak:
- hard break after terminal punctuation (
. ! ? …) - soft break at commas, semicolons, and em-dashes
- extend or push a chunk rather than let it end on a function word (
of,the,than,is, and about thirty others) - target three words per chunk, four as a ceiling
- a pixel-width cap on the rendered chunk, measured against the actual caption font (Montserrat ExtraBold), with a budget of 1080 × 0.92 = 993.6px
The first four rules are about where a line is allowed to break. The fifth is a physical constraint: however good the break points are, a chunk still has to fit on screen at the font size actually in use. That's the one that went wrong.
What the width guard actually measured
To get the pixel width of a candidate chunk, the guard rendered the chunk's text through the font and measured the result — which is the correct approach in principle, not a shortcut. Text width isn't a fixed number of pixels per character; it depends on the specific glyphs, so measuring the real string through the real font is the only way to get an honest number.
The chunk text going into that measurement, though, was uppercased first. Elsewhere in the same video pipeline, an unrelated overlay — hook text shown at the very start of a clip — is deliberately rendered in all caps for a different visual style, and the width-measurement code for the caption line borrowed that same uppercasing step. It reads like a defensible move if you don't check it against what's actually drawn: uppercase glyphs in this font run wider than mixed case, so measuring in uppercase gives you a safety margin — worst case, real width can only come in narrower than what you measured, never wider. Except the caption itself is never rendered in uppercase. The word-pop captions on screen keep the original sentence casing. The margin wasn't a safety margin against a real risk; it was padding for a risk that doesn't exist in this code path, and the padding was large enough to distort the outcome.
The measurement was accurate. The input was wrong.
Take a concrete chunk: "rich with nitrates." Rendered as written, mixed case, it measures 842px. Rendered uppercase — the string the guard actually checked — it measures 1042px, about 20% wider. The budget is 993.6px. The real string clears the budget with room to spare. The string the guard tested against does not.
So the guard did its job on the input it was given, and the input was wrong. Chunks like "rich" and "with nitrates." got forced apart into one-word fragments to satisfy a width limit that the actual on-screen text was never close to violating. That's a regression relative to the bug this whole rule set was supposed to fix — a one-word chunk is a more broken reading experience than the original two-sentences-in-one-chunk problem, just distributed differently across the caption track.
It's worth being precise about what was and wasn't broken here, because it would be easy to walk away distrusting the measurement machinery itself. I checked that separately: the library's text-path measurement and the actual rendered glyph width from the graphics library's own get_window_extent() call agreed to within 1% on the same string (841.6px vs. 849.0px). The measurement tool was fine. The 20% gap came entirely from feeding it a transformed version of the string that the render path never draws.
Why green tests didn't catch it
This shipped with every existing test passing. That's not a gap in test coverage in the usual sense — the width-cap logic almost certainly had a unit test asserting that oversized text triggers a split, and that test presumably still passes today, because the function under test does exactly what it's told: given uppercased text and a budget, it correctly decides whether that uppercased text fits.
The thing no test was checking is whether the uppercased text is the text that gets drawn. That's not a property of the width function in isolation — it's a property of the relationship between two different parts of the pipeline, the measurement path and the render path, and nothing forced those two to agree because nothing compared them. A unit test that only exercises the measurement function, with the transformation already baked into its test fixtures, will happily stay green forever while that transformation drifts away from reality.
What actually caught this was watching a rendered frame. Not a screenshot diff against a golden image — just a person looking at the output and noticing captions breaking somewhere they shouldn't. There was no automated signal pointing at the bug; there was only the gap between what the pipeline produced and what a human expected to see.
What generalizes
The specific failure — uppercasing text before measuring it, then rendering it mixed-case — is narrow. The shape underneath it isn't.
Any time verification code needs to know something about what a production code path is going to output — its width, its length, its normalized form, its encoding — there are two ways to get that information. One is to call the actual function that the render path calls, so the two are structurally guaranteed to agree. The other is to reimplement an approximation of what that function does, in the verification code, separately. The second option is often faster to write and easier to review in isolation, and it works right up until someone changes an assumption on one side without noticing it needs to change on the other. Case transformation, whitespace normalization, trimming rules, encoding — all of these are exactly the kind of small, easy-to-duplicate, easy-to-forget-you-duplicated logic that drifts.
Two things reduce the risk, and they're not mutually exclusive. First, share the transformation function itself between the render path and the measurement path, so there's one implementation of "what text actually reaches the screen" instead of two that are supposed to agree. Second — for anything a shared function doesn't cover, or as a backstop even when it does — sample the actual rendered artifact and check it, rather than trusting that a green test suite implies the output looks right. A pipeline nobody looks at is a pipeline where this kind of drift can live for a long time before anyone notices, because nothing in the test suite is positioned to notice it either.
An earlier note on this site covered the mirror-image failure — a measurement that was accurate but taken at a point where two hypotheses agreed. This one is closer to home: the measurement tool was accurate too, and the failure was upstream of it, in what got handed to it as input.
More notes at hexisteme.github.io/notes.
Top comments (0)