The Assert Counted 100 Dots. The Screen Showed Two.

Originally published on hexisteme notes.

I had one requirement for a 45-second statistics short about the inspection paradox: everything the episode claimed had to be countable on screen. The claim was a bus schedule with gaps of 2, 2, 2, 2, and 12 minutes — the 12-minute gap alone is 60% of the full 20-minute cycle, so a passenger arriving at a random moment lands inside it 60% of the time. The proof, visually, was 100 dots for 100 passengers, 60 of them sitting on the long gap.

I delegated the scene code to a subagent. It came back green on every check I had, and the rendered frames showed nothing to count.

Every gate came back green

The report listed, truthfully:

• [VERIFIED] on all 9 constants — samples_per_gap=(10,10,10,10,60) asserted at import time as computed, not hardcoded (10 per short gap, 60 on the long one)
• assert len(dots) == 100, passing
• [SAFE-MARGIN OK] 18/18 — the worker added the left/right check itself, catching 3 real overflow bugs that had been shipping silently
• zero errors, warnings, tracebacks
• exact fraction arithmetic (Fraction) so gap-boundary timestamps couldn't misclassify on floating-point rounding

Every line was true. I opened the frame: of the 100 dots the assertion confirmed existed, 2 were actually identifiable. The other 98 did nothing a viewer could see.

The bug was one line

color = LONG_AMBER if idx == LONG_IDX else SHORT_GRAY
dot   = Dot(point=pt(time_to_x(t_minutes), TIMELINE_Y), radius=0.045, color=color)

Every dot was painted the same color as the band it sat on, at TIMELINE_Y — dead center of that band. Amber dot on an amber band, gray on gray.

All 100 were exactly where they belonged: right count, right coordinates, right color rule — long gap amber, short gaps gray, exactly as designed. Every one was rendered, with total precision, in camouflage. The 2 that survived straddled a color boundary, where the background under them changed.

What each gate actually asked

Check	What it asked	Result
assert len(dots) == 100	Does it exist	Pass
samples_per_gap constant assert	Is the value correct	Pass
assert_within_safe_margins	Is it inside the frame	Pass
Renderer exit code	Did it crash	Pass
Code review	Is the logic correct	Pass
A human looks at the frame	Can you see it	Fail

Five of six gates were green, and none of the five had asked whether anything was visible — they couldn't. Every one inspects the scene graph: does the object exist, where are its coordinates. The defect lived in the rendered pixels, one transformation downstream of all five.

assert_within_safe_margins is worth a closer look — a genuinely good check that caught 3 real bugs. But its name promises "stays inside the margin," and a passing report reads it as "laid out correctly." The bug lived in that gap. Elsewhere in the same render, three text elements overlapped until letters ran together, and the margin check passed in full — it was never built to check whether two elements' boxes intersect, only their distance from the frame's edge.

Once is an accident, twice is a pattern

This was the second time this shape showed up in the same project.

The first, on 2026-08-23, was a generated video shot meant to show dice pips. The generator couldn't control how many pips it drew, so the contract got rewritten to ask for a blank face instead — and QC scored the blank output as a pass. The episode's claim was about pip counts; the screen showed none. The checker asked "does this match the contract" — the contract had already quietly dropped what the episode was supposed to prove.

Different surface — one contract abandoned its subject, one render hid it — same structure: the question a checker asks and the proposition that has to be true for the artifact to hold are two different questions.

The defense built afterward, serves_line, makes a shot's contract declare which sentence it enacts, and throws at load time if the contract bans that sentence's core nouns — two deterministic rules, no LLM call. It blocks an exact repeat of incident one. It did not block this one — sentence and contract agreed completely; the defect was one layer further down, where serves_line has no visibility.

Push a defense down one layer, and the next defect moves down with it.

Watch first, then encode what you watched

The fix isn't "add more checks" — another check at the same layer reproduces the same blind spot one layer lower. Order matters:

A human looks first. A green report doesn't substitute for opening the render — that gap only closes when someone who knows it exists looks, once, at the actual output.

Then what got seen becomes an invariant — observation first, automation second. Two got planted here:

• A contrast invariant: every dot's color must differ from whatever it sits on — band fill on a band, background on the background. An uncountable count isn't a count.
• An overlap invariant: assert_no_overlap(a, b), forbidding bounding-box intersection between text elements sharing the screen. The margin check only ever looked at the frame's boundary, never elements against each other.

And doubt how much a checker's name promises. assert_within_safe_margins is honest — it says exactly what it checks. The failure was reading "passes the margin check" as "laid out correctly." A green light isn't evidence until you've confirmed its scope.

Where the same gap hides off-screen

Manim is the domain here, but the shape isn't specific to it — it shows up in any pipeline emitting a visual artifact: chart, PDF report, UI screenshot, game frame, dataviz panel.

It shows up outside rendering too:

• Logs/alerts — event recorded, not whether a human receives it. An alarm that's always ringing is no alarm.
• Error handling — exception caught, not whether it's visible to the user after. A swallowed exception leaves a passing test.
• Accessibility — element in the DOM, not contrast ratio or screen-reader order. This bug is a literal 1:1 contrast accident.
• Documentation — section exists, not whether it's findable.
• Monitoring dashboards — metric collected, not rendered on the panel.

Automated checks read the representation inside the pipeline; a human reads the artifact at its end. Any transformation between the two stays unchecked.

Hours later, my own caveat stopped being true

I'd written down a condition that would prove "human looks first" wrong: a cheap automated check that inspects the rendered artifact itself and catches this class ahead of a human. I noted, honestly, that I hadn't tried one.

That stopped being true within hours. The same afternoon, wiring a new scene type that ingests externally rendered clips, I added a publish-gate check that opens the clip's hook frame with ffmpeg, measures RGB stddev and unique-color count, and fails near-monochrome frames. Not scene-graph — rendered pixels.

It caught a real defect on first use:

t= 0.00s  stddev=   0.000  unique_colors=     1   <- pure black
t= 0.25s  stddev=   7.674  unique_colors=    17
t= 1.00s  stddev=  13.915  unique_colors=   136

The scene was fading in from black — in a short-form feed the hook frame is where a scroll stops, so this was real, and the machine caught it before I saw the frame myself.

Did that flip the lesson? No — it made it more precise. That check exists because of the dot-camouflage incident. I'd specified that this scene type's hook check shouldn't auto-pass like the others — it had to grab a real frame and measure it. Without that instruction, a bare pass: True would have gone there instead, as it did for the sibling type next to it. Watching first produced the invariant, and the invariant caught the next instance before I did.

Then it passed a 10.5-second hole anyway

Later that afternoon, the hook check failed a clip, the worker fixed it, and the check passed — and the clip that had just passed still had 4.5 seconds of blackout in it. A separate re-measurement of the copy actually live in the episode found something worse: 10.5 continuous seconds empty, 0.25s to 10.25s. The hook check had caught only the leading edge of a longer hole. It touched the mouth, never the hole.

The third incident landed at the exact layer where the invariant lived.

The location was right — open the pixels, don't trust the scene graph. What was wrong was what got measured. A human judges occupancy: how much of the screen is filled. The check measured contrast: is the frame monochrome. They don't move together — one white line on a black background looks, to a human, about as empty as a frame gets, while variance goes up.

Two counterexamples from the clip make it concrete (1080×1920 @ 30fps, 98 samples):

Time	What a human sees	stddev	Unique colors	Bright pixels	Existing check
t=8.0s	Just one horizontal line	17.838	1258	0.613%	Pass
t=45.5s	Completely black screen	2.648	22	0.000%	Pass

The second row is the whole problem: a frame with zero bright pixels slides under stddev <= 2.0 because compression noise alone nudges variance up. Denser sampling doesn't fix a wrong ruler.

The replacement metric came from the same 98 samples, empirically: 18 dead samples topped out at 2.766% bright pixels, 80 live samples started at 3.174%, nothing in between — threshold set at 3.0%, inside that gap.

Still a check built from one clip. No evidence the split holds on other scenes; 3.0% is what separates two groups in this sample, not a validated constant.

What actually held

For the camouflaged dots and the 10.5-second hole, what finally caught the defect was a human looking at an actual frame; the blank dice faces sailed through QC the same way. Automation helped: the hook check does catch the one property it was built for. But the order needs one more clause:

An invariant has to measure the same quantity a human actually judged, not a proxy that happened to agree on the frames already seen. A check on the wrong quantity is worse than no check — "no check" is visible, and a green light on a wrong quantity isn't.

This isn't the only way "checks passed" and "visible" come apart in a render pipeline. A few near neighbors, same family of bug: The AI Can't See What It Drew is about an agent with no way to look at the image it just produced; my caption-width guard measured a text property the renderer never produced; a collapsed canvas traced dead space back to one rendering parameter reshaping the frame. This one sits on a different axis: the check and I looked at the same rendered artifact and asked two different questions of it. Existence and visibility are separate predicates over the same thing, and passing one tells you nothing about the other.

Three incidents in, the checks keep getting sharper about the thing that just bit me. None can tell me, in advance, which quantity I'll measure wrong next — that still needs a person looking at a frame.

More notes at hexisteme.github.io/notes.