Three of my test gates had never heard of the game I was shipping

My browser arcade at play.dhseadev.online deploys when I push to main. Nothing ships unless verify-all.sh prints ALL GATES PASS. This week I was adding a ninth entry, a card game called Prismwar (not live yet). I built it, the index listed it, dist/ contained it.

Three of the gates in that suite had no idea it existed.

The list that didn't grow

The arcade is driven by one table, ENTRIES. The build reads it, and so do the index, the sitemap and llms.txt. Three test scripts didn't: the boot/paint/save gate, the stress battery and the SEO-heading gate. Each one had its own hardcoded list of pages, written back when the arcade was smaller.

So the new row got built, emitted and linked, and none of those three scripts ever loaded it. They reported on everything they knew about, and all of it passed. Nothing in the output was false. It just described a smaller arcade than the one I was shipping.

// what the gates had (simplified)
const PAGES = ['prism-cascade', 'lumenreel', 'veilfall' /* ...frozen in time */];

// what they should have
import { ENTRIES } from '../build.mjs';
const PAGES = ENTRIES.map(e => e.id);

For today I added the missing rows by hand. After that the full run included Prismwar and passed. The real fix is deriving every gate's list from ENTRIES, so a new entry can't be built without also being tested.

This wasn't a one-off. The same week, an idle game I'm building for the arcade gave me five more green results that weren't telling the truth.

A sabotage that everything catches proves nothing

Ashline, an idle auto-battler, has an "instrument proof": break the engine in specific ways on purpose, then confirm the balance suite goes red for each one.

The first run caught four sabotages out of four, and every one reported the same thing: the army stalled at wave 1.

That's what gave it away. Four different breakages shouldn't produce one identical symptom. The unbroken game was also stuck at wave 1, because of a cold-start bug where the starting army did zero damage. The suite wasn't detecting my sabotage. It was detecting a softlock that was already there, four times over.

The fix was a control. Run the unbroken build first and require it to pass. A sabotage only counts as caught if the baseline was green and the broken build fails a check that fits the breakage. After that, each sabotage tripped its own band.

Buttons that exist vs. buttons you can press

Ashline's DOM smoke test was 13 for 13: boot, five buy buttons, accessible names, HUD updates, save writes, offline settle, graceful storage failure.

I opened the game and saw a line across the screen with nothing to click.

On load, the game called buyLoop(). That's the automated player the simulation uses in place of a human. It spent the opening currency before first paint, so every buy button rendered disabled. The test checked that the buttons existed and had names. It never checked whether anyone could press one.

I added four assertions:

• an action is available at load
• a purchase is affordable at load
• the starting grant hasn't been spent
• clicking alone clears wave 1 within 12 seconds

Then I put buyLoop() back into load() on purpose. The suite dropped to 15/17, failing on exactly the two grant checks. That's when I started trusting it.

Three asserts that couldn't fail

The quieter version of the problem is an assertion that can't go red at all:

• A gravity check for particles that returned true no matter what. It never read velocity before and after a step. Now it does, and it measures +90 vy per 0.1s at G=900.
• check('champion drawn larger', true). A literal. I wrote it, and it would pass on a blank canvas forever.
• A reduced-motion check that passed for the wrong reason. The attack has a 250ms cooldown, so the second call was refused, and "motion on" and "motion off" looked identical. Once I fixed that, the test failed on the good build. The harness pins Date.now() to 0, so the cooldown never cleared. That's two harness bugs and zero game bugs.

The literal became a spy that records what the renderer actually received:

// passes forever
check('champion drawn larger', true);

// passes only if the renderer actually did it (simplified)
const heights = [];
const draw = Champion.draw;
Champion.draw = (ctx, x, y, h, ...rest) => { heights.push(h); return draw(ctx, x, y, h, ...rest); };
renderLine(); renderArena();
check('champion drawn larger in arena', heights[1] > heights[0]);

The real numbers are 52px on the battle line and 86px in the arena.

The copy that was a version behind

Ashline also has a delivery-path check. It runs the balance bands against the engine extracted from the built HTML, not the source module, because the HTML is what players get.

Good idea, bad execution. My extracted copy of the sim was stale. It predated the army-combat rewrite, so it reported 7 bands under an old label when the real suite had 10. Now both derived files are regenerated from the build on every run, and a copy is never reused.

What I check before I believe a green run

1. Does the gate's inventory match the product's? Compare what it tested with what shipped.
2. Was the control green, and did each break fail differently? If every break fails the same way, you're looking at one bug that was already there.
3. Does the assertion test use, or just existence? "The button exists" and "the button can be pressed" are different tests.
4. Can I make it fail? Break the thing on purpose. If the test stays green, the test is the bug.
5. Is the artifact current? Regenerate anything derived. A stale copy tests yesterday's build.

Then open the thing. A gate caught every problem above except the unplayable build. I found that one by clicking on it.

A passing gate is making a claim. The only way I've found to check that claim is to watch the gate fail first.