Quick take
Quick question. That guard you added to CI last month — have you ever watched it refuse anything?
Most of us test in one direction. We prove the thing works. We almost never prove the thing can fail.
So a guard gets written, the pipeline goes green, and everyone moves on. Green means the guard passed. It does not mean the guard would have caught anything. Those are different sentences, and only one of them is on the screen.
I ran this check on my own repo. Four guards. One of them had been green since the day it was merged, for a reason nobody enjoys hearing: it compared the wrong two values, so it agreed with everything.
The fix is not clever. Before you merge a guard, break something on purpose and watch it turn red. Thirty seconds, once, forever:
# 1. break the thing the guard protects
sed -i 's/EXPECTED/WRONG/' config.yaml
# 2. the guard MUST fail here
npm run guard && echo "the guard is decoration" && exit 1
# 3. put it back
git checkout config.yaml
If step 2 prints that line, you did not ship a guard. You shipped a decoration with a green light attached.
The long version — four guards, four different ways to be green and useless, and the one question missing from most acceptance criteria — is here: My own CI gate rejected me 4 times.
I build cachly — persistent memory for AI coding assistants, over MCP. Your assistant re-reads your codebase every morning. It does not have to.
Free tier, hosted in the EU: cachly.dev
Top comments (2)
A gate that never says no is usually documentation, not enforcement. I like testing gates with known-bad fixtures the same way you test auth with denied cases. Otherwise the team only knows the happy path is wired.
"Documentation, not enforcement" — I'm stealing that. It names the thing better than anything in my post.
Your fixture point pushed me to actually measure it rather than eyeball it. Every gate run in our ledger records a per-check result, so I asked one question across all of them: when did each check last reject something?
build turned out to be fine — I broke a Go file on purpose and it exited 1, exactly as it should. testTask did not. It skips when the task has no acceptance file, and 87% of ours don't, so in 50 of 64 runs it tested nothing and reported green. Not a wrong comparison like the one in my post — an opt-out that got recorded as a pass.
That's the version of your idea I'd now recommend to anyone: known-bad fixtures when you write the gate, and a "days since last no" column for the ones already running. The second one is nearly free if your CI records per-check results — you're reading data you already have.