Before anything else, the part that's still embarrassing at publication time: the broken version I'm about to describe is, as of this writing, still the live action.yml on our main branch. The fix exists, it's verified, and it hasn't landed on the public repo yet. You can open the file and read the exact defect I'm confessing to. I'd rather publish in that state than time this post to land after the cleanup and pretend the window never existed.
The project is a reversibility gate for agent tool calls. Its one-sentence pitch is fail-closed: when the machinery can't verify something, it stops, it doesn't wave things through. We say this in the README, in the limits page, in the receipts design. It's the property everything else hangs off.
The repo root also carries a composite GitHub Action, meant to let CI verify execution receipts. During a triage pass over our own issue tracker on 2026-08-31, I actually read it. The verification step does this: if the gx binary is on the PATH, run the verifier. Otherwise, print "Tracefold gate validation passed (stub check)." and exit 0.
Exit zero. On the branch where the verifier doesn't exist.
It gets worse before it gets better
The other branch had its own problem. The command it invokes, gx verify-receipts, isn't a subcommand our CLI ships. The real one is gx receipt verify, which our own example workflow in the same repo calls correctly. So with the binary installed, the step fails on an unknown subcommand.
Put the two branches together and the actual behaviour of the shipped action was: fail when the verifier is present, pass when it's absent. The only reachable green was the one that means nothing. A fail-open stub, in the repo whose entire thesis is fail-closed, wearing the word "passed" in its output.
Nobody attacked anything. Nobody even hit it, as far as I can tell; the action isn't published to the marketplace and the issue asking to make it real is still open. But "nobody used the broken thing yet" is luck, not a property.
The minimum honest fix
The proper fix is blocked on shipping pre-built binaries, which is its own open issue. What could be fixed immediately was the lie, so that's what the patch does, and only that.
Binary absent: the step now prints an error saying the check cannot pass without the verifier, and exits 1. The "passed (stub check)" line is retired, kept as a comment in the file for the record rather than deleted, so the history of the defect stays readable in place.
Binary present: the invocation is deliberately unchanged, with a comment stating the known limit, that the subcommand is wrong and even an installed gx fails this step today. That branch was already fail-closed by accident. The comment just makes it fail-closed on purpose.
Verifying a YAML fix is awkward, so the test extracts the step's script and runs it under bash -e -o pipefail in both states. Absent: exit 1, error line present. Present: a small recorder stands in for gx, confirms it receives the same arguments as before, and its exit code propagates, tested with 0 and with 7, the code our verifier uses for a receipt that fails verification. Both branches now end in a truthful exit code.
What I take from finding this in my own repo
The stub didn't sneak past review because it was subtle. It's four lines. It got in because a scaffold file was treated as packaging rather than as code, and packaging doesn't get adversarial reading. But CI config is exactly where fail-open hides best: a green check mark looks identical whether it was earned or defaulted, and nothing downstream distinguishes them. If your pipeline has a step that can pass without its tool being installed, you have this bug too; ours just had the bad taste to print the word "passed" while doing it.
The disclosure went on the public issue the same day, before the fix, and this post is the longer version. The acceptance criteria for a real action, install the binary, propagate the verifier's exit code, remain unmet and tracked openly. The only thing removed so far is the false green, which is the part that couldn't wait.
Code is at github.com/TraceFold/tracefold, Rust, Apache-2.0. Issue #5 has the timestamped trail.
Not released as a package. The v0.1.0-alpha tag picked up a single Linux x86_64 tarball on 2026-08-31, built outside CI. The action still doesn't install it, so the acceptance criteria above stay open.
Top comments (2)
Fail open stub is bad, but the shape under it is more general and worse. Any branch you add to make one environment convenient becomes the branch nothing exercises, and the test file still looks complete, because the tests go through that same branch. In my Go project a constructor delegated to a version with injectable timeout, every test built through the injectable one, so the production default had zero coverage while four confidently named tests stayed green. Only mutation testing found it, reading the test file would never show it, because what is missing does not appear anywhere.
"the test file still looks complete, because the tests go through that same branch" is the sentence that gets me. Same shape hit me this week, different domain: a rehydrate function needed to rebuild a value it had hardcoded to a placeholder since the feature that would fill it in didn't exist yet. No caller ever disagreed with the placeholder, so cargo check stayed green and nothing in the suite exercised the branch where they'd differ. It only broke once a real code path started producing a real value and the two sides stopped agreeing by accident. Two targeted probes caught it, not the full suite. Mutation testing is the sharper tool here for exactly the reason you're pointing at: it doesn't ask does this pass, it asks does anything notice if I break this, which a complete-looking test file can't answer about itself.