Every AI-and-engineering post right now is about adding a guardrail.
Lint rules the agent can't bypass. Evals before you ship a prompt change. A review bot on every PR. A regression suite built from real production failures. Architecture rules checked into the repo so the model reads them on the way in.
I have written some of those posts. I still believe in them. This is the work that makes AI usable by senior engineers instead of a liability.
But I have been in three separate conversations on this platform in the last week that were all, underneath, about the same thing, and it isn't about which guardrails to add.
A guardrail that has never fired and a guardrail that silently stopped running produce identical output.
Green.
The pipeline that had never been green
Vicente Reyes had a GitHub Actions workflow called Deploy to DigitalOcean. Fully wired. SSH action, secrets, the works. And every time he shipped a backend change he still SSH'd into the droplet and ran git pull by hand.
The deploy job was gated on CI passing.
on:
workflow_run:
workflows: ['CI']
branches: ['main']
types: [completed]
jobs:
deploy:
if: ${{ github.event.workflow_run.conclusion == 'success' }}
Sensible enough. The problem was that CI had never once gone green on main. Not flaky. Never. So every deploy run showed skipped, forever, and the pipeline sat permanently behind a gate that could not open.
His line about it is the one worth keeping. A pipeline that is silently and permanently blocked looks, from a distance, exactly like a pipeline that doesn't exist.
Nothing in the Actions UI says this workflow has not succeeded in forty runs. You have to go and ask.
The grader that can't say no
On a thread about designing trustworthy AI evals, Heinrich Neb made the point that every grader needs a known-bad twin. An input it is supposed to reject, plus a recorded date of when it last actually rejected something.
His framing is the sharpest version of this I have seen: a grader that has never failed and a grader that silently stopped running print the same green.
Same failure as Vicente's pipeline, one layer up. In his case the gate was stuck closed. In an eval suite the gate is stuck open, which is worse, because a stuck-closed gate is annoying enough that somebody eventually investigates. A stuck-open gate just keeps saying yes.
Think about how an eval suite actually rots. Somebody changes a prompt template and the grader's regex stops matching, so everything scores as pass. Somebody renames a dataset field, the loader returns an empty list, the suite runs zero cases in 0.4 seconds and reports 100%. A provider changes a default and your grader model gets more agreeable.
All three look like success.
The score with no provenance
The third one is mine, from the same thread.
An eval result with no harness version, no dataset snapshot and no prompt revision attached to it is not evidence. It is a self-reported claim.
Which is fine right up until the number moves. Then somebody asks whether the model got better or the suite got easier, and if you can't answer, you never had a measurement. You had a vibe with a decimal point on it.
That reflex comes from working in payments. In a regulated system nobody asks you to trust that a control ran. They ask you to demonstrate which control ran, on what input, at what time, under which version of the rules. Months later. To somebody who wasn't there and isn't inclined to take your word for it.
Engineering has quietly inherited that burden. Evals are increasingly the artefact a shipping decision rests on. They just haven't inherited the paperwork.
What actually ties these together
When you automate a check, you swap one question for another and don't notice.
Before automation the question is did someone look at this? You know the answer, because you can see the person and ask them.
After automation the question you think you are asking is still did the check pass? But the question you are now depending on is is the check alive?
Almost nobody instruments the second one.
This is well understood in operations. You don't only alert on errors, you alert on the absence of a heartbeat, because a monitoring system that dies looks exactly like a system with no problems. Dead man's switches exist for precisely this reason.
We have somehow not carried it across to the checks that gate our code. A CI workflow, a lint rule, an eval suite, an agent policy. These are all monitoring systems for correctness, and we run them with no heartbeat at all.
What a guardrail needs before you trust it
None of this is exotic.
A known-bad input it must reject. Every check needs a case it is supposed to fail on, running alongside the real ones. If your lint rule can't catch its own canary, the lint rule isn't running. If your eval's negative control scores as a pass, the grader is broken and every other number in that run is noise. Same idea as a negative control in a lab. Nobody trusts an assay that only ever comes back clean.
A recorded date of last rejection. Not when it last ran. When it last said no. A guardrail that hasn't rejected anything in four months is either protecting an unusually disciplined team or it broke in May, and those look identical on a dashboard. Put it in a column somewhere. If nobody can answer "when did this last catch something", it is decoration.
A run count somebody occasionally looks at. The zero-cases failure is the sneakiest one, because a suite that loads an empty dataset passes fast with a perfect score. Assert on the count. If it expects 240 cases and got 0, that is a hard failure, not a 100%.
Provenance on the result. Harness commit, dataset hash, prompt revision, model version, timestamp. Attached to the score, not sitting in a CI log with thirty-day retention. The test is simple. Six months from now, can you reproduce this exact number? If not, you can't use it to defend a decision, which means it was never really the reason for the decision.
Why this gets worse with agents, not better
The reason I keep coming back to this is that AI shifts the ratio.
The argument for agents in engineering, the one I actually believe, is that you get something close to an army of near-zero-mistake juniors. The constraint stops being how fast people can write code and becomes how fast people can review it. So you compensate by pushing more review into automation. More lint rules, more tests, more evals, more policy checks.
That is the right move. I would make it again.
But it means the fraction of your correctness resting on unattended machinery goes up sharply. When a human reviewed everything, a broken lint rule was a small hole in a large net. When automation reviews everything, the broken lint rule is the net.
The guardrails become load-bearing at exactly the moment nobody is watching them closely enough to notice they stopped.
Go and check one
Pick the guardrail you would be most upset to lose. The eval suite that gates prompt changes, the rule that stops an agent touching the payment path, whatever yours is.
Then answer one question about it. When did it last say no?
If you can find that out in under a minute, good. If you can't find it at all, you don't have a guardrail. You have a green light with nothing behind it, and you have been treating it as evidence.
Credit where it's due. The "never failed and stopped running print the same green" framing is Heinrich Neb's, and the permanently-gated pipeline is Vicente G. Reyes'. I just noticed they were the same bug.
I'm Arun, CTO and co-founder at Atoa. We build open banking payments for the UK. I write about AI, payments, and the messy parts of running engineering systems. @mickyarun.
Top comments (1)
The negative control canary is the only pattern that reliably catches parser drift in policy filters. When an agent runner updates its tool call schema or changes how whitespace is stripped from shell arguments, argument-matching regexes often stop matching the payload structure entirely. Because the parser sees no blacklist hits, every command passes through.
I ran into this after a runtime update changed JSON serialization on bash arguments. The filter silently evaluated every destructive command as benign because the regex was looking for a string pattern that no longer appeared in the raw input. The CI suite stayed green because nothing failed explicitly.
The fix that held up was bundling a synthetic poison payload into every filter test run. If the gate fails to reject the known bad command, the test harness hard-errors immediately. Treating a guardrail that never rejects anything as a test failure stops parser rot before code reaches production.