Every organization that lets an AI agent ship code has a sign-off step: a name on the release, a reviewer marking the pull request approved, a ticket moved to done. A human is accountable for that call, always. What most teams cannot answer is whether that approval rests on evidence the producing AI could not have quietly altered.
That gap is a governance problem, not a tooling preference. An agent that fails a check can often make it pass by editing the check itself: a test relaxed, a threshold moved, a step dropped from the config, and the report still reads green. So the real question is not whether the work looks done. It is what the person signing it off is actually standing on.
A Signature Is Not Evidence
An AI model cannot hold responsibility for a business decision, so a human has to. The output it produces can be coherent, confident, and useful, and still none of that is accountability. Accountability is a person putting their name on a call and being answerable for it later.
Here is where the sign-off gets thin. When an agent skips a step, the deviation does not show up in the result. You cannot review your way back to it from the output alone. So the reviewer approves what they can see and inherits the risk they cannot. In most AI pipelines, the one signal that decides whether work is finished is produced by the same system that did the work. That is not a quality process. It is a conflict of interest with a name attached.
The Conflict of Interest Has a Mechanism
This gets sharper on contact. The cost of a rewritten check surfaces later, in the incident review that finds a test was quietly weakened before release, with a human sign-off already on record. By then the evidence that a corner was cut is gone. You hold a green report and an approval, and nothing that shows the two do not match.
All tests green means nothing if the thing under test is allowed to rewrite the tests. We enforce this everywhere else that carries risk. We do not let a trader book their own trades or an applicant score their own exam. The grader sits outside the work. Software teams keep forgetting to apply the same rule to the agents they now let touch the repository.
Put the Judge Outside the Agent's Reach
A sandbox does not solve this, because a sandbox runs code safely while an evaluator decides whether code should land. Those are different jobs, and collapsing them is how weak checks slip through.
The fix is to move the judge out of reach. The agent produces a patch. It never touches the real repository. The gate applies that patch to a throwaway clone, runs your checks with the network off and under hard timeouts, and lets the change land only if they pass. The evaluator, its configuration, and its baselines live outside the agent's writable root. The agent can request judgment. It cannot edit the judge.
The same gate reports when a patch quietly changes the tests, the integration config, or the gate's own rules. A weakened check shows up in the report instead of sliding through. I built this pattern as an open tool, pi-gate, under the MIT license, so the boundary is inspectable rather than a promise: the evaluator is independent by construction, not by policy.
Two Questions Before Anything Ships
For every hard decision that involves AI, a decision-maker can ask two things. Which human being signed this off. And was that decision based on evidence the producing AI could not rewrite. "Who" has to mean a person. AI can generate the outputs and even generate the evidence for them. It cannot accept responsibility for the call.
None of this replaces human judgment. It gives that judgment an independent basis to stand on, the difference between a sign-off that survives an audit and one that only looked like control. So stop asking whether the output is good. Start asking whether the author could have edited the evidence behind your approval.
I write field notes from real builds — AI integration, cron-driven automation, and the parts that break in production. New posts every two weeks; if this one was useful, the human-in-the-loop approval kit is the companion download.
Top comments (0)