A deliberately strict process for preventing false completion in autonomous software development
In an earlier article, I argued that the model ...
For further actions, you may consider blocking this person and/or reporting abuse
Rule 2 has a gap with the same shape as the failure the whole process exists to catch, one level up. A locked test the implementation cannot satisfy and a locked test that is wrong produce the identical signal: the test fails and the run stops. The escalation carries nothing that separates them, so the "separate decision" gate is exactly as strong as whoever reads it, and an agent that cannot reach the bar has the same incentive to claim a bad oracle that it had to claim completion. Routing failures back to planning, as the thread suggests, does not close that either, because the router is the one who cannot tell.
The fix looks like it already lives in rule 1 rather than needing an eighth rule. If the product book has to name, per acceptance condition, one construction that must pass it, then "the test is wrong" becomes checkable: run the named construction against the test. It failing is evidence about the oracle. It passing means the implementation is the thing that missed, and the run does not get to restart from the specification boundary.
Good catch, Vinh. I agree that Rule 2 alone cannot distinguish a failing implementation from a broken oracle. Each critical test should first be checked against something we already know should pass. That lets us verify the test itself before development begins. I would not require a complete working example for every end-to-end requirement, since that could mean building the feature twice. But the basic principle belongs in Rule 1: important tests must be validated independently.
A known-pass construction only rules out one of the two ways an oracle can be wrong. A test that is too strict fails the construction and gets caught. A test that is too loose passes it, because a test that passes everything passes that too, and too-loose is the false completion case itself: the locked criteria agree with the agent that the work is done. So the thing worth attaching per critical test is a pair, one construction that must pass and one that must fail, and the must-fail half is usually the cheaper of the two to write.
On the cost, the must-pass half is not the feature. An acceptance test asserts over an artifact - a response body, a file, a resulting state - so a hand-written artifact validates the oracle with no implementation behind it. It only approaches building the thing twice where the condition is over a process rather than an artifact, and that set is small enough to name in the product book rather than exempt by default.
You’re right, Vinh. A test that's too loose is exactly what enables false completion. Pairing a 'positive' case with a 'negative' case gives the oracle both a positive and a negative control.
And your distinction between checking an artifact and checking a process also resolves most of my cost concern. For artifact-level checks, a hand written response, file, or state is enough; it does not require implementing the feature twice.
The refined rule becomes: "each critical acceptance condition in the product book should include both a must-pass and a must-fail case" or so.
Ran into this twice on data pipeline projects - the agent said everything was done because the files were there, and we only found the content was wrong on the third review. Your point about separating who defines done from who does the work is what actually mattered to us. We tried keeping the same session going across all the steps to save time but it got unreliable around step four or five. Starting fresh each pass was slower but we could actually trust what came out.
Yes Tae, that example is false completion in its cleanest form. Your observation on unreliable behavior around steps 4-5 also matches what I’ve seen: accumulated context initially saves time, but then carries stale assumptions forward and causes rapid deterioration. Fresh instances cost more, but a compact and explicit handoff doc makes their output way more trustworthy.
Hi Tae, your example of output files being present while their contents were wrong is exactly the distinction I’m trying to understand.
What check finally exposed the problem on the third review, and is that check now automated? I’m also curious whether starting fresh sessions resolved the issue or mainly made the remaining problems easier to catch.
I’m exploring developer tooling for checking agent completion. A short exchange here would be valuable; there’s no need to share private code or data.
The “done” state being outside the agent is probably the most important architectural point here. I’d frame the acceptance layer as an independent control plane: the builder can produce code and evidence, but it should never control the definition of success. One nuance worth adding is that hidden tests alone don’t solve false completion the oracle itself can be wrong or incomplete. The strongest setup is locked acceptance criteria + independent evaluation + deterministic checks, with failures routing back to planning rather than letting the coding loop redefine the target.
Indeed, Mateo. 'Independent control plane' is a good name for what I was trying to describe here. Hidden tests can also be wrong. If that happens, someone outside the coding loop i.e. human, should review the tests; and in any case the builder should not be allowed to change the definition of success.
The scary one isn't a red test. It's the agent saying done while the thing a person would actually use is still missing.
I lock what done looks like before it starts writing, the click or flow a person hits. Then a separate go to build. The session that wrote it doesn't get to say it's finished.
If that path is still wrong, it isn't done.
Spot on, Ezra. That is the core of the 'False Completion' trap. Locking the definition of 'done' before any code is written and ensuring the builder never grades its own work are exactly how we stop an agent from moving the finish line. And as you said, if that pre-locked path is still wrong, it simply ain't done yet.
Hi Gilad,
Your article on false completion addresses a distinction I’m investigating: an agent can produce plausible work without satisfying the original requirements.
I’m exploring developer tooling in this area and would value your experience with the acceptance process you described. Which parts are enforced by software today, and which still depend on someone manually checking that the rules were followed?
Would you be open to a 20-minute conversation? I’m especially interested in where additional checks reduce work - and where they just create more process.