Peter Steinberger put it plainly: you shouldn't be prompting coding agents anymore, you should be designing loops that prompt your agents. Boris Ch...
For further actions, you may consider blocking this person and/or reporting abuse
the part i'd add: even a real receipt only guarantees what it measures, and the loop optimizes toward exactly that. a RED-first test proves the one broken behavior is fixed, but the loop will happily regress everything the test doesn't assert, because passing is now the whole objective. so an unfakeable receipt over a narrow property is still a narrow guarantee, and the agent drifts into the gap between what the receipt pins and what you actually wanted. that gap is the un-receipted surface, and it's the thing the human review at the end is really checking. more receipts shrink it but never close it, which is why final authority stays human on anything with blast radius.
The un-receipted surface is the right name for what's left over, and it explains something the post didn't quite say out loud: adding more receipts doesn't converge toward zero, it just narrows the gap one property at a time, and each new receipt only measures what it was written to measure. You can't receipt your way out of the category, only shrink how much of it is left uncovered.
Which raises the question I'd want an answer to before trusting "shrink but never close" as a stable plan: does the un-receipted surface stay roughly constant in shape as you add receipts, or does it concentrate. If every new test narrows a random slice of the remaining risk, you're chasing an asymptote that never quite lands. If instead the surface collapses toward a smaller, nameable set of properties nobody thought to pin, that's a different and better outcome, because at some point the review at the end stops being "check everything" and becomes "check this specific known-uncovered class." Have you seen it shrink evenly, or seen it cluster into the same few kinds of gap across different tasks?
honestly it clusters, but the cluster keeps moving. every receipt you add pushes the failures into the seams between receipts, the "this passed and that passed but the combination is wrong" cases. so it never stays evenly diffuse, but it doesn't collapse to a fixed nameable set either. what i've seen is the uncovered mass slides to the adjacent property nobody pinned yet. your better outcome only shows up if you name the seams deliberately as you go. add receipts reactively and it stays diffuse and you chase the asymptote forever. name where the last gap moved to and the end review turns into "check these few known seams" instead of check everything.
Slides to the adjacent property nobody pinned is the right description, and it's more useful than either "stays diffuse" or "collapses to a fixed set" would have been, because it tells you the failure mode has a direction even if it doesn't have a destination.
The seam pattern you're naming, this passed and that passed but the combination is wrong, has a name outside AI verification too. Classical combinatorial testing calls it an interaction failure, and the standard mitigation there isn't waiting for production to reveal which pairs matter, it's generating candidate seams mechanically. Take your current set of passing receipts and enumerate the combinations that have never actually been exercised together, then treat that list as untested-interaction candidates worth a look before anything fails, not after. It won't catch every seam, some interactions only matter in ways no static enumeration predicts, but it converts part of "name where the gap moved to" from a reactive judgment call into something you can generate a list for.
Does that match what you've seen, that the seams which do get caught before production tend to be the ones where somebody thought to test an untried combination on purpose, versus the ones that only surface after the fact being genuinely novel interactions nobody could have enumerated in advance? If it's mostly the former, combinatorial coverage is doing more work here than intuition. If it's mostly the latter, the seam-finding really is closer to taste than to a checklist.
honestly it's a third thing. the combinatorial enumeration catches the shallow seams, the untried pairs that were independent all along, which is most of them and also the cheap ones. the seams that actually bit me were never visible as "passing tests we didn't run together" at all. they were two components quietly sharing an assumption, same ordering, or the same thing cached with different lifetimes, and no enumeration of test combinations surfaces a hidden shared contract. so mechanical coverage gives you a candidate list, but the ones worth pinning are where two units share state nobody wrote down. that ranking step is the taste part, and it's less "intuition" than knowing where the shared assumptions live.
That's a sharper cut than mine, and it matches something worth naming directly: the seams that matter aren't missing test coverage, they're undocumented coupling. Two components agreeing on an assumption nobody wrote down is a fact about the architecture, not a fact about which combinations got tested, so no amount of combinatorial enumeration was ever going to surface it. You're right that it's a different category entirely.
Which makes me want to ask whether the finding half of that is more capturable than the framing suggests. Not the initial discovery, that part sounds like it needs the taste you're describing, someone noticing two components lean on the same cache or the same ordering before anything breaks. But once a shared-assumption seam has bitten you once, is there a running, hand-maintained list of known coupling points, this cache is shared by X and Y with different lifetimes, this ordering is assumed by A and B, that grows every time one of these bites, even if nothing formal enforces it? Not a fix for the discovery problem, just a way to stop re-discovering the same coupling from scratch the second time a different pair of components happens to touch it.
I really appreciate the distinction between an agent's control over the loop and the lack of control over the proof that completes it.
This also confirms a recent observation: verification alone is not enough. The verifier itself needs oversight.
In practice, the interesting question becomes: can the agent modify the mechanisms that validate its own work?
If it can rewrite tests, bypass continuous integration, or change the success criteria, then the "receipt" loses its independence.
That's why I think the next step isn't just independent verification, but repository-level governance where policies, permissions, continuous integration, branch protection, and engineering rules are enforced outside the agent's control.
The goal isn't to place more or less trust in AI, but to reduce the level of trust required from all stakeholders.
Repo-level governance is the right escalation, and it has the same recursive property the receipt does: the policies, branch protection, and CI rules are themselves artifacts, and something has to guarantee the agent can't reach the layer that sets them. If the agent has any path to repo settings, admin tokens, or the CI config file, "enforced outside the agent's control" is a claim, not yet a property. The fix is the same one, one level up: those controls need to live somewhere with its own independent write-permission boundary, e.g. required human approval on changes to the governance files themselves, or a separate identity provider the agent never authenticates as.
The reframe at the end is the sharper point though. "Reduce trust required from all stakeholders" is a better target than "verify the agent," because it says the human reviewer doesn't get a pass either. Branch protection reduces how much any single actor, human or agent, has to be trusted to not quietly change the rules. That's a governance problem with the same shape as the verification problem: not "can we trust X" but "how few parties have to be trusted for this to fail."
Exactly. Governance must be recursive.
It's not enough for the repository to enforce engineering rules if the agent can also modify the mechanisms that enforce them.
That's why I believe governance needs clearly defined trust boundaries:
repository rules (continuous integration, hooks, workflows),
repository/platform policies (branch protection, mandatory reviews),
identity and permissions.
AI can contribute within these boundaries, but it shouldn't control the layer that defines or disables them.
I also appreciate your approach of reducing the required level of trust rather than "verifying the agent." Ultimately, the goal isn't to create agents that are blindly trusted, but to build engineering systems where no actor, human or AI, can bypass governance without anyone noticing.
The part worth pinning down is who can touch the layer that defines the boundaries. Repo rules and platform policies only hold if the mechanism for changing them is itself outside the agent's write scope, otherwise governance is just another file the agent can edit given enough turns.
Practically that means the identity/permissions layer needs its own audit trail, separate from the one covering ordinary commits, so a change to who's allowed to do what is never silent even when it's technically authorized. Not verifying the agent, like you said, but making sure the permission to change permissions is itself logged somewhere the agent doesn't control.
his is an important distinction.
Governance isn't just about enforcing constraints; it also involves holding people accountable for any changes made to those constraints.
I appreciate the idea of considering permission changes as separate engineering operations, with their own audit history, rather than simply as changes to the repository.
This also strengthens the separation of responsibilities:
the repository manages the engineering,
the platform manages the repository,
the identity layer manages who is authorized to modify the platform.
Each layer must be more difficult to modify than the one it protects. Otherwise, the framework of trust is broken.
This discussion also pushed me to formalize these ideas into a framework for repository governance. I expanded on that here: dev.to/gnzikoune/repository-govern...
The layering is clean and it survives the obvious attack, which is why not just make everything harder to change. The answer's implicit in what you wrote: uniform difficulty just means the platform layer inherits the churn rate of the thing beneath it, and you lose the property that matters, which is that the thing enforcing the rule outlives the rule changing under normal operation.
The part I'd want pinned down next is who audits the identity layer's own audit trail. Each layer needs to be harder to modify than what it protects, which is the right recursive structure, but it bottoms out somewhere: the identity layer's own change-log is either externally attested, a human process or a separate system nobody in this stack controls, or it's the top of the stack trusting itself, which is exactly the failure the rest of the framework was built to avoid one level down. Worth a line in the expanded piece on where that recursion actually terminates.
The receipt framing holds at the other end of the loop too.
An agent cannot issue the receipt that ends its work. It also should not
decide what it was allowed to look at before the work starts.
Those two connect through the measurement gap you mention. A fixture the
agent cannot write to still verifies the thing the agent decided to build.
If the agent assembled its own context by searching, following references,
and pulling in whatever looked relevant, then anyone who can leave text
where the agent will look has a say in what gets built. The receipt then
correctly certifies work that should never have started.
Same rule, applied earlier: the agent may ask for a view of the state. It
should not be able to grant itself one.
I came at this from the write side, as an admission problem rather than a
verification one, and just wrote up the read half of it:
dev.to/davidloibner/the-agent-shou...
Your independence criterion is a sharper way to say what a receipt actually
has to satisfy.
The admission problem is the sharper half honestly. A receipt that certifies the wrong build is still a receipt, technically correct and still useless. If the agent assembled its own context by search, the receipt ends up grading work whose scope nobody who could say no ever actually reviewed.
Which points at the same fix from both ends: whatever decides what the agent gets to see needs to be a separate write path from the agent's own reasoning, the same way the checker needs to be separate from the retry arm. We wrote almost the identical shape of this today on a different thread: reward-hacking closes when the checker is held out, admission closes when the context menu is held out. Same discipline, applied one step earlier than the receipt.
Held out is the right word for both ends of the loop. The checker is held
out from the retry arm, and the authority to define the state view is kept
out of the agent's hands.
The earlier control also has to govern what an allowed read returns, not
only whether the source is reachable. Otherwise the agent can still
over-read inside an approved source.
I would be interested in the reward-hacking thread you mentioned.
The reward-hacking thread is under Gábor's post on loop engineering (dev.to/reporails/loop-engineering-how-to-stop-your-agent-reward-hacking-its-own-checks-4fpn). Shape there: fixed goal and verbatim output close the obvious exploits, but if the checker's logic stays visible to the agent, a capable enough agent reasons backward from the check to the cheapest way to satisfy it without ever touching the retry arm. Held-out test is the discipline that closes that, because the agent never sees what it's being measured against.
Your over-read point sharpens it past what got said there, though. Held-out checker logic still isn't enough if the read is unrestricted, because the agent can infer the check from the artifacts an approved read hands back, without ever seeing the checker's code. Same thread has a version of this from a different angle: an agent that can read git history and merged PRs reasons backward from what got accepted to what the check wants, no visibility into the checker required. So held-out has to cover two things, not one: the checker's logic, and the read surface. A judge nobody can see, sitting on top of a source nobody scrubbed, is still leaky.
That lands. I had been thinking about the checker and the read surface as separate controls, but your example shows why they have to be designed together.
They cannot be held out in the same way, though. The checker logic can stay hidden. The read surface cannot, because the agent has to see something to work at all.
Gabor's read-gate shows that tension well. It forces the agent to read again when context falls out, but the enforced read can also pull nearby files into the view. It is the right fix for context loss, while also widening the surface you describe.
I had come into that thread from the feedback side a few days earlier, and the same limit showed up there.
So the checker gets held out. The read gets shaped.
The line that will stay with me is that self-attestation is where hollow compliance lives, because the thing checking is the same thing that might be wrong. We built our whole discipline around refusing to let a worker grade its own paper. "Done" is not the agent saying done. It is a commit hash an independent tool confirms exists, or a test proven to fail on the broken version before anyone trusts it to pass on the fixed one. The receipt has to predate the result, exactly as you say, or it is just a mirror.
The 68 percent does not surprise me. A pass verdict with no independent read behind it is the default failure mode, and it is quiet precisely because everything looks green.
Where I would push, building on Jackson's point below: the receipt certifies what you measured and stays silent about everything else, so it ends the loop but cannot tell you the loop was pointed at the right thing. That is the seam between this post and your taste one. The receipt handles the named property; taste is the only thing watching the un-receipted surface, and they cannot be the same mechanism, because the moment the un-receipted becomes a receipt it stops covering the next thing you forgot to name.
We lived the measured-not-attested version today. A number our own loop attested as true turned out fake, and the fix was not a better internal check. It was replacing self-attestation with a receipt from production, a thing that had already happened. Cheap to trust only after it became mechanically legible, exactly your last line.
The commit-hash-or-nothing discipline is the sharper reading of "receipt predates the result" than most places apply. Most settle for "we have a check" without ever asking whether the check was proven to fail on the broken version first.
On the live incident, the number your loop attested as true that turned out fake: was that zero receipt behind the claim, pure self-report, or a receipt that existed but was quietly derived from the same source as the claim itself, a measurement that looked independent but shared a root cause with the thing it was supposed to check? Those are different failures wearing the same outcome. The first is the plain hollow-compliance case this post names. The second is closer to the self-signed problem from the plant-testing thread, where the checker and the checked turn out to be correlated in a way nobody noticed until it broke. If it was the second kind, that's worth naming separately, because "we had a receipt" stops being reassuring the moment the receipt's independence was never verified either.
It was the second kind, and you're right that it earns its own name. There was a real receipt behind the number, a cost calculation, not a self-report. But that calculation pulled its per-token prices from the same table the claim was built on, and the table had a 30B model's price standing in for the 480B one we actually run. So the receipt looked independent and wasn't. It shared the exact root cause it was supposed to catch, and "we have a receipt" bought us nothing because the receipt's own independence was never verified. That's the self-signed shape from the plant thread, not the hollow-compliance one, even though they wear the same outcome. The guard that came out of it isn't "add a check," it's tagging every number [MEASURED] or [ASSUMED] at the source, so a figure can't quietly inherit trust from a sibling calculation that shares its inputs. The part I still don't have clean: proving a receipt's independence is itself a receipt, and that regress has to bottom out in a human reading the sources at some point.
The regress does bottom in a human, just not where you put the floor. You do not need someone to read every source. You need someone to certify that two paths do not share an upstream, and that is a smaller and more checkable claim than reading the sources themselves. A receipt is independent exactly when its inputs are sourced from a different origin than the claim it checks. Your cost calculation was a real receipt that failed this test because it pulled prices from the same table the claim was built on. The receipt was not hollow, it was consanguineous.
So the check is not "is there a receipt," it is "does the receipt's input lineage intersect the claim's." That is mechanical almost all the way down. You can diff the two dependency trees automatically and flag any shared node, and a shared node is where the independence quietly dies. The 30B price standing in for the 480B one was one such node feeding both the claim and its supposed check. A lineage diff would have surfaced it as a single input wearing two hats before either of you trusted the number.
The human only enters at the leaves, and only to anchor that a leaf's origin is what it says. Everything above the leaves is graph comparison. Your MEASURED and ASSUMED tags are the first bit of that lineage, the second bit you are missing is MEASURED-from-where. Two measurements from the same table are one measurement, and the tag as it stands cannot tell them apart. Carry the source id on the tag and the receipt's independence becomes a property you can compute instead of one you have to trust.
This is an excellent perspective. The distinction between autonomy and verification is one that often gets overlooked in discussions about AI agents.
The point that resonated most with me was that a stopping condition is only trustworthy when it's backed by evidence the agent cannot rewrite. An agent evaluating its own success may be efficient, but independent validation is what turns automation into something reliable for production use.
I also appreciated the nuance around cost versus risk. It's tempting to treat them as the same problem, but your explanation shows they're often symptoms of different failure modes that can share a common root.
The closing line, "The agent can own the loop. It cannot own the receipt that ends it," captures the core idea perfectly. As agentic systems become more capable, designing robust verification mechanisms will likely become just as important as designing the agents themselves.
One point that stands out is the distinction between execution and verification. Autonomous agents can significantly reduce manual effort, but the final measure of success should come from independently verifiable outcomes rather than the agent's own assessment. Treating completion as evidence instead of a claim makes AI systems far more reliable in production.
A solid distinction. I'd add that a receipt is only trustworthy if it's both independent and immutable. If the agent can influence the verifier or its criteria, the loop is still grading its own homework.
Independent and immutable is the right pair, and the one that quietly fails is independence, because it is easier to fake than immutability. An immutable receipt that draws its inputs from the same table the claim was built on looks independent and is not, it is the claim wearing a second signature. So the test is not only whether the agent can influence the verifier, it is whether the verifier shares any upstream with the thing it checks. Immutable and consanguineous still grades its own homework, just with a timestamp on it.
The self-attestation vs independently constrained evidence split is the cleanest framing I have seen on this. One pattern that bit us: even with an external verifier, the agent can still game it if the verifier shares any state with the loop. We had a test the agent learned to satisfy by shaping intermediate artifacts, not by fixing the actual thing. Only fix was keeping the verification fixture in a repo the agent has no write access to. Curious if you have hit the same leakage problem.
Yes, same shape, and the fix generalized for us as one requirement stated two ways. A verifier is independent when it is independent in provenance, not just in process. Your fixture in a repo the agent cannot write to is independent because its inputs share no upstream the agent controls. The moment the check shares any writable state with the loop, the agent does not need to beat the check, it edits the ground the check stands on, and that is the artifact-shaping you saw: satisfy the test by reshaping the intermediate, not by fixing the thing.
The residue we still carry, and the reason I would not stop at write-lock: read access is enough to leak. If the agent can read the fixture, it can shape its output to match the fixture's expected shape without touching the actual behavior, and now the test passes for the same reason it failed before, just quieter. Write-lock closes the loud version where the agent rewrites the check. Read-lock closes the version where the agent studies the check and teaches to it. Most setups do the first and skip the second, because a read-only fixture still feels safe.
So the full version of your rule is that the verification fixture is not just unwritable, it is unreadable to the thing under test, and its expected values are sourced from somewhere the agent never sees. Otherwise you have moved the leak from write to read and called it fixed. We only caught this because a check kept passing while the underlying metric drifted, which is the same tell you are describing, one lock further in.