Verify, Don't Review: The Operating Discipline for Coding Agents
Simon Willison published a note today that distills the entire coding-agent skill into one sentence: "The key skill required to make productive use of coding agents is being able to confidently instruct them on how to make changes and then confidently verify that those changes have been applied in the correct way." Not read. Not review. Verify. That word choice is the whole argument.
📖 Read the full version with charts and embedded sources on AgentConn →
View original post on simonwillison.net →
The distinction matters because the old workflow is collapsing. SonarSource's State of Code 2026 report found that 96% of developers do not fully trust the functional accuracy of AI-generated code, yet only 48% actually verify it before committing. The other half are either reading diffs they cannot meaningfully process at agent speed, or merging on faith. Neither scales. Neither is a discipline.
Meanwhile, the opposite trend is also real. As Business Insider reported, developers are increasingly deploying AI-generated code without human review at all — a sign that growing confidence is outpacing growing discipline.
This article lays out the operating framework that bridges both sides: what confident instruction looks like, why verification beats review, the emerging cognitive phenomenon that makes review physically harder, and the concrete infrastructure teams need to make verification the default.
The Review Bottleneck Is Structural, Not Cultural
The numbers have been clear since early 2026. Faros AI analyzed production workflows and found that AI-assisted teams generate roughly 4x more code output but deliver only about 12% additional value. Review duration increased 441.5%. Code churn rose 861%. Defect rates climbed from 9% to 54% per developer.
As Addy Osmani documented in his Agentic Code Review analysis (also published on O'Reilly Radar), one developer's comment captured the situation precisely: reviewing an agent's PR made them "the first human being to ever lay eyes on this code." That is not code review. That is archaeology.
The bottleneck is not lazy developers. It is a structural mismatch: agents produce PRs at 10x the rate human review capacity can absorb. The queue grows faster than it drains. Adding more reviewers does not solve the problem because the bottleneck is cognitive bandwidth, not headcount. You cannot read code faster than an agent writes it. Trying to is the wrong response. The right response is to stop reading and start proving.
💡 Willison's framework in 5 words: At the Pragmatic Summit, Willison demonstrated that "use red-green TDD" is only five tokens, and it transforms the agent session from "generate code I have to read" to "generate code that proves itself." Every agent session should start with how to run tests.
Confident Instruction: The Input Discipline
Verification is only half the framework. The other half is confident instruction — knowing what to ask for with enough precision that the output is verifiable.
Willison's Agentic Engineering Patterns guide demonstrates this with annotated real-world prompts. A good instruction names the exact input file, describes the precise technical scope, references the testing tool, and specifies the expected behavior. A bad instruction says "make this better" and hopes the diff looks reasonable.
The distinction is not about prompt engineering tricks. It is about domain expertise. You need to know what you want built well enough to describe it precisely, and you need to know what "correct" looks like well enough to verify it without reading every line.
As Crickett put it: "Give the agent a way to verify its work before it hands the result back to you." This is where the "manage agents like junior developers" framing breaks down. Junior developers need guidance because they lack domain knowledge. Coding agents need constraint because they lack judgment. The operator's job is not mentoring — it is specification. The tighter the spec, the smaller the verification surface.
Why Your Brain Is Rejecting AI Output
Here is the twist that makes the verify-over-review discipline not just efficient but necessary: your brain is already refusing to do the review work.
Rafal Cymerys coined the term in a post that hit 478 points on Hacker News: "I'm becoming AI-blind." His thesis is that overexposure to low-effort AI-generated content has trained his brain to automatically dismiss it — a cognitive filter analogous to banner blindness. As he wrote: "I feel like I've been 'pre-trained' on all the AI-generated LinkedIn posts, emails and websites that are full of text but empty on meaning."
View discussion on Hacker News →
The Hacker News discussion validated the phenomenon at scale. The top comment, from user causal, described "some psychological mechanism by which my brain immediately recognizes AI generated text and just short-circuits to 'there is no information here.'" Another commenter, bluetomcat, identified the root cause: "The sentences and the paragraphs are a statistically viable derivation... but without an interesting semantic core."
This is not a preference. It is an involuntary cognitive response. And it has direct implications for code review. When AI-generated code shares the same statistical texture as AI-generated prose — syntactically valid, structurally reasonable, semantically hollow — the same short-circuit fires. Reviewers' eyes glaze. They scroll past. They approve.
The research confirms this. O'Reilly's Comprehension Debt analysis found that AI coding agents generate code 5-7x faster than humans can comprehend it. Developers who used AI for code generation delegation scored below 40% on comprehension tests, versus above 65% for those who used AI for conceptual inquiry. The act of delegating generation literally degrades your ability to understand the output.
⚠️ The comprehension debt trap: Margaret-Anne Storey's Triple Debt Model identifies three simultaneous debts from unchecked AI generation: technical debt (in the code), cognitive debt (in the people), and intent debt (the absence of documented rationale). When nobody on the team understands how the system works, even correct code becomes a liability. Verification proves the code works. It does not prove the team understands it. Both matter.
The Verification Stack: What Actually Works
If review is failing and your brain is fighting you, what does verification infrastructure actually look like? Willison has been the clearest voice here, both at the Pragmatic Summit and in his Agentic Engineering Patterns.
Layer 1: Red-green TDD. Tell the agent to use TDD. Write failing tests first, then make them pass. Five tokens. The agent now generates self-proving code. As the brgr.one analysis noted, tests written by agents are "effectively free" since generation costs approach zero — collapsing the old cost-benefit argument against comprehensive testing.
Layer 2: Smoke tests. Beyond unit tests, demand runtime proof. Does the server start? Does the curl command return 200? Does the CLI produce the expected output file? These are not sophisticated — they are binary. Works or does not work. An agent can run them and report results before you ever look at the code.
Layer 3: Conformance suites. Willison's most creative contribution. He had Claude build parallel implementations across six web frameworks, then used consistency across those implementations to reverse-engineer a reliable conformance standard. If the same behavior works identically across Flask, Django, Express, FastAPI, Rails, and Sinatra, the spec is probably correct. Cross-implementation verification as a substitute for reading any single implementation.
Layer 4: Sandboxing. Willison's "lethal trifecta" framework: an attack requires (1) model access to private data, (2) exposure to malicious instructions, and (3) exfiltration capability. Remove any one and the attack surface collapses. Containers, VMs, and permission-constrained execution environments limit blast radius so that even wrong code cannot cause catastrophic harm.
This is the same guardrail stack we have documented as "the harness" — the infrastructure layer that makes agent output trustworthy regardless of which model generated it. Verification is not a human activity bolted onto the end of the pipeline. It is infrastructure built into the pipeline itself.
The Watermarking Signal: Detection Is Coming
The verification discipline gains another dimension as detection infrastructure matures. Sebastian Raschka published a detailed technical walkthrough today on how Claude's watermarking works — tournament sampling that embeds statistically detectable but human-imperceptible signals during token selection.
As Raschka explained (1,927 likes, 222K views): the watermark is applied at the sampling stage, using a secret key derived from the API key and previous token context. Detection is computationally cheap — no LLM rerun needed. "The watermarking is nothing else for the end user than fixing a random seed and making this sampling kind of deterministic."
View original post on Substack →
This matters for the verification discipline because it introduces a new layer: provenance verification. Not just "does this code work?" but "where did this code come from?" The EU AI Act (Article 50, enforceable August 2026) requires AI outputs to be detectable as artificially generated. Watermarking is the mechanism.
For teams building verification infrastructure, provenance is not the priority — correctness is. But the fact that detection is becoming technically feasible and legally required reinforces the core argument: the industry is moving from "trust the output" to "prove the output," at every level of the stack.
The Comprehension Budget: What Verification Misses
Here is the contrarian take that the pure-verification camp underplays: verification proves correctness, but it does not build understanding.
If your entire workflow is "instruct agent, run tests, merge" — and the tests pass — you have a working system that nobody understands. Six months later, when a requirement changes and the tests need updating, nobody on the team can confidently modify the code because nobody built a mental model of it. This is comprehension debt, and it compounds silently.
The answer is not "go back to reading every line." It is budgeting comprehension time separately from verification time. Willison acknowledges this tension: "Speed and understanding are always in tension. Agents give you speed. Walkthroughs and interactive explanations are how you buy back understanding."
The practical discipline looks like this:
- Verify first — TDD, smoke tests, conformance suites prove the code works. This is the gate for merging.
- Comprehend second — ask the agent to explain what it built. Walk through the architecture. Read the key abstractions. This is the gate for confidence.
- Document third — the agent can generate ADRs, architecture docs, and inline comments explaining rationale. This is the gate for team knowledge.
The distinction between verification and comprehension is the most important nuance in the coding-agent discipline. Teams that treat them as the same activity — review — do neither well. Teams that separate them can be fast (verification is automated) and informed (comprehension is deliberate).
💡 The verification checklist for teams:
- Every agent session starts with "use red-green TDD" and how to run tests
- Instructions name exact files, scope, and expected behavior
- Smoke tests run before any human looks at the diff
- Sandbox or container limits blast radius for untrusted operations
- Post-merge: agent explains what it built (comprehension budget)
- Architecture decisions documented as ADRs, not left implicit
- Anti-slop gates (quality linters) catch low-effort patterns before review
- Judge layers (runtime validation) validate output at the infrastructure level
What This Means for You
The operating discipline for coding agents is not "be careful with AI." It is a specific, trainable skill with two components:
Confident instruction — you know your domain well enough to write precise prompts that constrain the agent's scope to something verifiable. This is domain expertise, not prompt engineering.
Confident verification — you have infrastructure (tests, smoke checks, conformance suites, sandboxes, quality gates) that proves correctness without requiring you to read every line. This is engineering discipline, not trust.
The AI-blind phenomenon is not a bug — it is your brain correctly identifying that line-by-line reading of AI-generated output is a poor use of cognitive bandwidth. The discipline is to stop fighting that instinct and instead build the infrastructure that makes reading unnecessary for the correctness question, while deliberately budgeting time for the comprehension question.
Willison's framework is not just practical advice. It is the operating discipline for a new mode of engineering — one where the human's job is specification and verification, and the agent's job is everything in between.
Originally published at AgentConn
Top comments (0)