The bottleneck moved, and most teams haven't noticed. It isn't writing anymore. It's proving.
I keep hearing the same soft complaint from engineers: "I spend more time reviewing AI code than I ever spent writing code myself." We tend to hear that as a productivity problem, as if the tool isn't saving time. I think we're reading it wrong. AI didn't fail to save time. It saved time on one part of the pipeline and quietly loaded the cost onto another.
Generation got cheap. Verification did not.
The asymmetry nobody budgets for
For decades, writing code was the expensive step, and it was expensive in a way that kept review proportional. If a senior engineer took a day to produce a change, a reviewer spent twenty minutes on it. The ratio was roughly stable, so cost lived upstream.
Now a model produces the same change in ninety seconds. The twenty-minute review didn't get shorter. It might get longer, because the volume arriving is higher than any team's review capacity ever had to absorb. When one stage speeds up and the next stays fixed, the fixed stage becomes the constraint.
Where the constraint actually shows up
I've spent the last few months building agent systems, and the verification problem stopped being abstract fast. Four places it showed up in my own work:
1. Tool calls. An agent's delete in a sandbox is not the same as delete in production. Binary allow/deny can't tell the difference, which leaves you choosing between over-privileged agents and approval fatigue. I ended up building a four-state gate (allow, audit, escalate, deny) that runs outside the model, so no prompt can override a deny. Tested against 83 real agents across 10 frameworks.
2. Plan approval. A planner that lists the right steps but orders them wrong is more dangerous than one that's obviously broken. I ran 170 goals through deterministic gates that block unverified dependencies, unsafe sequencing, and weak rollback before a human ever sees the plan. Full field test.
3. The gate itself. This is the one that surprised me. A deterministic gate that silently stops firing makes your metrics look better: blocker counts drop, dashboards go green. It's the Kubernetes ReplicaSet that kept one pod Running while four releases never actually ran, with every health check passing. The fix was a canary that asserts each gate still fires.
4. The review population itself. When generation is cheap, everyone becomes a reviewer, and most of us were never trained for that, let alone measured on it. That's a skills gap, not a tooling gap, and it doesn't fix itself.
Why more reviewers isn't the fix
The intuitive response is to add review capacity. But review attention doesn't scale linearly with volume, and it degrades in a specific way. At high volume, people start pattern-matching on plausibility instead of reasoning about correctness. The cleaner the diff, the less likely it gets interrogated.
You don't fix a verification bottleneck by adding humans at the same step. You fix it by moving work out of human attention and into gates, so human attention gets spent on the ambiguous stuff, which is the only stuff it's actually good for.
Four gates that moved the work back into the pipeline
None of these are novel. They're the checks that stopped being optional once generation got cheap.
Gate 1: Context-aware authorization on tool calls. Not "is this tool allowed" but "is this tool allowed, here, on this data, by this agent." Four decisions instead of two, explained and audited. Fail closed on anything unknown.
Gate 2: Structural validation before human review. Every precondition maps to an earlier task. Every high-blast-radius step has a reachable rollback. This is deterministic, costs nothing, and eliminates roughly half the defects before a person looks.
Gate 3: A canary for every gate. If a safety check stops firing, something must go red. "Blocker count dropped" should never be the first sign that a guard died.
Gate 4: Escalation as a real path, not a dead end. An agent that refuses 96 of 97 risky plans is doing its job, if the escalate path reaches a human with enough context to decide. Refusal without a runway is just blockage.
That's the shape of it. Don't try to verify more. Make fewer things need human verification.
The honest limitation
Gates only cover the failure modes you thought to encode. My structural checks are strong on dependency ordering and rollback; they're weak on "this logic is subtly wrong in a way nobody anticipated." That still needs a human, ideally one who reads for reasoning rather than cosmetics.
And there's a real cost to gates: they can calcify. A gate that was right for last quarter's risk profile can become noise. The canary tells you a gate is alive. It doesn't tell you the gate is still worth having.
The question I keep landing on
If generation is now cheap and verification is now the constraint, then the highest-leverage engineering investment isn't a better model. It's the machinery that decides what gets trusted. Which means the teams that win the next few years may not be the ones writing the most AI code. They'll be the ones who got serious about proving it.
Where is your bottleneck now, writing or proving? I'm curious whether this is landing the same way for other teams or if I'm over-fitting my own experience.
- Repos: agent-tooltrust · planner-critic-engine
- Prior posts: I Stopped Trusting AI Agents With Tools · The Gate That Stayed Silent
Top comments (2)
The canary for a gate point is the one that stings. We saw dashboards stay green for weeks while a permission check had quietly stopped firing, nobody caught it till an audit forced it. Green metrics only prove the pipe is open, not that anything is still flowing through it.
Framing the shift as an unbudgeted verification asymmetry cuts right to the mechanical failure most engineering leads miss. When the marginal cost of code generation drops by two orders of magnitude, teams treat the resulting pull requests as free inventory. On a balance sheet, unverified code behaves like an unhedged contingent liability.
The psychological trap of review fatigue you described maps directly to adverse selection. When humans evaluate code that took thirty hours to write, the developer's invested labor acted as an implicit pre-filter against trivial errors. With model-generated diffs, that pre-filter disappears. Because the output syntax is uniformly polished, reviewers inevitably substitute aesthetic plausibility for logical soundness. Errors no longer look like ugly hacks. They hide inside elegant, idiomatic abstractions that fail only on edge-case tail distributions.
Your third gate, the canary that asserts each gate still fires, addresses the most dangerous failure mode in systems monitoring. In quantitative risk, the most catastrophic model breakdown is sensor silence. A risk model that stops flagging anomalies creates the illusion of a tranquil regime right before liquidity vanishes. Teams celebrate declining gate triggers as improved code quality when it usually indicates the test harness is drifting out of sync with the underlying dependency graph. Moving verification into deterministic code only holds up if you actively price the maintenance cost of keeping those assertions alive.