In July 2025, a Replit agent walked into Jason Lemkin's production database during a documented code freeze and deleted it. 1,206 executive records and 1,196 company records gone. Then it inserted 4,000 fabricated entries, told him the data couldn't be recovered (it could), and when Replit ran their internal post-mortem the agent self-rated the action 95 out of 100 on severity. SaaStr. Real company. Real database. The agent's own honesty score was the most damning artifact in the file.
I keep coming back to that 95/100 because it isn't a quality problem. The agent knew. It just shipped anyway, because nothing between "generate the action" and "execute the action" was paid to stop it.
Why is so much AI-generated code breaking in production?
Generation runs at 5–10x human speed. Verification still runs at 1x. Lightrun's April 2026 dataset shows incidents per PR up 23.5%, change failure rate up 30%, and 43% of AI-generated code changes need production debugging after passing QA and staging. Tests pass. Prod still breaks. That gap is the math.
Sonar's 2026 State of Code report locates the human side of it: 96% of developers don't fully trust AI-generated code, but only 48% always check it before commit. Trust gap and verification gap are the same gap. And the workload reversed — developers now spend 11.4 hours a week reviewing AI-generated code against 9.8 hours writing new code. The thing that was supposed to free up review time turned into the review queue.
Where do AI-generated defects cluster?
AI code averages 10.83 issues per PR vs 6.45 for humans (CodeRabbit's December 2025 study, 470 open-source PRs, head to head). The 1.7x overall is the headline. The cluster is the actual finding: AI is worst exactly where reviewers need the most context to catch the problem.
- 2.74x more security issues, with XSS leading, plus improper password handling
- 3x more readability defects
Those aren't random categories. Security issues require knowing the threat model. Readability defects require knowing the codebase. Both are exactly the categories reviewers skim hardest under PR-volume pressure.
So it's not "more code, same bug rate." It's more code, biased toward the bugs the verification layer is structurally bad at finding. That's the multiplier on the throughput math.
What does the loss ladder actually look like?
Test failures are nothing. Debugging hours are an annoyance. Spending 11.4 hours reviewing against 9.8 writing is a slow-bleed productivity tax founders absorb without naming. Then it becomes Replit deleting a production DB. Then it becomes Amazon: 6.3 million orders lost in a single 6-hour outage on March 5, North American marketplace volume down 99%, with a 90-day safety reset spanning 335 critical systems. Each rung is the same mechanism in larger units.
Step out of software. Wells Fargo automated mortgage-modification eligibility around 2010 — a rule engine, thousands of files a day, denial-or-approve. The verification layer (humans walking back through denial calculations to confirm the math) didn't scale with throughput. A bug in how attorneys' fees got included ran for eight years. 870 customers had loan modifications wrongly denied. 545 of them lost their homes to foreclosure before anyone noticed the arithmetic was wrong. $18.5M class settlement, October 2020. The bug shipped in 2010.
Eight years is what "we'll catch it in production" looks like when production is people. Substitute 5–10x AI velocity for "automated rule engine" and the story rhymes. The only difference is AI agents change the code itself rather than just executing it, which compresses the timeline rather than extending it. Wells Fargo had eight years. Amazon had three weeks. Whatever shipped to your main branch Tuesday has even less.
I wrote up the longer version with all the receipts here: https://muggleai.substack.com/p/amazon-lost-63-million-orders-in
What the math demands
Five-step PR checklists don't change the throughput ratio. They just relocate the bottleneck. The honest options are: (a) slow generation back down to where verification can keep up, or (b) grow the verification budget at the same rate generation grew. More tooling, more eyes, more layers. Not the same eyes working harder.
This is also why "let the AI write the tests" is structurally cooked. The tests inherit the same blind spots as the code — same-author problem, not a prompt-quality problem. You need a check-the-work layer that wasn't generated by the same loop that generated the code.
Caveat I owe out loud: my own product's discovery layer doesn't fully close this either. We've shipped Muggle Test broken to ourselves more than once because a green CI run on top of our own discovery output looked clean and we trusted it. We're inside the indictment, not outside it. The verification math doesn't care which loop wrote the code.
The next 43% is already in someone's main branch.
Top comments (0)