Cross-post note: Condensed syndication of the full article on
aymen.benyedder.top.
Thecanonical_urlpoints at the original so search engines attribute ranking to the source.
The one-line summary
41% of all code produced globally is now AI-generated — and in some organizations AI authorship already sits at 90%. Only 29% of developers trust the accuracy of the tools that write it, and 66% report spending more time debugging AI output than expected.
Treat AI code like code from a capable junior who is fast, confident, and occasionally hallucinating: verify behavior by execution, not by reading; treat tests as evidence, not proof; resolve every suggested dependency against the registry; and never auto-merge. The bottleneck has moved from generation to verification.
The trust collapse is measurable
Stack Overflow's 2025 survey (49,009 developers) shows a workforce that adopted AI universally and lost faith in it simultaneously:
- 84% use or plan to use AI tools, up from 76% in 2024
- Trust in accuracy of AI answers fell 40% → 29% in a single year
- Only 3% "highly trust" AI output; 46% actively distrust it
- Between 45% and 66% name "almost right, but not quite" as their top frustration — the exact failure mode that turns a five-minute review into a forty-minute archaeology session
The perception gap: devs think it's faster. It isn't.
METR's randomized controlled trial (July 2025, arXiv:2507.09089) gave 16 experienced OSS developers 246 real issues from their own codebases with randomized AI access. The result was a clean three-way split between belief and measurement:
- Forecast: 24% faster
- Self-assessed after the study: ~20% faster
- Measured effect: 19% slower (CI: +2% to +39% — no scenario constituted a speedup)
Developers accepted fewer than 44% of AI suggestions. The February 2026 follow-up narrowed the deficit to ~4% slower — less negative, but still no speedup. The constraint has moved: the old constraint was writing code. The new constraint is reviewing it.
The security data nobody reads
- 45% of AI-generated code introduces an OWASP Top 10-class flaw (Veracode, 100+ LLMs, 80 tasks)
- AI-generated code carries 2.74x more vulnerabilities than human-written code
- Java is the worst case: 70%+ failure rate; 86% of AI-generated XSS attempts fail
- The trend line is the alarm: syntax pass rate climbed ~50% → ~95% since 2023, while the security pass rate stayed flat at 45–55%
CodeRabbit's analysis of 470 open-source PRs independently found AI-contributed PRs create 1.7x more issues and are 1.88x more likely to introduce a vulnerability.
The maintainer revolt
- cURL shut down its $86,000 bug bounty: 20% of submissions were AI garbage, valid-report rate fell 15% → 5%
- Ghostty: zero tolerance for drive-by AI PRs — "not anti-AI... anti-idiot"
- QEMU, Gentoo, NetBSD, Debian, Cloud Hypervisor formally ban or restrict AI contributions (QEMU's Developer's Certificate of Origin argument)
- Flux CD's Stefan Prodan: "AI slop is DDOSing OSS maintainers"
- GitHub shipped the ability to disable pull requests entirely
The 10-point verification checklist
- Read the tests first. Do they assert behavior, or echo the implementation? Delete any test that re-implements the logic under test.
- Run the code, don't read it. Execute the happy path and the error path.
- Check for deletion smell. A PR that only adds code and never removes any is building the wrong abstraction.
- Verify dependencies resolve. Sonatype found 27.76% of AI upgrade recommendations reference versions that don't exist.
- Search for silent swallow. Empty catches, ignored return values, discarded errors — where incidents are born.
- Trace security-sensitive paths yourself. Auth, crypto, payments, filesystem: manual trace from input to trust boundary.
- Run SAST/SCA + mutation testing. Mutation testing specifically catches self-referential tests: if killing a mutant doesn't fail the suite, the tests aren't testing the code.
- Check conventions. Must match your repository, not the model's training distribution.
- Verify error paths actually trigger. A generated error path that can never fire is dead code that lies.
- Timebox verification. Reviewing a 400-line generated diff is not faster than writing 80 lines by hand.
Trust tiers — escalate by risk, not by volume
- Tier 0 — Boilerplate, config, docs: light review. Run it once, move on.
- Tier 1 — Business logic, CRUD, tests: normal review plus running the suite.
- Tier 2 — Auth, crypto, payments, infrastructure, concurrency: full senior review, SAST, and a threat model. Never auto-merge. Treat generated tests as inadmissible evidence until mutation-tested.
Hybrid-strict gates (auto-approving nothing above Tier 1, named human approver at Tier 2) show 1.7% defect escape vs 4.1% under auto-approve — and the tier system only works if the gates are enforced mechanically, not by honor.
Read the full version
The complete guide — the failure-mode taxonomy, the tooling stack that catches what eyes miss (SAST, SCA, secrets detection, provenance attribution), and 18 sourced references — is on
aymen.benyedder.top.
Top comments (1)
If AI writes 41% of code, who tests the testers?