A preprint that hit arXiv in August (Michels et al.) put a number on something every senior dev already felt: code-review time is up 441% under vibe coding. Generation got fast. Verification didn't.
I'm building Code Sonar, a tool that scans GitHub repos and scores code health from 300 to 850, with plain-English findings that include the file, line, severity, and a fix-time estimate. Free scans are coming — there's a waitlist at the end of this post. But regardless of what tooling you use, here's the 10-minute health check I run on every AI-generated pull request before it gets near main.
1. Read the diff, not the summary
The PR summary the agent wrote is marketing copy. The diff is the product. I scroll the actual changes top to bottom, and I specifically look at files I didn't expect the change to touch. AI-generated PRs have a habit of "helpfully" refactoring adjacent code or touching config nobody asked about.
2. Hunt for duplicated utilities
This is the single most common thing I find. GitClear's analysis of 211 million lines of code found copy-paste patterns rose 48% after AI adoption while refactoring dropped 60%. The model doesn't know your codebase has a formatCurrency helper three files over — it just writes another one. Search for the function name and its obvious synonyms before approving.
3. Interrogate the unhappy paths
AI-generated code is relentlessly optimistic. It handles the happy path beautifully and waves vaguely at everything else. My mental checklist: What happens when the API returns an error? When the token expires? When the input is empty, malformed, or enormous? A password-reset flow I once reviewed worked perfectly in testing — and quietly never invalidated old reset tokens, so every link stayed valid forever. The code looked like a tutorial because it basically was one.
4. Ask "where are the tests?"
Researchers at Missouri S&T gave this a name in 2026: GIST, GenAI-induced self-admitted technical debt. When they combed GitHub for AI-related TODO/FIXME comments, developers most often flagged AI code for incomplete implementation and deferred testing — not design flaws. The structure usually looks fine. The verification is what's missing. If a PR adds behavior without tests, that's the review, not a footnote.
5. Get a second pair of eyes that isn't you
You're tired, the diff is 400 lines, and it "looks right." That's exactly the moment bugs slip through — an Anthropic study found AI-assisted engineers scored 17% lower on code comprehension quizzes, not because of the tool but because of the posture: passive "just make it work" delegation. Run your linter, your type checker, your security scanner. Whatever you run, run it on AI-generated code with more suspicion, not less.
The uncomfortable math
None of this is anti-AI. I use these tools every day — the velocity is real. But output volume got conflated with productivity, and the verification bill always comes due. Ten minutes of structured skepticism per PR is the cheapest insurance in software right now.
I'm building Code Sonar to automate the boring parts of this checklist: point it at a GitHub repo, get a 300–850 code-health score and plain-English findings with file, line, severity, and fix-time estimates. Free scans are on the way — join the waitlist here: https://muse.ai/s/waitlist-page-zxc6q7xbxii91n
What's the worst thing you've caught in an AI-generated PR? I want the horror stories.
Top comments (0)