Three weeks ago I merged a PR that Claude Code wrote for a side project.
The diff was clean, the tests passed, CI was green. I skimmed it, approved it, moved on. Two weeks later I found out the authorization check on a delete endpoint had quietly disappeared during a refactor earlier in that same session. Nobody exploited it. I just got lucky.
That's when it clicked that AI-generated code doesn't fail the way human-written code fails. A human engineer forgets an edge case or doesn't know a library has a built-in for something. An AI coding agent writes a plausible-looking auth check, a plausible-looking test, a plausible-looking API call, and "plausible" is exactly the failure mode that slips past a reviewer skimming a diff at 6pm. None of it shows up as a red squiggly line. It just looks finished.
So I went back through a stack of AI-generated PRs, mine and a few teammates shared with me, looking for a pattern instead of individual bugs. Six showed up over and over: hardcoded secrets dropped in for a "working example," authorization checks that check you're logged in but not that you're allowed to touch this specific resource, tests that mock the exact function they're supposed to be testing (so they pass no matter what the implementation does), API calls that look right but don't match the actually-installed library version, edge cases that fail silently instead of throwing, and scope creep where the agent "improves" an adjacent file nobody asked it to touch.
I built a toolkit around those six patterns: what each one looks like in a real diff, the exact review question that catches it, a 45-item checklist organized by risk category, a 1-5 risk scoring rubric so you know how deep to review before you start, and a companion spreadsheet that logs every PR you review so the pattern becomes visible over time instead of staying anecdotal, which AI tool in your stack actually produces the most rework, not a guess.
It's for solo developers and small teams (2-15 people) who've adopted Claude Code, Cursor, or GitHub Copilot for a meaningful share of their code and are currently reviewing AI-generated PRs the same way they'd review a junior engineer's: reading the diff, trusting green CI, moving on. If that's you, the 5-minute triage checklist alone is worth running on your last three merged PRs before you read anything else. Most people find something on the first pass.
If you want the full toolkit, a 16-page PDF plus the PR risk tracker workbook, it's here: https://logi2.gumroad.com/l/lpdebo
The thing I keep coming back to is that "the code looks finished" and "the code has actually been checked" are two different claims, and AI coding agents are very good at making the first one true regardless of the second. Worth remembering the next time a diff looks a little too clean.
Originally published on Medium.
Top comments (0)