TL;DR: Traditional code review checklists are tuned to catch human failure modes — typos, inconsistent naming, forgotten edge cases someone was rushing past. AI-agent-written code fails differently: it's fluent, consistent, and confidently wrong in ways that don't trigger a reviewer's normal pattern-matching. Below is the checklist we actually use now, split into what stayed, what we added, and what we deleted because it stopped being useful.
Why the old checklist quietly stopped working
A traditional review checklist assumes the person who wrote the code was rushing, tired, or unfamiliar with part of the system — so it optimizes for catching inconsistency. AI-agent output isn't inconsistent. It's uniformly fluent, which is exactly the problem: uniform fluency reads as competence to a reviewer's eye, even when the underlying logic has a real gap.
We didn't redesign our process on a hunch. We redesigned it after tracing a handful of production bugs back to PRs that had technically passed every item on our old checklist.
What we kept, unchanged
Some things don't care who — or what — wrote the code:
- Does this match the actual ticket, not an adjacent interpretation of it
- Are tests testing behavior, not just re-asserting the implementation
- Does the diff touch anything outside its stated scope
- Is there a rollback plan for anything touching production data
What we added specifically for agent-written PRs
1. "Explain why, not what" as a required PR comment.
We require one sentence per non-trivial change explaining why this approach was chosen, written by the human who reviewed and merged it — not copy-pasted from the agent's own summary. If the reviewer can't articulate why, that's the signal to slow down, not a formality to skip.
2. A dedicated pass for "plausible but untested" edge cases.
Agents are good at handling the edge cases you explicitly describe and surprisingly inconsistent about the ones you didn't think to mention. We now require reviewers to list two edge cases not covered by existing tests and manually verify behavior for at least one, specifically on agent-authored logic.
3. Mandatory second reviewer on auth, payments, and external data boundaries.
Independent analysis of AI-coauthored pull requests has found meaningfully elevated vulnerability rates compared to human-only PRs. We treat any agent-touched file in these categories as requiring two sign-offs, full stop — no exceptions for "it's a small change."
4. A "regenerate vs. refactor" checkpoint.
When something needs fixing, it's faster to ask an agent to regenerate a working version than to refactor the existing structure. That's exactly how duplicated logic quietly accumulates across a codebase. We now flag any PR where an agent produced a parallel implementation of something that already exists elsewhere, and require a decision — consolidate or document why not — before merge.
5. Provenance labeling on every PR.
Every PR gets tagged with roughly how much of the diff is agent-generated versus human-written. Not for blame — for calibrating how much scrutiny it gets. A PR that's 80% agent-generated gets read differently than one that's a two-line human fix.
What we deleted
- Style nitpicking as a review category, almost entirely. Agents are more consistent about formatting and naming conventions than most humans ever were, so linting handles this now and reviewers stopped spending time on it.
- "Did you write a test" as a checklist item. Agents write tests by default, often more of them than a rushed human would. The relevant question moved from did you write a test to does this test actually assert the right thing — which is a harder, more useful question.
A quick comparison, if you're deciding what to change first
| Review focus | Old checklist | Rebuilt checklist |
|---|---|---|
| Primary failure mode assumed | Human rushing / inconsistency | Confident but subtly wrong logic |
| Style and formatting | Manual reviewer check | Delegated to linting |
| Test coverage | "Is there a test" | "Does the test assert the right behavior" |
| Security-sensitive files | Standard single review | Mandatory second reviewer if agent-touched |
| Duplicated logic | Caught opportunistically | Explicit regenerate-vs-refactor checkpoint |
The honest tradeoff
This is more review overhead per PR, not less — which runs against the "AI makes everything faster" narrative you'll see elsewhere. It's true that agents make writing code faster. Review didn't get faster; it got more targeted. Net delivery speed still went up for us, but not because review shrank — because the boilerplate and scaffolding time it used to take disappeared, and that gap was bigger than the review overhead we added.
At 6senseHQ, this checklist is now baseline practice across client work, not an internal experiment — because "the agent wrote it and it looked fine" isn't an answer we're willing to give a client when something breaks in production.
FAQ
Should every AI-agent-generated PR get extra scrutiny, or just some?
Scope it to risk, not volume. Low-stakes, well-tested, non-sensitive changes don't need extra process. Anything touching auth, payments, external data, or shared core logic does, regardless of how small the diff looks.
Does provenance labeling slow teams down?
Marginally, and it's worth it. Tagging a PR takes seconds; the reviewer time it saves by setting the right scrutiny level upfront is larger than that cost.
Is this checklist specific to any particular AI coding tool?
No — it's built around failure patterns common to agentic code generation broadly, not any single vendor's tool. The specific tool matters less than whether your review process accounts for how it fails.
Most "code review in the AI era" advice online is either "trust the AI" or "don't trust the AI." Neither is a process. Pick the two or three items above that address your team's actual failure pattern and start there — the whole checklist rebuilt itself gradually, one incident at a time, not in a single meeting.
Top comments (0)