I have a question for those who are using AI to write code.
What is the basis for your "OK" during the review?
There are 40 files changed, and there’s no time to read them all. The tests pass. It looks fine at a glance. So, merge it. — This is not a review; it is a ritual of approval.
While building 10 personal apps in three months, I encountered this state repeatedly.
To escape it, I didn’t change the reviewer’s ability; I changed the basis for judgment.
Why It Becomes a Formality
There are two reasons.
1. No Criteria for Judgment
If the issue only says "Implement login functionality," the reviewer has no choice but to judge based on
"Whether the login functionality works." This cannot be judged objectively.
It passes if it looks like it works.
2. The Author Is Reviewing Their Own Work
Having AI write the code and then asking it to "review this" in the same conversation is the most dangerous pattern.
The reviewer holds all the context of the implementation process. Because they know "why it was written this way,"
they fill in the gaps for what isn't written. This is the same reason humans cannot effectively review their own code.
What I Changed
Write Acceptance Criteria Before Implementation
This had the biggest impact.
| Writing Style | Judgable? |
|---|---|
| "Login functionality works correctly" | No |
"Logging in with an unregistered email returns 401 and USER_NOT_FOUND" |
Yes |
| "Improve performance" | No |
| "Initial list display loads 200 items within 500ms (local measurement)" | Yes |
Writing it before implementation is crucial. If you write it later, the criteria will just match the existing implementation.
Don't Give the Reviewer Context
Perform reviews in a separate session or as a different persona. There are only two inputs to provide:
- Acceptance criteria from the Issue
- The PR diff
Do not pass the conversation history of the implementation. Not having context allows you to notice what isn't written.
You might want to add explanations like "This PR intends to clean up authentication...", but
don't do it. If an explanation is needed, it should be in the Issue or PR description.
Check Cheap Things First
Judgment requires order.
- Scope Violations — Can be judged just by looking at the list of changed files
- Tests & Lint — Already judged by machines
- Acceptance Criteria — Requires reading code
If a rejection is decided in an earlier stage, do not look further. Providing all comments before returning it only increases rework because fixing one thing changes others.
And step 1 is really cheap.
gh pr diff <number> --name-only
This alone tells you if anything outside the scope defined in the Issue has been touched.
Even with a diff of 40 files, looking at the list immediately reveals "why routing changed for an authentication issue."
Reject Features Not in the Criteria
Often overlooked, but this is effective.
"I made it convenient while I was at it" means unreviewed changes are entering main.
Features not in the acceptance criteria have no basis for judgment.
The countermeasure is to ask the author to split that part into a separate Issue. We are not discarding it.
How to Write Rejections
When rejecting, include three things:
- What is the problem — Relevant file and line
- Why it is a problem — Which acceptance criterion or scope it violates
- What needs to happen to pass — In a judgable form
"Please clean this up more" is not a rejection. Since you cannot judge what needs to be done to pass, the next submission will also fail.
Delegate State Management to Machines
To focus on judgment during reviews, I delegate label management to GitHub Actions.
on:
pull_request:
types: [opened, reopened, ready_for_review, closed]
pull_request_review:
types: [submitted]
Identify the Issue from Closes #<number> in the PR description and swap labels (status:review / status:changes-requested / status:done) based on PR activity.
Leaving label management to a persona inevitably leads to missed updates.
It is more stable for machines to hold state, while personas focus only on "what was done."
Acknowledging Limitations
The reviewer uses the same model as the implementer.
Even with separated context, they are fundamentally the same model. Systematic errors inherent to the model cannot be caught by the reviewer. If a library's API is consistently misremembered, the implementer will make a mistake, and the reviewer will judge it as correct.
This cannot be solved by configuration. The countermeasure is to include executable verification in the acceptance criteria.
- [ ] Logging in with an unregistered email returns 401 and USER_NOT_FOUND
→ Verify that a test exists for this
"Tests passing" is one of the few ways to externally verify the model's assumptions.
Therefore, write acceptance criteria in a form that can be converted into tests as much as possible.
Summary
- Write judgable acceptance criteria before implementation
- Do not give reviewers context about the implementation process
- Check scope violations first. Judging via file lists is cheap
- Reject features not in the criteria. Split them into separate Issues
- Delegate state management to machines to focus on judgment
- Cannot catch model-common errors. Convert acceptance criteria into executable verification
I have documented this judgment procedure as a definition for the reviewer persona and distributed a configuration combining it with implementer and designer personas. The free version for 4 personas is available under MIT license.
I publish the configuration for splitting Claude Code into separate personas —
Architect, Coder, Reviewer, Conflict Resolver — under MIT. Copy it, run
./setup.sh, and it works. It does not depend on your tech stack.
https://github.com/quintetkit/quartet
I built one real tool using nothing but this workflow. Every Issue, PR, review
and merge is still there. The parts that went wrong were not deleted.
https://github.com/quintetkit/mdlinkcheck
The version that adds a UI Designer persona, review criteria, a per-Issue
parallel execution script and a 10-chapter guide is on the
product page.
Top comments (0)