Reviewing PRs in an Agentic Era

Code Review When Agents Do the First Pass: What Still Matters for Humans

A couple of weeks ago, Anthropic published their model for internal code review: several agents review each PR, a second group votes on the issues they find, before it gets to human evaluation. The risk level assigned determines how many human reviewers (if any) are needed.

That model is compelling and is the future for many engineering teams, but it assumes a lot of tokens and a full agentic setup. Many engineering teams don’t have that budget, and many don’t need the full machinery right now, but, even so, most teams are using some form of AI in code review. So, if agents are already reviewing the PR, what changes for us as human reviewers?

What code review is for

Anthropic noted the two jobs of code review:

1. Find and fix issues before they ship.
2. Build shared context and codebase understanding across the team.

Agents are increasingly good at the first but the second is where the human role needs rethinking.

Human Role

Anthropic’s article lists the decisions that machines handle poorly and that humans should keep:

• Should we build this at all? Direction, taste, and scope of the changes.
• Risky code. Anything where a mistake is expensive.
• Architecture that sets a pattern.
• Reading the evidence. Test outputs, screenshots, and the reviewer’s findings.
• Owning the outcome. Accountability for every change that ships

What’s changed

Accountability is thinner on the approver side. The author still owns the change. But when an agent has already caught most mechanical mistakes, a human approver can drift into rubber-stamping. That’s somewhat okay, since AI is good at catching errors. It’s a problem for the things AI is less good at. As approvers, we still need to take accountability for the code that’s being merged - it’s just that we should focus less on catching bugs and more on whether the code is good taste.

Sean Goedecke describes engineering taste as ‘composed of the set of engineering values you find most important’. One project prioritises scalability over readability; another wants simple, well documented code. Of course, in a well setup project, our AI tools should be equipped to write code according to our taste - it should know for example whether to favour detailed or simple docstrings. In practice, it doesn’t always hold. Agents don’t always reliably code to your taste - this is where human review comes in.

Experienced engineers carry a mental map of the codebase. When they review a change, they know what to check based on that map. Agents don’t have it by default, but you can give it to them. Good context practices help here: CLAUDE.md files for project conventions, and review.md files that spell out what a reviewer should look for in this codebase. You might add something like the below into your files.

# CLAUDE.md  
## Code taste
- Names say what a thing *is* or *does*, not how it's implemented.
  `unpaidInvoices`, not `filteredList`.
- Prefer boring, explicit code over abstractions. Three similar blocks
  are fine; extract only when a fourth appears or the duplication has
  already caused a bug.
- Functions do one thing. If you need "and" to describe it, split it.
- Handle errors where you can do something useful about them. Otherwise
  let them propagate with context. Never swallow an exception silently.
- Comments explain *why*, not *what*. If the what needs explaining,
  rewrite the code.
- Delete dead code; don't comment it out. Git remembers.

# review.md

Review for correctness and risk first, style last. Linters handle formatting;
don't comment on it.

## How to write feedback 
- Lead with blocking issues, clearly labelled **[blocking]**.
  Mark everything else **[nit]** or **[question]**.
- Say *why* something is a problem and suggest a fix where possible.
- If the PR is fine, say so briefly. Don't invent feedback to look thorough.
- Do not approve if you couldn't verify something important; say what
  you couldn't check.

## 1. Does it do what it claims?
- Read the PR description, then check the diff actually delivers it,
  and nothing else.
- Walk the unhappy paths: empty input, nulls, timeouts, partial failure,
  retries, concurrent requests.

## 2. What could break that isn't in the diff?
- xxx

## 3. Security and data
- xxx

## 4. Operability 
- xxx 

## 5. Tests  
- xxx

The review itself now needs review. If agents are doing the first pass, their output needs measuring. Run evals on agentic reviews, check where they miss things your team cares about, and update the context files accordingly. The mental map improves over time only if you maintain it.

A practical starting point

For teams without a large agentic review budget, a lighter setup might look like this:

• Keep human review for risky paths (big changes, core parts of the codebase, code that is susceptible to breaking changes) and changes that set patterns.
• Write CLAUDE.md and review.md files that capture your taste.
• Track the misses: when a human catches something the agent didn’t, record it and feed it back into the context files.
• Make the approver’s responsibility explicit, so “claude looked at it” never substitutes for owning the change.

Conclusion

Agents change the mechanics of code review more than its purpose. The first job, catching issues, is increasingly automated. The second job, building shared understanding, depends on keeping the team’s taste and codebase knowledge written down where the agents can use it, and checked by humans who still own what ships.