- Book: Observability for LLM Applications
- Also by me: Thinking in Go (2-book series) — Complete Guide to Go Programming + Hexagonal Architecture in Go
- My project: Hermes IDE | GitHub — an IDE for developers who ship with Claude Code and other AI coding tools
- Me: xgabriel.com | GitHub
Anthropic won't merge a pull request to the Claude Code repos while a review finding is still open. Code Review for Claude Code, the product Anthropic took out of research preview on October 15, never fails a check at all. The same day, the Claude Code team published how it reviews its own code, and the gap is right there in Anthropic's own words.
Inside Anthropic: "A PR can't merge while any finding from the bug review or the security review is still open" (Claude Code team).
In the product you can buy: "A review never approves or blocks your pull request, and the check run never concludes as a failure" (Code Review docs).
That is deliberate. The team post repeats the product's position: approving a pull request is "still a human call; the product posts findings, not approvals" (source). If you want findings to stop a merge, that part is your job, and the docs show two ways to do it.
What went GA on October 15
Code Review is now "available to Claude Team and Enterprise organizations." Admins choose "Quick, Standard (default), or Deep for each repository," and "Most reviews now finish about 3x faster (~20 minutes down to 6-10 minutes) compared to during preview" (announcement).
Per the docs, a few agents read the diff and the code around it in parallel, and a verification pass throws out false positives before anything is posted. On the PR you get inline comments plus a check run named Claude Code Review, with each finding tagged Important, Nit, or Pre-existing (severity levels). A review runs on Anthropic's infrastructure with read-only access and never builds or runs your code.
It works only with repositories on GitHub, and it isn't available to organizations with Zero Data Retention, the HIPAA configuration, or customer-managed encryption keys. On any plan, you can still run /code-review locally in a Claude Code session.
The numbers behind it, and who they come from
The team post says "Claude writes about 90 percent of Claude Code's own code," and that "From June to September, merged pull requests to Claude Code's codebase more than doubled while the share that had to be reverted fell by more than 40 percent" (source). These are Anthropic's numbers about Anthropic's repos, given without base counts. The Next Web notes that Anthropic's performance claims for the product haven't been independently tested.
The goal: "most pull requests in the Claude Code repos ... merge with at most one human in the loop, who can be the author, with no increase in reverts or incidents." A classifier reads each change and picks a risk tier (source):
| Risk | Examples from the post | Human approvals |
|---|---|---|
| Low | Docs, copy, internal-only features, additive changes behind a flag | None, once automated review and security checks are green |
| Medium | Ordinary product changes, caching, the auto-updater, system prompts | One |
| High | The highest-risk code | Two |
A few rules from the post are worth copying even if you never buy the product. "The change sets the tier, not the author." "Whoever prompted the change is on the hook for it." "An approval covers one specific commit." And "Some paths always need a person, anything owners have listed as sensitive and any CI workflow file."
The post is upfront about the gap: "We don't yet have circuit breakers that tighten approval when those numbers move; engineers here want them before more code moves to zero-human merges."
Make findings block a merge in your repo
"You can merge without responding to a finding, unless your repository's own rules require it" (docs). Two rules do it.
Option 1: require resolved conversations. GitHub branch protection has a rule that "Requires all comments on the pull request to be resolved before it can be merged to a protected branch" (GitHub docs). Findings are posted as inline threads, so an open one blocks the merge. Push a fix and the next review resolves its own thread when it judges the issue fixed, or resolve it by hand: "Replying alone doesn't dismiss a finding." The catch is that the rule covers every thread, nits and human comments included. It also can't see a finding GitHub wouldn't accept as an inline comment: that one shows only as an annotation in the Files changed tab and in the check run's Details table (docs).
Option 2: gate on Important findings only. The check run's Details text ends with a machine-readable line that gh can read. Its normal key counts Important findings, including open ones from earlier reviews. The script below chains the two gh api calls from the docs:
#!/usr/bin/env bash
# Fail when Claude Code Review has open Important findings.
# Usage: gate.sh OWNER/REPO <head-commit-sha>
set -euo pipefail
repo="$1"
sha="$2"
id=$(gh api "repos/$repo/commits/$sha/check-runs" --jq '
[.check_runs[]
| select(.name == "Claude Code Review"
and .status == "completed") | .id]
| max // empty')
if [ -z "$id" ]; then
echo "No completed Claude Code Review check on $sha" >&2
exit 1
fi
important=$(gh api "repos/$repo/check-runs/$id" --jq '
(.output.text // "")
| (split("bughunter-severity: ")[1] // empty)
| split(" -->")[0] | fromjson | .normal // empty')
if [ -z "$important" ]; then
echo "No verdict line on check run $id" >&2
exit 1
fi
echo "Open Important findings: $important"
[ "$important" -eq 0 ]
Run it as a pre-merge step once the Claude Code Review check run has completed, not on PR open or push. A review takes minutes, so a job that fires earlier finds no completed run and fails every PR. The script also fails closed on purpose: the docs say to treat a missing line "as no verdict rather than as zero findings," and that "some completed reviews have none either." Expect the odd clean PR to need a fresh @claude review comment before the gate passes. I haven't run this against a live Code Review check.
Before you trust either gate, protect your default branch. REVIEW.md can lower severity, and Option 2 counts severity, so who can edit that file matters. Code Review reads REVIEW.md "from the pull request's base branch or from the default branch, never from the pull request's own changes" (customize docs), so a PR can't loosen its own review. Rules that lower severity, exclude paths, or cap nits are honored from the root REVIEW.md only when the default branch is protected. The local /code-review reads CLAUDE.md, not REVIEW.md.
What a review costs
There is "no flat per-review price." Reviews are "billed on token usage through usage credits, separately from your plan's included usage," to the organization (costs docs). During the preview, reviews cost $15 to $25 each, The Next Web reports.
Watch the default: a repo you add "starts at After every push," which multiplies cost by the number of pushes. The docs suggest piloting a few repos on "Once after PR creation" or Manual, reading the Average cost column, and setting a monthly spend limit. When the limit runs out, reviews are skipped and the PR gets a comment saying why. Eligible Enterprise organizations can claim "at least $10,000 in Code Review credits by January 15, 2027," terms apply (announcement).
The layer before the PR
Code Review starts once a pull request exists. Anthropic's first layer runs earlier: Claude runs the tests, build and linter, and a subagent reviews the diff in a fresh context before the PR opens. The team post says Boris Cherny, who created Claude Code, estimates that this verification makes the final result two to three times better.
I build Hermes IDE, so weigh this with that in mind. That pre-PR layer is where Hermes sits. Each task runs on its own hermes/<name> branch in its own worktree, and every agent turn is saved. The Review Desk (⌘G, Ctrl+Shift+G on Windows and Linux) shows changes by turn or by file with risk flags, and sends a line comment back to the agent that wrote the code. Its Land step then commits, opens the pull request, or merges locally, and you can undo it.
The Review Desk (⌘G) in By turn view: two turns of a Claude Code task, turn 1's diff of src/db.ts, and a commit message drafted from the turns.
Hermes doesn't run Code Review and doesn't replace CI. It's free, source-available under BSL 1.1, on macOS, Windows and Linux, and not affiliated with Anthropic or any agent vendor.
Would you merge on zero human approvals?
Anthropic's own repos already let owners decide which low-risk changes, such as additive changes behind a flag, merge on automated checks alone (source). They do it without the circuit breakers their engineers want in place before more code merges with zero humans.
Would you let a classifier pick the tier for your repo, or does every PR still need a person? Tell me in the comments.
If this was useful
If you want an agent's work checked before it ever reaches a PR, download Hermes IDE and open the Review Desk on its branch.
An AI reviewer is an LLM feature in production, and Anthropic runs it like one: past incidents become an eval set the reviewer runs against daily, and noise is measured by what people do with each comment. Observability for LLM Applications covers that work for your own LLM features, from traces and evals to cost and quality regressions. If you're about to put an AI reviewer on your PRs, it's the part to plan before the bill arrives.
Top comments (0)