Uber writes that its AI code review platform, uReview, treats automation "not as a substitute for human insight, but as a scalable partner that enhances engineering productivity", and reports over 65% of uReview's posted comments addressed, counting a comment as addressed when five re-runs on the final commit no longer produce a similar one. DoorDash reports that 60.2% of settled high and critical findings ended with the engineer changing code before merge (n=2,256). Atlassian reports 38.70% of its reviewer's comments followed by a code change in the next commit, and a 44.45% resolution rate for human comments. Each company counts a different event, so these are three counting rules rather than one score, and none shows whether a comment caught what mattered.
Every number below comes from the company's or the authors' own page as I read it on 25 October 2026; none of them saw this text before publication.
What companies measured on their own pull requests
Seven sources report what an AI reviewer did on real pull requests, each with its own counting rule and its own scale.
| Who, when | What they counted | What they report, with the counting rule | Scale and period |
|---|---|---|---|
| An AI review tool at one company, in a study presented at ICSE 2025 (arXiv, revised 28 December 2024) | automated comments that developers acted on; time to close a pull request | "73.8% of automated comments were resolved.", counted from "comment labels indicating whether developers acted on the automated comments". Average pull request closure time went from five hours 52 minutes to eight hours 20 minutes, "with varying trends across projects". | "three projects with 4,335 pull requests, 1,568 of which underwent automated reviews" |
| Microsoft, Engineering@Microsoft, 14 July 2025 | median time to complete a pull request | Median PR completion time improved by 10 to 20 percent in 5,000 repositories onboarded to Microsoft's AI reviewer, by what the post calls early experiments and data science studies. | the reviewer "started as an internal experiment and now has scaled to support over 90% of PRs across the company impacting more than 600K pull requests per month" |
| Uber, uReview, 12 August 2025 | posted comments addressed in the same changeset; comments rated useful | Over 65% of posted comments were addressed, where a comment counts as addressed when none of five re-runs of uReview on the final commit reproduces a semantically similar comment. "Engineers who interact with the tool mark 75% of its comments as useful". Separately, "Internal audits show that only 51% of human-written comments are considered as bugs by the author and addressed in the same changeset." | "Each week, uReview processes over 10,000 commits, excluding configuration files." |
| Anthropic, Code Review launch article, 9 March 2026 | findings that Anthropic's engineers marked incorrect; pull requests with substantive review comments; both on Anthropic's own pull requests | "Engineers largely agree with what it surfaces: less than 1% of findings are marked incorrect." and "Before, 16% of PRs got substantive review comments. Now 54% do." | "We run Code Review on nearly every PR at Anthropic."; "We've been running Code Review internally for months" |
| Atlassian, its AI code reviewer, post of 7 April 2026 and a paper accepted at ICSE 2026 | comments followed by a code change in the next commit; median PR cycle time | 38.70% of the reviewer's generated comments "led directly to code changes in the subsequent commit", and the resolution rate for human comments was 44.45%. Median PR cycle time fell by 30.8%. | "a year-long, large-scale online evaluation across over 1,900 repositories" |
| Cloudflare, 20 April 2026 | times engineers overrode the reviewer with a break glass comment; findings per review | When a person comments break glass, "the system forces an approval regardless of what the AI found"; engineers needed it 288 times, 0.6% of merge requests. "That is about 1.2 findings per review on average", a number Cloudflare calls "deliberately low". | 48,095 merge requests in 5,169 repositories, 10 March to 9 April 2026 |
| DoorDash, 11 May 2026 | settled high and critical findings followed by a code change before merge | "60.2% of settled high and critical findings result in the engineer changing code before merge (n=2,256), up from a 46% acceptance rate with our previous third-party review agent." The rule: "We measure this on if the human acted on it or not." | "more than 10,000 pull requests across 56 onboarded repositories" in a typical week |
Each row comes from the company's or the authors' own page, read on 25 October 2026; the rows count different things, so no number in one row compares with a number in another.
What each number counts
Most of these numbers count what engineers did after the reviewer spoke, and the event counted changes from page to page. Uber automates its count: "It automatically evaluates if a given posted comment has been addressed by re-running uReview five times on the final commit." Its 75% comes from the engineers who interact with the tool, who can rate each comment as useful or not useful. DoorDash's 60.2% counts only settled findings of high and critical severity, and its 46% belongs to the third-party review agent it used before.
Atlassian counts a comment when the next commit changes the code; its ICSE 2026 paper describes the same 38.70% as "comments that triggered code changes in the subsequent commits". Anthropic counts the findings its own engineers marked incorrect, on its own pull requests. Cloudflare's 0.6% counts the times engineers overrode the reviewer's verdict. Microsoft's figure is a change in time, not a count of comments, and the ICSE 2025 study counts labels on each automated comment.
GitHub defines the "addressed rate" of Copilot code review (CCR) with a model as the judge: it "is the percentage of CCR comments that an LLM determines prompted a developer to make a corresponding code change, based on the diff, thread, reactions, resolution state, and post-review code." A study of 2,604 bot comments at Beko, accepted to EASE 2026, compared two automated ways of grading such comments, G-Eval and an LLM-as-a-Judge pipeline, with the fixed or wontFix label an engineer gave each one, and found "Agreement ratios range from approximately 0.44 to 0.62". Its authors add that "Developer actions such as resolving or ignoring comments reflect not only comment quality, but also contextual constraints, prioritization decisions, and workflow dynamics that are difficult to capture through static artifacts."
A comment that was acted on is not always a comment that caught something important; Has your AI reviewer ever caught something your team missed that actually mattered? is about telling the two apart on your own pull requests.
What research sets built from human reviews find
Research sets start from issues that human reviewers flagged and ask how many of them an AI finds, which is a different question from whether engineers act on AI comments. SWE-PRBench, posted to arXiv on 27 March 2026 as "a benchmark of 350 pull requests with human-annotated ground truth", reports that "8 frontier models detect only 15-31% of human-flagged issues on the diff-only configuration", scored by an LLM judge "validated at kappa=0.75"; these are models given a diff, not review products. c-CRAB, posted to arXiv on 24 March 2026 and "systematically constructed from human reviews", evaluated one open-source and three commercial review agents and reports that "the existing review agents taken together can solve only around 40% of the c-CRAB tasks". Benchmarks that vendors run on their own tools use their own judges and datasets, and Should we trust AI code review benchmarks? goes through how they are built.
Why the teams behind these numbers post fewer comments
Uber, DoorDash, Cloudflare and Atlassian each put a filter between the model and the pull request, and three of them say plainly that they chose fewer comments over more. Uber writes "Early in development, we learned that comment quality matters far more than quantity." and "To preserve trust, we focused on delivering fewer but more useful comments."; its category classifier "suppresses those from categories with historically low developer value", and "Readability nits, minor logging tweaks, low-impact performance optimizations, and stylistic issues consistently received poor ratings."
At DoorDash, "Before any comment is posted, it has to survive a disprove-it pass, an explicit step where the system tries to falsify its own finding." The post ties its number to that choice: "The 60% acceptance number is the output of that tradeoff. We'd rather miss some issues than become noise. A reviewer that's muted catches nothing." Cloudflare explains its average of 1.2 findings per review with "We biased hard for signal over noise". Atlassian runs a model judge that "acts as a gatekeeper, reviewing every generated comment for factual correctness against the code change", then a classifier trained "to classify comments based on whether they led to a code resolution".
Sriramprabhu Rajendran, an engineering leader who wrote on dev.to on 12 October 2026 about adding an LLM reviewer to a Java and Spring Boot pull request process, gives one rule for that reviewer: "The false positive rate should be kept under 15%. Otherwise, developers will dismiss all comments."
Two products describe a check of the same kind on their own pages. Anthropic's docs for Claude Code Review say "A verification step checks findings against the code before they're reported.", and CodeRabbit's architecture page lists a Verification agent among the agents that work in parallel on a review. Sigma (Mnemoverse, the author's company), a reviewer for GitHub pull requests in closed beta, is described on its homepage as "A useful second look". Its docs describe a fresh verifier that "confirms, rejects or reclassifies each one" of the candidate issues, say "only confirmed candidates are verified findings" while suspicions no verifier confirmed stay labeled separately, and add "A verification label is agent-produced evidence, not proof that the code is correct." Sigma runs on Claude Code and OpenAI models. Teams can request access on its homepage, which says "A request does not grant access automatically."
What it does not replace
Five of the companies that built these reviewers write that it works next to people, not in their place. DoorDash writes "The agent is not trying to replace human review.", Atlassian "Code reviewer isn't meant to replace human judgment", and Cloudflare "This isn't a replacement for human code review, at least not yet with today's models." Uber designed uReview "to augment the code review process with a second AI reviewer", and Microsoft gives its goal as "freeing human reviewers to focus on higher-level concerns".
Time moves in both directions in these reports. Atlassian measured a 30.8% reduction in median PR cycle time over its year of evaluation. Microsoft reports a 10 to 20 percent improvement in median PR completion time from its early experiments and data science studies. In the study presented at ICSE 2025, the authors write that the tool "proved useful in software development, enhancing bug detection", while average pull request closure time grew from five hours 52 minutes to eight hours 20 minutes, and they name "drawbacks like faulty reviews, unnecessary corrections, and irrelevant comments".
Anthropic's docs for Claude Code Review state the limit plainly: "A review with no findings doesn't mean your code is error-free or secure."
Sources and method
The numbers are quoted with the counting rule each page gives, none of them was recomputed, the table rows run in order of publication, and each quotation was checked word for word against its page on the reading day.
Sources, all read on 25 October 2026; this text describes them as of that date. Uber: uReview. DoorDash: an AI code reviewer engineers listen to. Atlassian: code reviewer post, ICSE 2026 paper. Microsoft: AI-powered code reviews. Cloudflare: AI code review at scale. Anthropic: Code Review launch, Code Review docs. GitHub: ReviewBench. CodeRabbit: architecture. Studies: Automated Code Review In Practice, ICSE 2025, automated evaluation of review bots at Beko, EASE 2026, SWE-PRBench, c-CRAB. Sriramprabhu Rajendran: LLM Code Reviews Are Not What You Think. Sigma: homepage, reading a review.
Top comments (0)