DEV Community

Tess Ainsley
Tess Ainsley

Posted on

Cutting PR review time is an orchestration problem, not a reviewer problem

The usual answer to "how do we reduce PR review time" is to make the reviewer faster. Vendor pages recommend an AI reviewer that reads a diff in seconds and posts comments, so the bottleneck moves. A new field study from a real industrial repository says the lever is somewhere else entirely: how you slice the change into commits, batches, and CI jobs.

The preprint arXiv:2609.29172, "Orchestrating AI-Assisted Code Remediation: Socio-Technical Bottlenecks in a Large Industrial Repository" by Andreas Bexell, Lo Gullstrand Heander, and Emma Söderberg (24 Sep 2026), reports a 15-day single-case study in a closed-source industrial C++ codebase. An experienced developer used a command-line AI coding buddy to remediate widespread issues across the repo. When source editing got cheap, the developer generated hundreds of commits touching thousands of lines. The result is the part most tool advertising skips: the AI did not slow down. CI and the reviewers did.

The bottleneck moves downstream of generation

It is easy to read this as only a CI story, and the paper does report a concrete CI failure. The naive approach, one commit per file, overloaded build-on-commit CI because every commit triggered a build. Switching to directory-based batching and capping the number of files per change restored throughput.

But the authors are explicit that CI was not the only saturation point. Reviewer attention saturated too. The developer had to explicitly solicit reviews, negotiate what commit granularity was acceptable to the team, and run iterative follow-up to resolve build and static-analysis failures that the first pass missed. The paper calls CI capacity, review effort, and change orchestration the primary bottlenecks once mechanical editing is cheap.

That matches what GitHub documents for Copilot code review. GitHub's using-Copilot-code-review documentation explains that Copilot runs its review through GitHub Actions, that a review "usually takes less than 30 seconds," and that reviewers can pick a Lite or Balanced effort level. Generation and even model review are fast. The wall clock is elsewhere.

Review time is not the metric vendors advertise

Most articles ranking on this query measure "review time" as the flat latency of a request, which an AI reviewer makes nearly instant, and call that progress. The study suggests the cost that grew was not the reviewer reading but the orchestration around it: commit batching decisions, CI job counts, build and static-analysis failures, negotiation about granularity, and re-review loops after follow-up fixes.

This is the same shape as the measured numbers already in the field notes. Review time under AI-assisted development went up 441% in the telemetry surveyed by the vibe-coding review, and Salesforce found that review time on its largest pull requests plateaued precisely because reviewers had stopped deeply engaging. When a team reports "review got faster," the question is which interval got faster. If it is only the automated comment step, the expensive part is still there.

Part of the confusion is that "review time" gets used to mean two different things. There is the human minutes a reviewer spends reading and commenting, and there is the wall-clock cycle time from open to merge. A fast AI reviewer makes the first nearly zero and barely touches the second, because most cycle time is waiting, batching, CI runs, and follow-up. That is why a flat "review time" number from a vendor page tells you so little.

What actually reduced the bottleneck

Two mechanisms in the study directly cut the load, and both are about batching rather than reading speed.

First, directory-based batching with a cap on files per change restored CI throughput. This is not a review decision; it is a pipeline design decision. It changes how many builds a given volume of change triggers. The authors observed that naive per-file commits turned CI into a backlog machine, where the queue length and rerun cost dominated elapsed time regardless of how fast any single review finished.

Second, treating a semantic change set as a first-class unit of work that can be sliced differently for the developer, the reviewer, and CI. The study's framing is that "fix all instances of warning X" is a meaningful unit, and the developer should not be forced to ship it as either one massive commit or hundreds of per-file commits. Different consumers want different slices: CI wants bounded jobs, the reviewer wants a coherent change, and the developer wants to verify progressively. Allowing those slices to diverge is what lets a large remediation move without stalling either the pipeline or the humans.

This is a pipeline and process problem, which is why a reviewer tool alone does not answer the question. A reviewer that posts comments in seconds does nothing about the per-file commit flood that saturates CI, or the backlog of build failures that need follow-up. The tools that help here are the ones that let a team control change granularity at review time, from diff grouping on the review side to commit and batch limits. Reviewer tools that run inside the existing pull request flow, which includes Kodus and the batched review controls in Copilot, live in this space. But the study's point is that no reviewer fixes the commit granularity decision the team has to make upstream.

What a team should change first

The transferable lesson from this 15-day study is to check where the latency actually sits before buying another review tool. If a surge of AI-generated commits is hitting CI and reviewer attention, the first lever is commit and batch granularity: cap files per change, batch by directory or semantic intent, and give CI bounded jobs so it stops being the traffic jam. Then negotiate what a reviewable unit looks like with the team before the volume arrives, not after.

There is a sequence worth following rather than bolting on tools. Start by instrumenting the pipeline so you can see where time actually goes, whether it is CI queue length, review wait, or follow-up loops. Then cap the batch size so a volume of change maps to a bounded number of builds and a bounded number of diffs a human must read. Only after the granularity is under control does adding an AI reviewer to close the loop on comments make sense, because it is now operating on a change a human can actually absorb.

A second study, also posted 24 Sep 2026, suggests why the pipeline matters so much. In Between the Commits: Process, Error, and Claim Reliability in a Wholly AI-Authored Codebase, Douglas Leith traces a 21,000-line Python tool built entirely by Claude and finds 14.3% of AI code-generation events contained a real error that was later caught by the AI-authored test suite. That means automated detection running in the pipeline will catch a meaningful share of defects before a human ever looks, which is exactly the load that batching and CI capacity decide whether gets absorbed.

So the honest answer to reducing PR review time under AI load is not "get a faster reviewer." It is to make the change arrive in slices the pipeline and the human can actually process. The preprint read as of 27 Sep 2026 supports that; it is a case study of one repository, so treat the specific numbers as directional. The mechanism, cheap generation pushes the bottleneck to orchestration, is consistent with the larger survey evidence.

Top comments (0)