DEV Community

Cover image for AI Agents Write 30% More Code. Shipped Software Did Not Move. Code Review Is Why
jamilxt
jamilxt

Posted on

AI Agents Write 30% More Code. Shipped Software Did Not Move. Code Review Is Why

#ai #codereview #softwareengineering #productivity

Two of the most important software engineering studies of 2026 landed within days of each other, and they point at the same uncomfortable conclusion.

A Harvard working paper analyzed roughly 300 million work events across more than 700 software firms. After teams adopted AI coding agents, the code each developer produced rose about 30 percent. The amount of finished software those firms shipped did not move in any statistically significant way.

A Bain survey of nearly 300 senior technology leaders found the same pattern from the other direction. Developers complete roughly 21 percent more tasks. Time spent reviewing code rose about 91 percent.

The typing got faster. The shipping did not. And the reason sits in one place: code review.

What the Harvard study actually found

The paper, "Artificial Intelligence in the Firm: Bottlenecks in Software Production" by Harvard economists Fiona Chen and James Stratton, paired engineering telemetry from the Jellyfish analytics platform with GitHub activity across 718 firms and more than 725,000 workers, covering January 2021 through March 2026. It is the first large-scale empirical look at what happens to a company, not an individual developer, when coding agents arrive.

The numbers, as reported by Ars Technica and other outlets covering the paper:

  • Lines of code per worker rose about 30 percent, roughly 1,494 additional lines per worker-month against a baseline of about 5,017.
  • Commits rose about 20 percent and pull requests about 23 percent.
  • The resolution rate of tracked issues and completed epics, the study's proxy for shipped work, showed no statistically significant increase.
  • Time from pull request submission to merge rose 49 percent, about 3.45 days on a baseline of 7.03 days.
  • The share of pull requests sent back for changes nearly doubled.
  • Review comments per pull request rose 35 percent.
  • The share of workers assigned to code review rose 14 percent.

That last group of numbers is the mechanism. Agents generate code faster than the organization can absorb it, and the surplus piles up in the review queue. The authors put it plainly: "code review becomes the bottleneck." Their prescription is equally blunt. Realizing the benefits of AI adoption requires complementary investments in downstream review capacity, not more code generation.

One detail deserves its own sentence. By March 2026, 80 percent of firms in the dataset were running AI code review tools. Yet only 23 percent of review comments were AI-generated. Teams bought the tools. The tools did not remove the human constraint.

The Bain survey says the same thing from the manager's side

Bain's 2026 Global Technology Report, released September 29, surveyed nearly 300 senior technology leaders. The findings line up with the Harvard data almost number for number:

  • Developers complete roughly 21 percent more tasks with AI tools.
  • Time spent reviewing AI output rose about 91 percent.
  • Developers now juggle 47 percent more simultaneous workstreams.
  • Leaders expect 148 percent faster release cycles and 95 percent higher developer productivity within one to two years, while organizations capture only 20 to 27 percent today.

Purna Doddapaneni, a Bain partner and one of the report's authors, told CIO Dive: "The bottleneck has moved from writing code to trusting it. Every change still has to be understood, reviewed, tested, and secured, and most of that work still runs through people."

That gap between expected and captured gains is the whole story in one line. The reports are measuring the same thing Harvard measured, which is that output rises at the individual level and settles at the speed of the slowest human checkpoint at the organizational level.

Even the fix-for-the-fix is struggling

If review is the bottleneck and AI review tools are the obvious answer, the first leaderboard for that category should show a solved problem. It shows the opposite.

On October 5, GitHub launched ReviewBench, an open benchmark for code review agents, built from 219 real pull requests across 187 public open source repositories in 19 languages, modeled on the distribution of 103.9 million real GitHub pull requests. GitHub Copilot's code review agent leads the inaugural leaderboard with a grounded F1 score of 40.1 percent.

Read that number again. The best-scoring agent on a benchmark designed and hosted by its own vendor catches less than half of what the ground truth set says matters, at a precision-recall balance the benchmark rewards. Meanwhile Faros AI's 2026 telemetry, covering 22,000 developers across more than 4,000 teams, found that high AI adoption came with average pull request size up 51 percent and bugs per developer up 54 percent.

So the loop currently looks like this: agents write bigger pull requests, bigger pull requests take longer to review, AI reviewers catch less than half of the issues, and the extra review load lands on humans. Amazon's internal response, cited in the Bain report, is a mandate to push up to 70 percent of code reviews to AI. That may work at Amazon's scale with Amazon's investment. For most teams, the honest reading of the data is that AI review is a filter, not a replacement.

Why review, specifically, breaks first

It is worth being precise about why the constraint moved here and not somewhere else, because the answer tells you where to spend effort.

  • Review is serial and human-gated. Generation scales with compute. Review scales with the number of people who can say "yes, this is correct." Adding compute to the first does nothing for the second.
  • Review requires the most context. An agent can write a plausible function from a prompt. Judging whether that function belongs in the codebase, whether it duplicates something, whether it breaks an invariant documented nowhere but in a maintainer's head, requires context that does not fit in a prompt.
  • Trust does not parallelize. A team can merge five AI-generated branches, but if nobody deeply understood any of them, the team has traded development speed for operational risk. The Harvard revision numbers, nearly doubled change requests, show reviewers rejecting work at a much higher rate. Volume without trust is just queue.
  • Bigger diffs break reviewer attention. Research on code review has shown for years that defect detection drops sharply as changesets grow. Faros measured average PR size up 51 to 154 percent depending on the dataset. The input quality to the review process got worse at exactly the moment volume went up.

A checklist for teams drowning in review

If your team adopted coding agents and your pull requests are aging, this is the checklist I would work through, in order. Each item maps to something the data says is actually broken.

  • Cap generated pull request size. Set a hard norm, around 400 changed lines, and require agents to split work across stacked PRs. The single highest-leverage change, because it restores the diff size where human review actually catches defects.
  • Budget review capacity like a resource, not a favor. If agents raise output 30 percent, roughly 14 percent more of the team's time must move to review, per the Harvard data. That time has to come from somewhere explicit. If it comes from nowhere, the queue absorbs it and merge times stretch.
  • Tier your review depth. Not every change deserves equal scrutiny. Auto-generated boilerplate, test scaffolding, and docs can pass a light gate. Changes to auth, payments, data migration, and concurrency get senior eyes regardless of who or what wrote them.
  • Use AI review as a pre-filter, never as the gate. The ReviewBench 40 percent F1 number is your expectation setting. Let AI reviewers strip out the mechanical comments, style, dead code, obvious bugs, so humans spend attention on design and correctness. Do not let a green AI review stand in for a human approval on anything risky.
  • Measure merge time and change-request rate, not commit count. If your dashboards celebrate commits and lines, they are rewarding the exact behavior that broke your pipeline. Merge time per PR and revision rate tell you whether review is keeping up.
  • Prompt agents with your review standards. Most revision cycles come from agents not knowing house rules. Encoding your conventions, error-handling patterns, and forbidden patterns into the agent's instructions attacks the near-doubled revision rate at the source.

The honest takeaway

None of these studies say AI coding agents are useless. They say the industry bought the first half of the pipeline, generation, and is now paying for having ignored the second half, verification. Writing code was never the only constraint; it was just the constraint that a chat box could relax.

The teams capturing real gains in this data are not the ones generating the most code. They are the ones who reinvested in review capacity, kept their diffs small, and treated AI reviewers as a first pass rather than a verdict. Bain's own numbers show organizations currently capture a fifth to a quarter of the gains their leaders expect. The difference lives almost entirely downstream of the code editor.

The next real productivity jump in software will not come from a model that writes faster. It will come from making trust scale: better review tooling, smaller changes, and engineers whose job shifts from producing code to certifying it. The reports agree on that. Now the process has to catch up.

Sources and further reading:

I write about backend engineering, developer tools, and practical AI adoption every week. Subscribe, it's free.

Teams are clearly splitting on this: some have put AI review in front of every PR, others refuse it outright. Which side is your team on, and what did it do to your merge times? Tell me in the comments.

Top comments (0)