A CTO gave the engineering team a target: double productivity with AI.
The team hit it.
Merged pull requests per developer per month increased from 21.2 to 44.3. That is approximately 2.09 times the original output.
But there was another number to watch.
The review workload per reviewer roughly doubled, too.
The team was generating more code. Reviewers now had more code to understand, evaluate, and approve. The organisation had increased the output of one part of the system without automatically increasing the capacity of the part that followed it.
That is the productivity paradox at the centre of this chapter: individual velocity gains can be real and substantial while organisational delivery remains constrained by the work that moves downstream.
- Five studies. Five different measurements. The debate about AI coding productivity often looks like a fight between studies. One study reports a substantial speed-up. Another finds a smaller gain. Another finds that experienced developers take longer with AI. The chapter brings five important findings together: • +55.8%: Peng and colleagues measured faster completion on a greenfield JavaScript HTTP-server task. • +26.08%: Cui and colleagues reported an increase in completed tasks across 4,867 developers. • +21%: Google's enterprise trial reported faster task completion, although the result lost statistical significance after controls. • Approximately +40%: Heilman and colleagues found more pull requests during the heaviest-use weeks among 16,223 Microsoft engineers. • −19%: METR's randomised controlled trial found that 16 experienced developers took longer on 246 tasks in repositories they already knew. These findings do not need to be treated as a simple contest in which one study must defeat another. They measure different kinds of work, different populations, and different conditions. The METR study, in particular, examined experienced developers working in familiar repositories—a setting closer to the day-to-day work of senior engineers than a bounded greenfield task. The effect changes as the work changes. The chapter's argument is not that one of these findings must be discarded. It is that task selection matters when interpreting the measured effect. Figure 3.1 belongs here. Caption: No study here is wrong. The effect fades as the work gets closer to what senior engineers really do.
- Three instruments. Three answers. There is another reason productivity measurements can appear contradictory: asking people what they expect, measuring how long they take, and observing what they choose to do are three different instruments. The chapter distinguishes between three kinds of evidence. Stated preference captures what people believe or report. In the METR study, developers forecast that AI would make them 24% faster. After the work, they still estimated that it had made them about 20% faster. Measured outcome captures what happened on the tasks being evaluated. In the same METR experiment, the measured result was the opposite: developers took 19% longer. Revealed preference looks at what people choose when there is a real cost attached. At $50 per hour, roughly 30–50% of developers refused to submit tasks they were unwilling to do without AI. These instruments answer different questions. What people expect, what the clock records, and what people choose under a financial trade-off should not be collapsed into one number. For an organisation, the important question extends beyond whether an individual finishes a task faster. Does the work move through the entire delivery system more effectively?
- The mandate was met. What happened next? Return to the CTO's target. The designated metric was merged pull requests per engineer per month. The reported average rose from 21.2 to 44.3. The chapter's enterprise telemetry covers 802 developers, 196,212 non-bot pull requests, and 364 repositories between January 2024 and April 2026. The increase was accompanied by substantial changes elsewhere: • Company-wide pull-request volume increased 3.1 times. • Token spending rose roughly 20 times. • AI-authored pull requests reached approximately 90%. • Among 246 developers present throughout the period, throughput increased 2.3 times. • Review workload per reviewer roughly doubled. The work did not simply disappear. More code was generated, and more work arrived at the review boundary. The size of the work items changed as well. Average pull-request size rose 37% after adoption and a further 33% after the mandate. Review time increased by 12% on adoption, while review comments increased by 41% after the mandate. These figures matter because a pull request is not just a unit of output. It is also a unit of work for the person reviewing it. When pull requests become larger, reviewers must absorb more change in each item. Counting more merged pull requests can therefore tell only part of the story. The organisation met its chosen metric. But the metric did not, by itself, establish that the whole delivery system had become twice as capable.
- The review queue is where the maths changes. Code generation is elastic: adding agents can increase how much code is produced. Human review is different. Reviewers have finite time and attention. Queueing theory describes this using an arrival rate, λ, and a service capacity, μ. As incoming work approaches the capacity available to process it, waiting time rises nonlinearly. In the simple queue model used in the chapter, the time in the system is: W = 1 / (μ − λ) As the arrival rate approaches service capacity, the gap between the two becomes smaller. Waiting time can increase sharply. If arrivals exceed capacity, the system cannot sustainably process the work without something changing. This is why doubling code generation does not necessarily double delivery. If review capacity remains fixed, more generated work can mean a longer queue rather than proportionally faster delivery. When the review system comes under pressure, the chapter identifies four possible responses—or valves. Valve 1: Let the queue grow. More work waits, increasing lead time. Valve 2: Review faster by reducing scrutiny. Throughput may improve, but less scrutiny changes the quality of the review. Valve 3: Increase batch size. Fewer, larger items can reduce the number of arrivals, but larger pull requests become harder to review well. Valve 4: Change the server. Mechanise verification and increase review capacity, including through automated review. The first three responses trade one cost for another. The fourth is different because it adds capacity to the system. Figure 3.4 belongs here. Caption: Three valves trade one cost for another. Only the fourth raises capacity. The enterprise evidence shows changes in both batch size and review automation. Automated review overtook human review; bots accounted for approximately 38% of review events and roughly half of review comments. The chapter's point is not that automation makes every review problem disappear. It is that a saturated review system needs a deliberate response—and adding capacity is different from simply asking the generation side to produce more.
- “Merge and revert rates held steady” needs a time window. The enterprise study reports that merge and revert rates did not move. That is meaningful evidence against the claim that AI-generated code necessarily causes an immediate, measurable quality penalty. The chapter takes that finding seriously. But the study's own description matters: the measures are “coarse, short-horizon quality proxies.” A stable short-term measure is not the same as proof that no longer-term effects exist. The chapter contrasts this result with observational research on Cursor adoption in public repositories, which reports substantial, persistent increases in complexity and static-analysis warnings alongside velocity gains. The evidence does not settle every question about software quality. Different study designs and measurement windows support different conclusions. The practical lesson is to report the measurement horizon alongside the metric. A claim about quality needs to say what was measured and for how long.
- The metric was met. Was the constraint addressed? The organisation's chosen metric—merged pull requests per engineer—rose substantially. That gain should not be dismissed as imaginary. But average pull-request size also increased. A pull request in April 2026 was not necessarily the same-sized unit of work as a pull request in April 2025. The chapter connects this to Goodhart's law: when a measure becomes a target, it can stop being a good measure of the underlying goal. Here, the argument is not that developers necessarily manipulated the metric. The target was met, but the growing size of the work items made reviewability harder. The chapter's conclusion is that a generation mandate raises the arrival rate of work. It does not automatically raise the capacity available to review that work. The bottleneck moved. The productivity problem did not disappear. Try this Compare how long pull requests wait for their first human review this month against three months ago. Look at the waiting time alongside pull-request size and review time. More output is useful, but the time work spends waiting at the next stage matters too. Read the full chapter This post summarises the argument. The full chapter contains the research evidence, measurement details, queueing explanation, and qualifications behind it. Read Chapter 3: The Productivity Paradox Next week: what shows up later—and why short-term productivity and quality measurements may not tell the whole story. Has your review queue grown since AI agents arrived? Share what you have observed in the comments.
Top comments (1)
Merged PRs per eng is a generation metric, not a delivery one.
Watch time-to-first-human-review and average PR size week over week—if both climb, you just moved the pile.
Adding more agents without review capacity is valve 1: longer queue.