I have been telling myself a story for the past year. The story goes like this: I use AI coding tools, I ship more code, therefore I am more productive. It feels true. It looks true when I check my commit history. Every metric I can see confirms it.
Then I read the METR study, and the story fell apart.
Sixteen experienced open-source developers completed 246 tasks on their own repositories, the codebases they know best, the projects they have contributed to for years. Half the time they used AI tools (primarily Cursor Pro with Claude). Half the time they worked without them. The result: developers using AI took 19 percent longer to complete their tasks.
Not faster. Slower.
But here is the part that got me. Before the study, the developers predicted AI would make them 24 percent faster. After experiencing the actual slowdown, they still believed they had been 20 percent faster. That is a 39-percentage-point gap between perception and reality. These are experienced developers working on their own code, and they could not accurately judge their own speed.
I have been telling myself the same story they were telling themselves. And I suspect you have too.
The Numbers Nobody Wants to Hear
The METR study is not an isolated finding. The data from multiple sources paints a consistent picture that is uncomfortable for anyone who has built their workflow around AI coding tools.
Google's DORA 2024 report surveyed 39,000 professionals. Seventy-five percent of developers said they felt more productive with AI. But every 25 percent increase in AI adoption correlated with a 1.5 percent decrease in delivery speed and a 7.2 percent drop in system stability. The developers felt faster. The systems got slower and less stable.
Faros AI analyzed telemetry from over 10,000 developers across 1,255 teams. Their findings add another layer to the paradox. Developers on high-AI-adoption teams complete 21 percent more tasks and merge 98 percent more pull requests. That sounds like a clear win. Except PR review time increased by 91 percent on those same teams. Pull request size ballooned by 154 percent. And bug rates went up 9 percent per developer.
The picture that emerges is not "AI makes developers unproductive." It is something more nuanced and harder to fix: AI makes developers produce more output while creating bottlenecks that absorb all the gains before they reach the organization.
The National Bureau of Economic Research put a number on the organizational impact. Across all occupations studied, AI adoption resulted in just a 3 percent time savings. No significant impact on earnings or hours worked. Three percent. For a technology that supposedly transforms how we work.
Why Fast Code Does Not Equal Fast Teams
The disconnect between individual output and organizational productivity is not mysterious once you look at where the work actually flows.
A developer using Claude Code or Cursor can generate a complete feature implementation in fifteen minutes that would have taken two to three hours by hand. That is a real, measurable speedup at the individual level. The code exists. The tests pass. The PR is opened.
Now what?
Someone has to review it. And this is where things break down.
AI-generated PRs are 154 percent larger on average. They touch more files. The code is well-structured but unfamiliar to the reviewer because nobody actually wrote it line by line. The reviewer cannot skim it the way they would skim a colleague's code that follows patterns they have discussed and agreed on. Every decision in the AI-generated code needs to be evaluated independently because the reviewer cannot assume shared intent.
This is why review time goes up 91 percent. Not because reviewers are slow, but because reviewing AI-generated code is fundamentally different from reviewing human-written code. You are not checking whether your colleague implemented the approach you discussed. You are evaluating whether an external system made reasonable choices across dozens of decision points you never discussed with it.
The bottleneck moves from code production to code review, and most teams have not adjusted for this. They have the same number of senior engineers reviewing PRs, the same review processes, the same expectations for turnaround time. But the volume and complexity of what needs review has roughly doubled.
This is the paradox in action. AI speeds up the fast part (writing code) and slows down the slow part (reviewing and validating code). Net result at the team level: roughly zero improvement, or sometimes worse.
The Perception Gap Is the Real Problem
What bothers me most about the METR study is not the 19 percent slowdown. It is that developers could not tell.
When I use AI tools, the experience feels productive. There is a constant sense of progress. Code appears on screen. Problems get solved. Things happen fast. The dopamine loop of "ask a question, get an answer, see code appear" is genuinely engaging.
But feeling productive and being productive are different things. The METR researchers identified several reasons why the subjective experience misleads:
Context switching overhead. When you delegate to an AI agent, you enter a review and correction cycle that is cognitively different from writing code. You read generated code, evaluate it, identify problems, explain corrections, review the regenerated output, and iterate. Each cycle involves switching between "writing" mode and "reviewing" mode. That switching has a real cognitive cost that developers do not account for.
The illusion of effortless output. When code appears without you typing it, it feels like it was free. But evaluating AI output, providing corrections, managing context, and dealing with the agent going in wrong directions all take time. You just do not track that time the way you track time spent actively typing.
Completion bias. AI helps you finish tasks. Finishing feels productive. But if the task takes longer to finish with AI than without it, the feeling of completion is misleading. You finished, but you finished slower.
The sunk cost trap with AI sessions. When an AI agent goes down the wrong path, developers often spend time trying to redirect it rather than starting over or doing the task manually. The time spent "fixing" the AI's approach can exceed the time it would have taken to write the code yourself. But once you have invested in the AI session, it feels wasteful to abandon it.
I recognized myself in every single one of these patterns. The question is not whether I use AI tools. I do, daily, and I am going to keep using them. The question is whether I am using them in ways that are actually faster, or in ways that just feel faster.
Where AI Actually Makes You Faster (and Where It Does Not)
The METR study and the Faros data are averages. Averages hide important variation. Not all tasks respond to AI the same way, and understanding the difference is where the real productivity gains live.
Tasks where AI genuinely speeds you up:
Greenfield code with clear requirements. When you are writing something new with a well-defined scope, AI tools produce real speedups. The Microsoft 2023 study showed developers completing tasks 55.8 percent faster with GitHub Copilot on self-contained problems. That finding is not wrong. It just does not apply to most of what experienced developers actually spend their time on.
Boilerplate and repetitive patterns. Tests, API endpoint scaffolding, type definitions, configuration files. Anything where the pattern is clear and the variation is minimal. This is where AI autocomplete and generation save genuine time with minimal review overhead.
Code you understand well in domains you know deeply. When you have enough expertise to review AI output quickly and catch errors immediately, the speedup is real. The review bottleneck shrinks because you can evaluate the code at a glance.
Tasks where AI slows you down:
Complex modifications to existing code. The METR study specifically tested experienced developers working on their own mature repositories. These codebases averaged over a million lines of code and ten years of history. The AI tools struggled with the interconnected, context-heavy nature of real-world systems. Understanding existing code well enough to modify it correctly requires deep context that AI tools, even with large context windows, do not fully capture.
Debugging production issues. Debugging requires building a mental model of what is happening at runtime, not just what the code says. AI can help search for patterns and suggest fixes, but the cognitive work of understanding the actual problem is not something you can delegate effectively.
Architecture and design decisions. Deciding how something should be structured is different from implementing a structure that has been decided. AI is good at the second part and mediocre at the first. When you use AI for architectural decisions, you often spend more time evaluating and rejecting bad suggestions than you would have spent thinking through the problem yourself.
Tasks requiring cross-system understanding. When a change touches authentication, database access, caching, and the API layer, the AI agent's limited context window means it makes locally reasonable decisions that are globally inconsistent. You end up doing cleanup work that would not have been necessary if you had written the code yourself with full system context in your head.
The Organizational Failure Mode
The paradox does not just affect individual developers. It creates a specific organizational failure mode that I see happening everywhere.
The pattern looks like this:
- A team adopts AI coding tools
- Individual developers produce more code and more pull requests
- Managers see increased output metrics and declare AI adoption a success
- The review bottleneck grows silently because nobody is measuring it
- PR merge times increase, but this gets attributed to "process issues" rather than AI volume
- Code quality subtly degrades because review quality drops under volume pressure
- Bug rates increase, creating more work downstream
- The team is measurably busier but not measurably more effective
Fortune reported that thousands of CEOs admitted AI had no impact on employment or productivity, even as 374 companies in the S&P 500 mentioned AI positively in earnings calls. The gap between the story companies tell about AI and the results they actually see is enormous.
This is what the Faros AI report means when it says there is "no significant correlation between AI adoption and improvements at the company level." The gains are real at the individual level. They just get absorbed by downstream bottlenecks before they reach the organization.
What I Changed After Reading the Research
I am not going to pretend I stopped using AI tools. That would be a stupid conclusion from data that shows AI tools are slower on specific task types. The right conclusion is to be deliberate about when and how I use them.
Here is what I actually changed:
I started timing myself. Not formally, but I set a rough timer when I start a task and note whether I use AI or not. After two weeks of this, I have a much better sense of where AI saves me time and where it does not. The results roughly match what the research predicts: new, self-contained work is faster with AI. Modifications to existing complex code are often faster without it.
I stopped using AI for tasks I can do in under ten minutes manually. The overhead of setting up context, reviewing output, and correcting mistakes means AI only wins on tasks above a certain complexity threshold. For quick fixes, small refactors, and changes where I already know exactly what to write, typing it myself is faster.
I restructured my review process. For AI-generated PRs, I now do what I described in my article on AI-generated technical debt: trace one complex path end-to-end rather than skimming the whole diff. This catches more issues and, counterintuitively, makes review faster because I am focused instead of scattered.
I batch AI-generated work. Instead of using AI for every task throughout the day, I batch the AI-appropriate tasks (scaffolding, boilerplate, test generation, well-defined feature implementation) into focused sessions. This reduces context switching and keeps the non-AI work (debugging, architecture, complex modifications) in a separate mental mode where I am more effective.
I invest more in context engineering. The METR study tested AI tools on complex, mature codebases where context is everything. The developers who use AI most effectively are the ones who invest in giving the AI better context, not just better prompts. My CLAUDE.md, scoped rules, and project documentation are not overhead. They are what make the difference between AI output that needs heavy revision and AI output that ships with minimal changes.
The Uncomfortable Truth About Speed
The deeper issue the paradox reveals is that our industry conflates speed with productivity. They are not the same thing.
Speed is how fast you produce output. Productivity is how effectively that output creates value. A developer who writes 500 lines of well-architected, thoroughly tested code per day is more productive than a developer who generates 5,000 lines of AI-generated code that creates review bottlenecks, introduces bugs, and requires rework.
The AI brain fry research showed that more AI tools do not equal more productivity. The productivity paradox research shows that more AI-generated code does not equal more productivity either. The pattern is consistent: the relationship between AI usage and actual outcomes is not linear. There is a sweet spot, and most developers are overshooting it.
The BCG study that coined "brain fry" found that workers using three or fewer AI tools reported genuine gains. Beyond four tools, productivity collapsed. I suspect a similar curve exists for how much of your coding you delegate to AI. Some delegation is clearly beneficial. Too much creates overhead that exceeds the time saved.
What the Research Predicts for the Next Year
METR is redesigning their study methodology because of an interesting problem: 30 to 50 percent of developers in their follow-up study refused to submit tasks they might have to complete without AI. The tools have become so embedded in workflows that developers resist working without them, even in a research context.
This suggests the paradox may be self-reinforcing. Developers become dependent on AI tools, lose calibration on how long tasks actually take without them, and then cannot accurately assess whether the tools are helping. The perception gap widens over time rather than narrowing.
But there is reason for cautious optimism. The METR team noted that based on conversations with participants, developers are likely getting more value from AI tools in early 2026 than they were in early 2025. The tools are improving. Context windows are larger. Agent capabilities are more reliable. The agentic coding workflow I described, where the agent plans and executes multi-step tasks autonomously, is genuinely more effective than the autocomplete-style assistance that was standard a year ago.
The paradox is not permanent. But it is real right now, and pretending it does not exist is worse than understanding it and adjusting.
The Honest Takeaway
I still use AI coding tools for hours every day. I am still more effective with them than without them for the right tasks. But I stopped assuming that every task is the right task.
The productivity paradox is not about whether AI tools work. They do. It is about whether you are measuring the right things and using the tools where they actually help versus where they just feel like they help.
The developers who will get the most from AI in the next year are not the ones who use it for everything. They are the ones who know when to use it, when to type the code themselves, and how to structure their workflow so that AI-generated output does not create more problems downstream than it solves.
That requires honest self-assessment, and the research suggests that honest self-assessment is exactly the thing AI tools make harder. The feeling of productivity is so strong that it overrides the evidence.
Track your time. Measure your rework rate. Pay attention to how long your PRs sit in review. The numbers will tell you what the feeling will not.
Top comments (0)