It feels like the industry’s new mantra isn't building software anymore, but manufacturing "application factories." As engineers, we have to ask ourselves: are we actually optimizing delivery, or are we just playing with a new tool? (Because let's be honest, raise your hand if you’re guilty of this. I’ll be the first.)
Over the past few weeks, I’ve seen numerous posts discussing the paper "Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity." The study presents a striking paradox: while developers perceived a 20% boost in their performance, the empirical data revealed they actually took 19% longer to resolve issues when using AI.
It’s worth highlighting (as the title of the study suggests) that this study focused exclusively on senior engineers. The nearly 40-point gap between perception and reality might simply come down to intuition and deep experience. A seasoned expert often knows which variable to tweak in under a minute. Working with AI, by contrast, requires opening the tool, crafting prompts, waiting for the MCP to index files, and letting it process the context. This leads to the surprising finding that senior developers took 19% longer to close issues when AI tools were part of their workflow.
Naturally, these findings aren't conclusive, the sample size was limited to just 16 developers, which is quite small. Still, it sparks a necessary conversation:
- Does this productivity gain actually exist for staff-level and principal engineers, or is AI far more effective for mid-level developers?
- How do you explain this massive gap between perceived efficiency and actual output?
- How much weight do you give to studies like this in your day-to-day workflow?
I’d love to hear your take in the comments.
Top comments (0)