The lie everyone in this industry is currently telling itself
Somewhere right now, an engineering leader is building a slide deck. It has a chart. The chart shows "AI-generated code" trending up and to the right. It will be used to justify next quarter's headcount plan.
That chart is built on a foundation of myths so widespread that a team of researchers — Jenna Butler, Margaret-Anne Storey, Travis Lowdermilk, Steven Clarke, and Emerson Murphy-Hill — just published a paper in ACM Queue specifically to kill them. It's called "Eight Myths on Software Engineering and GenAI," it hit the HN front page with 200+ points and 160+ comments in under a day, and it says the quiet part out loud: most of what you believe about AI and developer productivity is either unproven or flatly wrong.
I dug through the paper, the underlying studies, and the discourse it kicked off. Here's what survives contact with actual data.
Myth 1: "AI will dramatically speed up coding, therefore it speeds up engineering"
The math doesn't work, and it never did. Developers spend roughly 14% of their time writing code. The rest is design, meetings, code review, debugging, and coordination overhead.
Even if you doubled coding speed to infinity, you're capping your total productivity gain at under 15%. In practice, org-wide code throughput increases from AI tooling land around 7.8% — nowhere near the "10x" language vendors put in their pitch decks.
If your AI adoption strategy only targets the keystroke phase of software delivery, you're optimizing 14% of the job and calling it a transformation.
Myth 2: Lines of code is a productivity metric
It wasn't valid in 2014 when the first studies debunked it, and generating more code faster doesn't fix that.
"Measuring software productivity by lines of code is like measuring progress on an airplane by how much it weighs."
Yet LoC-generated-by-AI is exactly the metric showing up in engineering dashboards right now, because it's the easiest number to pull from a Copilot API. Easy to measure and meaningful are not the same thing. Optimizing for it just inflates diffs and review burden — which brings us to:
Myth 3: AI-generated code gets accepted like human code
Even inside orgs running internal AI coding agents at scale, only about half of AI-generated PRs get merged. Roughly 15% get abandoned outright. Another 15% sit stuck waiting on a human reviewer who doesn't trust it enough to fast-track it.
That's not a merge pipeline. That's a queue of work nobody wants to own.
Myth 4: AI benefits are consistent across tasks
This is the myth that should worry you most if you've been extrapolating from a demo. Results are wildly inconsistent depending on task type, and in at least one well-known controlled study — the METR trial — experienced open-source developers using AI tools were 19% slower, not faster, on real tasks in codebases they knew well.
The kicker: those same developers predicted they'd be 24% faster going in, and even after finishing slower, they still walked away believing AI had sped them up by ~20%. Self-reported productivity is not a metric. It's a vibe, and the vibe is currently lying to you.
Myth 5: Prompts are deterministic enough that context doesn't matter
Run two semantically identical prompts and you don't get semantically identical code:
Prompt A: "Write a function that validates a US phone number"
Prompt B: "Return true if the input string is a properly formatted
US phone number, false otherwise"
These read the same to a human. In practice, researchers found semantically equivalent prompts produced different code 46% of the time, and changed functional correctness in 28% of cases. Familiar tasks benefit more than unfamiliar ones. Your mileage varies not because you're prompting wrong, but because the tool is fundamentally non-deterministic in ways lines-of-code dashboards can't see.
Myth 6: Developers are terrified of being replaced
The doom narrative is mostly a media narrative. Only about 10% of developers report real concern about job displacement. Most see AI as freeing them up for architecture, mentorship, and the parts of the job that were never going to be automated anyway. If your team's morale problem is "AI is coming for us," that's a management story, not the median developer's actual position.
Myth 7: High tool usage means high trust
This is the gap that should terrify anyone shipping AI-generated code to production: 80%+ of developers use AI tools regularly, but only 29% trust the output's accuracy. Adoption and confidence have completely decoupled.
Worse, there's a documented "competence penalty" — code known to be AI-assisted gets evaluated more harshly by reviewers than identical code presented as human-written. You're not just fighting bugs. You're fighting a credibility tax on every PR with a Copilot commit trailer.
Myth 8: Enterprises can move at startup speed because they have GenAI now
No. Startups have small, recent, uniform codebases that look like the internet-scale open source data these models trained on. Enterprises have two decades of proprietary code, legacy systems the model has never seen, compliance requirements, and security review gates. GenAI doesn't dissolve institutional inertia — it just gives you a faster way to generate work for the humans still gating that inertia.
And the meta-myth underneath all of this: individual optimization drives productivity gains. It doesn't. Every real productivity revolution in engineering history — CI/CD, version control, the assembly line before that — came from systemic organizational redesign, not individuals getting better at using a tool in isolation. Right now, companies are spending millions on AI licenses and outsourcing the entire adoption strategy to "figure it out yourself" at the individual engineer level. That's not a strategy. That's hoping.
What to actually do with this
Stop tracking AI-generated LoC. Start tracking PR acceptance rate, review cycle time, and defect escape rate on AI-assisted code specifically — 41% of teams pushing heavy AI-generated code saw bug rates rise, and you won't catch that if your dashboard only measures volume.
Stop treating self-reported "I feel faster" surveys as data. Run the METR-style controlled comparison on your own team before you rewrite your roadmap around a feeling.
Stop pretending adoption equals trust. If 7 out of 10 of your engineers don't trust the tool's output, your rollout plan needs a review gate, not a mandate.
The technology is genuinely useful. The myths built around it are what's going to get engineering orgs to make expensive decisions they can't walk back. Read the data before you build the slide deck.
Sources: ACM Queue — Eight Myths on Software Engineering and GenAI · Hacker News discussion · GetDX summary · METR AI developer productivity study
Top comments (0)