Ask a senior engineer whether they could do their job without AI and you'll often get a contradiction that shouldn't be possible: "I'd be slower… but honestly, happier."
Slower and happier. Those don't go together — unless the thing speeding you up is also quietly costing you something the speed doesn't show. That one sentence is the entire AI debate compressed, and once you pull it apart with actual studies, a clean line appears. Everything that follows is on one side of it or the other.
Here's the line, then five findings with the methodology attached, then the single mechanism underneath all of them.
There are two AIs, and we keep confusing them
Not two models — two situations. The same LLM behaves like a different product depending on who is holding it and whether they can check its work.
| The AI you're forced to use | The AI someone chose | |
|---|---|---|
| Who selects it | A search engine, an employer, a PM | A domain expert who already knows the answer's shape |
| Typical job | Search summary, chatbot, "write my email" | Crunch 100k log lines, triage a dataset, flag edge cases |
| Can the receiver verify it? | No — you asked because you didn't know | Yes — the expert catches the mistakes |
| Cost of a wrong answer | You act on it and lose | Caught in review; no harm |
| Who absorbs the error | The least-equipped person in the chain | The most-equipped person in the chain |
| Net result | Usually a hidden tax | Genuinely useful |
Every "AI changed my work" story lives in the right column. Every "AI wasted my afternoon" story lives in the left. Same technology, opposite outcome — and the only variable that actually moved is whether the person receiving the output can tell when it's lying.
Now the receipts, each with the study design attached, because "a study found" is exactly the kind of ungrounded claim this article is about.
Fact 1 — "Faster" is, for experienced people, measurably backwards
The study: In 2025, METR ran the test everyone assumed had already been done — a randomized controlled trial of AI's effect on real work, not a survey. 16 experienced open-source developers. 246 genuine tasks drawn from their own large repositories (repos they knew cold, averaging 1M+ lines, 22k+ stars). Each task was randomly assigned "AI allowed" or "AI forbidden." Most used Cursor Pro with Claude 3.5/3.7 Sonnet — current, capable tooling.
The headline:
Developer speed WITH AI tools — METR 2025 RCT
Predicted beforehand +24% ▲ faster (what they expected)
Felt afterward +20% ▲ faster (what they believed even after)
Actually measured −19% ▼ SLOWER (what the clock said)
They were 19% slower with AI — while remaining convinced it had sped them up by 20%. Think about the size of that gap: a 39-point swing between perception and reality, after they'd finished the work.
The micro-breakdown — where the time actually went (this is the part the headlines skip):
| Activity | What happened with AI allowed |
|---|---|
| Writing code by hand | ↓ went down |
| Searching / reading docs | ↓ went down |
| Prompting the model | ↑ new time sink |
| Waiting on generations | ↑ new time sink |
| Reviewing + cleaning output | ↑ ~9% of total task time |
| Idle | ↑ went up |
And the number that explains the whole thing: developers accepted fewer than 44% of the AI-generated code. More than half of what the model produced was thrown away or heavily rewritten. The work didn't disappear — it moved, from writing (which an expert is fast at) to reviewing someone else's plausible-but-maybe-wrong code (which is slow and draining).
The honest caveat: this is 16 experts on code they know intimately. 4 of the 16 did speed up. Beginners, unfamiliar codebases, and greenfield work may genuinely gain — which is not a loophole, it's the thesis: the benefit tracks who's holding it and whether they can verify the output cheaply.
Fact 2 — The AI most people actually touch is the one they never chose, and it's wrong a lot
For the average person, "AI" isn't a chatbot they opened on purpose. It's the summary that appears above the search results whether they asked for it or not. That is the left column, at planetary scale.
The measurements:
- An analysis using the Oumi / New York Times methodology found the model behind those overviews (Gemini 3) returns incorrect information on ~28% of queries on its own; it improves when fused with live search.
- Even when the final overview looks correct, the analysis found 56% of those accurate-looking answers were "ungrounded" — they cited sources that didn't actually support the claim being made.
- Of the 5,380 sources the summaries leaned on, the second- and fourth-most-cited were Facebook and Reddit. Authoritative-sounding answers, substantially assembled from forum posts.
- Critics extrapolate that even a 9% inaccuracy rate across ~50% of searches implies on the order of hundreds of millions of misleading summaries per day (an estimate — Google disputes the methodology; its own data shows Gemini 3 wrong on ~28% of queries before search grounding).
Then the part that makes error dangerous rather than annoying — Pew Research:
How much of an AI search summary gets fact-checked by the reader
██ 1% clicked a source link
░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░ 99% took it as-is
When users encounter one of these overviews, they click a source link 1% of the time. The summary is the answer for 99% of people. A left-column error isn't a wrong draft an expert will catch — it's a confident wrong answer delivered to the exact person who asked because they didn't already know. That's the person who drains the wrong oil on a two-stroke engine, misses a legal deadline, or follows a config that doesn't exist — and finds out only after it's too late to matter.
Fact 3 — When "chosen" AI leaks into someone else's "forced" column, it has a price tag
Here's the twist most AI-productivity debates miss: the two columns aren't fixed. Your chosen AI becomes your coworker's forced AI the moment you paste its output into a doc and hit send.
Researchers named this. A September 2025 Harvard Business Review study by BetterUp Labs and Stanford's Social Media Lab (survey of 1,150 full-time U.S. desk workers, Aug–Sept 2025) coined the term "workslop": AI-generated work that looks polished but lacks the substance to actually move a task forward, dumping the real thinking onto the recipient.
The findings:
| Metric | Figure |
|---|---|
| Workers who received workslop in the prior month | ~40% |
| Average time to untangle each incident | 1 hour 56 minutes |
| Estimated invisible cost per employee | ~$186 / month |
| Scaled to a 10,000-person org | ~$9M / year in lost productivity |
| Recipients who trusted the sender less afterward | ~half saw them as less capable/reliable |
More than half of recipients reported being annoyed; 22% were offended. (Caveat: cost and time figures are self-reported, and BetterUp sells coaching — treat the $9M as the vendor's framing, not gospel.) But the direction is the point: the productivity AI "saved" the sender reappears as a larger cost on the receiver's desk, plus a trust hit. It didn't vanish. It got transferred — and taxed interest.
That's the left/right columns colliding: an expert used the tool, skipped the verification step that makes it safe, and shipped the unverified output to someone who now has to do the checking and the fixing.
Fact 4 — The "slop feeding on slop" fear has a name and a Nature paper
As the open web fills with generated text, models increasingly train on the output of earlier models. Shumailov et al., "AI models collapse when trained on recursively generated data" (Nature, July 2024) tested exactly that failure mode.
The method: they fine-tuned a language model on a Wikipedia-derived dataset, then fine-tuned the next model on the previous model's own output — repeated across nine generations.
The slop loop (model collapse)
Humans post AI text → next model trains on it → rare/correct details fade
↑ │
└────────────── output degrades into confident nonsense ┘
The result: each generation, the tails of the distribution — the rare, specific, correct details — quietly disappeared first, until the output degraded into fluent nonsense. They proved it for LLMs, variational autoencoders, and Gaussian mixture models (so it's a property of the recursive-training setup, not one model's quirk). Mixing ~10% real human data into each generation slowed the rot; it did not stop it.
"Garbage in, garbage out" stops being a boomer joke and becomes the training-data crisis the labs are now spending real money to avoid — and it's the same mechanism as workslop and ungrounded AI Overviews, one level up: a system that consumes its own confident output eventually can't tell signal from noise.
Fact 5 — The wariness is the median, and there's a measured cost per prompt too
If you suspect the skeptics are a loud minority — Pew Research, June 2025, 5,023 U.S. adults:
| How AI in daily life makes Americans feel | Share |
|---|---|
| More concerned than excited | 50% |
| Equally concerned and excited | 38% |
| More excited than concerned | 10% |
57% rate AI's societal risks as high; only 25% rate its benefits as high. That's not a bubble — it's the center of the distribution.
And a measured aside the hype usually skips: every one of those forced, often-unread, 99%-unchecked summaries runs real hardware. Credible estimates land around ~0.3 Wh of electricity per average query (Epoch AI; OpenAI's own figure is similar) and roughly ~519 ml of water per 100-word response (UC Riverside, via The Washington Post). The per-query numbers are small and genuinely contested — but multiply by every search getting an overview nobody asked for and the left column has a utility bill, not just an accuracy problem.
The mechanism: why the line exists at all
Five findings, one underlying cause. It's worth naming, because it predicts new cases, not just explains old ones.
1. The verification tax. An LLM produces output that is plausible but not guaranteed. Someone has to close that gap by checking it. If the checker is an expert on a bounded task (right column), checking is cheap and the tool is a net win. If the checker can't evaluate it — or doesn't bother (99% don't) — the gap doesn't get closed; it gets shipped. METR's 9% review time and <44% acceptance rate is the verification tax, visible on a stopwatch.
2. Removal cost is asymmetric, and so is value. The tool's value is highest exactly where the user least needs it (an expert who could do the job anyway, verifying fast) and lowest where the user most needs it (a non-expert who can't verify at all). We deployed it by default to the second group — the search bar, the forced copilot, the chatbot that replaced the support line — which is precisely backwards from where it pays off.
3. Skill atrophy closes the escape hatch. The whole system assumes an expert is standing by to catch mistakes. But if juniors learn on AI output instead of on the problem, the supply of people who can catch mistakes shrinks — right as the volume of plausible-wrong output explodes. Today the senior catches it; the open question is who catches it in ten years.
That's the engine. Wherever output is plausible-but-unverified and the receiver can't cheaply check it, you get a tax. Wherever an expert verifies a bounded task on a dataset too big to read by hand, you get a genuine win.
Where the right column genuinely wins
To be clear this isn't "AI bad" — the real wins are specific and they all share a shape:
- Log and data triage. Point it at 100k lines of logs or a messy ERP export; it surfaces the standouts in minutes. The human still verifies; the machine just does the reading nobody wants to.
- Large-dataset pattern-finding. Triaging huge cancer-cell or bloodwork datasets to narrow options — hours instead of months — with an expert reading the shortlist.
- Static analysis and security. Genuinely strong at studying a fixed artifact (a codebase) and flagging edge cases a human would miss.
Every win: a domain expert, a bounded and checkable task, a dataset too big to read by hand. The AI isn't trusted — it's used, by someone who can catch it the instant it's wrong.
Which gives the honest one-liner:
AI isn't good or bad. It's a power tool that's safe in the hands of someone who could do the job without it — and dangerous the instant it's handed to someone who can't tell when it's lying.
The test you can actually use
Before you — or your company — drop AI into a workflow, ask one question:
"Can the person receiving this output tell when it's wrong?"
- Yes → right column. Ship it. (Log-crunching, data triage, a first draft an expert will edit.)
- No → you've built a confident-wrong-answer machine aimed at the one person who can't catch it. That's the forced-search-summary trap, the workslop trap, the junior-shipping-code-they-can't-review trap.
Nearly every argument about whether AI is "overhyped" or "revolutionary" collapses into that line. People are usually just standing in different columns, describing different products, using the same word.
Your turn
The question I actually want answered in the comments:
What's the one task where AI genuinely made you faster — and be honest: could you catch it when it got that task wrong? If the answer is "no," I'd love to hear how you found out it was wrong, and what it cost.
I write about where AI actually earns its keep versus where it's just expensive confidence — the measured version, not the hype version. If that's your thing, follow me here.
Sources: METR 2025 RCT — methodology & time breakdown (The Register) · METR slowdown detail (TechRadar) · Google AI Overviews accuracy analysis (TechRepublic) · AI Overviews ungrounded / Reddit-cited (Shopifreaks) · "Workslop" HBR / BetterUp / Stanford (Axios) · Model collapse, Nature 2024 (Tech Xplore) · Pew Research — how Americans view AI, Sept 2025 · Per-query energy/water estimates (Epoch AI; UC Riverside)
Top comments (2)
The "verification tax" framing lines up with what makes test-fixture generation so different from feature work.
When an LLM generates adversarial test inputs, such as malformed binary headers, truncated streams, or edge-case form dictionaries, you do not have to read every byte to trust it. Your parser, schema validator, and regression suite act as a cheap deterministic oracle that passes or fails in milliseconds.
Writing stateful application code flips that cost. Reading someone else's plausible-looking control flow to verify subtle boundary conditions takes longer than writing it from scratch, which explains why acceptance stays under 44% even when developers feel faster.
This is the sharper version of the whole argument, and I think it rewrites my own framing. I said "expert + bounded task," but the real variable is whether a cheap, sound oracle exists. Expertise only matters because it's usually what stands in for the oracle — when a real one exists, you barely need the expert to read anything.
It cleanly predicts the whole right column:
The <44% acceptance falls straight out of the bottom rows: no oracle means verification collapses back into re-derivation, so rejecting and rewriting is the rational move.
One wrinkle I'd add, because it's where fixture generation isn't quite free: the oracle checks well-formedness, not intent. Your parser confirms the bytes are a valid-but-malformed header; it can't confirm the fixture stresses the boundary you actually meant to hit. So there's a residual tax even in the good case — just a much smaller surface than reading control flow.
Which is my question back: for generated adversarial inputs, do you trust the suite to tell you the corpus is actually adversarial, or do you still eyeball a sample to confirm it's hitting the edge you cared about — and if so, how big a sample before you trust the generator?