DEV Community

Sho Naka
Sho Naka

Posted on Edited on

McDonald's ran an AI drive-thru pilot for 3 years across 100+ stores, then killed it. It wasn't an accuracy problem.

#aiadoption #digitaltransformation #productivity #genai

McDonald's ran an AI drive-thru voice system with IBM across more than 100 stores, starting in 2021. In July 2024, they ended the entire test. It wasn't an accuracy problem.

TL;DR: A 2025 S&P Global survey found that the share of companies scrapping most of their AI initiatives jumped to 42%, up from 17% the year before — and the average organization abandons 46% of its AI proof-of-concepts before they ever reach production. In AI adoption support work, the stalls keep tracing back to the same pattern: teams design and validate for "does it work under lab conditions," and never design for "who reviews the output, and what happens when it's wrong." Those are the two questions that actually determine whether an AI deployment survives contact with a live environment.

What actually happened

The McDonald's/IBM drive-thru pilot worked in controlled testing. In the field, it didn't hold up against ambient noise, regional accents, and multiple customers talking over each other. A system that performs correctly in a quiet lab and fails against wind noise and a fast regional accent isn't a smaller version of the same problem — it's a different problem that was never tested for.

MD Anderson Cancer Center's IBM Watson oncology project — a $62 million investment — followed the same shape. It worked against curated, cleaned datasets. Real patient records — missing fields, ambiguous notation, inconsistent timelines — broke it. The project ended with zero real patients ever treated using the system.

The number that changed my mind 

S&P Global (2025): companies scrapping most AI initiatives    = 42% (up from 17% the year before)
S&P Global (2025): AI proof-of-concepts abandoned pre-production = 46% (average org)
TIS survey (domestic): companies with org-wide AI adoption     = 21.9%
Enter fullscreen mode Exit fullscreen mode

Roughly half of AI proof-of-concepts never make it to production. S&P Global's respondents pointed to cost, data privacy, and security risk as the biggest blockers — the kind of thing that shows up on a budget spreadsheet, not in a demo. But blocker categories that broad don't explain why a specific rollout stalls. In the adoption support work I do, the proximate cause is almost always narrower and more mundane: getting output out of an AI is trivial now. The unsolved part is where that output plugs into the actual workflow, who reviews it, and how you judge whether it worked.

AI-generated copy tends to contain output that reads as plausible but is subtly wrong. Reviewing that kind of output for correctness takes longer than an experienced writer drafting from scratch — because starting from a blank page lets you set your own pace, and reading someone else's (or something else's) draft to judge its correctness is a different cognitive task entirely. That's the mechanism behind "have the AI draft it, then fix it" turning out slower than drafting from scratch.

The rule I extracted

Cross-referencing the public case studies and survey data against what adoption support work actually involves, three recurring failure patterns account for most of the stalls:

  1. The AI got bolted onto the existing workflow instead of replacing a step in it. A human still does the work, the AI's output gets added as an extra check, and total steps — and total time — go up.
  2. Nobody defined what "review" means, specifically. "Check the AI's output" without a specific checklist turns into re-reading the entire thing every time, which is slower than not using AI at all. Left undefined, that review step calcifies into "read everything, just in case it's AI-written" — and the team ends up feeling like adopting AI added work rather than removed it.
  3. Nobody designed the fallback for when it's wrong. The first bad output triggers a manager escalation, and the team quietly stops using the tool rather than build a recovery path.

A pilot answers "does it work." Production requires answering "who catches it when it doesn't, and what do they do next." Skipping the second question is why pilots don't survive contact with a live environment.

The inverse also holds, in deployments that stuck: review criteria were specific (check the numbers, check proper-noun consistency, check tone against a guideline) rather than "read the whole thing." That turns review from re-reading into cross-checking — faster, and it doesn't wear the reviewer down over time. And a fallback path existed before the first failure happened: "if this output looks wrong, fall back to this template and proceed manually." That single sentence, decided in advance, is the difference between a stalled rollout and one that survives its first bad output.

Try it yourself

Before choosing a tool, write down the judgment calls that repeat in the workflow you're targeting — "check whether this reply matches the client's tone," "check whether this number crosses a threshold," "pick the matching item from this list." For each one, ask which part is information-gathering (an AI can plausibly do this) versus which part is the actual decision (a human still makes this call). Measure the current baseline first — minutes per task, task volume per month, current error rate — because you can't tell if the AI helped without a number to compare against.

What's the specific judgment call, repeated daily in your own workflow, that you've never actually written down?

Sho Naka (nomurasan). I do AI adoption support work day to day; these patterns come from cross-referencing published case studies and survey data against what that work actually involves, not from a single case.

This piece was adapted from a Japanese essay, with AI assistance for the cross-language rewrite. The reasoning, data, and conclusions are mine.

References

Correction (2026-08-19)

An earlier version of this article cited a "Cisco 2025 survey" figure of 5% for AI pilots reaching production. That number traced back to a 2026 conference remark, scoped to AI agents specifically — not AI pilots in general. The TL;DR and data block above have been replaced with S&P Global survey figures. A claim that the top adoption blocker "wasn't security or cost concerns" has also been removed: the survey it referenced reported the opposite.

Top comments (0)