By 2026, writing PRDs is one of the top things product managers hand to AI. That's the good news — first drafts are grunt work, and grunt work is exactly what AI should do. The problem is what a PRD is: not a summary you skim and forget, but the decision artifact that tells engineers what to build. If the doc is confidently wrong, the wrong thing gets built — and almost nobody is checking. AI generates fluent, finished-looking output at hallucination rates that, depending on the task, run anywhere from ~1% to over 50%; meanwhile most workers admit they don't verify it. This isn't an argument against using AI to draft. It's an argument for one rule: if a machine wrote it, the evidence under every claim has to be checkable — otherwise you're shipping confidence, not truth.
AI writes the artifacts now
The adoption data is unambiguous: 94% of product professionals use AI daily or often, and among the very top use cases is writing PRDs — alongside decks, competitive research, and roadmaps (Productboard, October 2025, n=379). Generating the product document is no longer the bottleneck. Which means the bottleneck moved to the thing after it: knowing whether the document is right.
AI is confidently wrong — at rates that matter
"Hallucination" isn't one number; it's a family of failure modes that spike or shrink with the task. But the ranges are sobering for anyone using AI on factual, decision-shaping work:
- Top models on grounded tasks (where they must stick to provided sources) get hallucination rates down to ~0.7–1.5%.
- Left to answer from their own weights, the same class of models ranges much higher — a 2026 benchmark across 37 models found rates between ~15% and 52%, depending on task and domain.
- In adversarial citation tests, fabricated-citation rates as high as ~94% have been recorded — models inventing sources that look real.
The pattern is the tell: hallucination collapses when the model is grounded in real evidence, and balloons when it's free-associating. A PRD generated from nothing but a prompt is the second case wearing the formatting of the first.
And almost nobody is checking
Even where the output is wrong, verification is the exception, not the rule:
- 66% of people rely on AI output without evaluating its accuracy (KPMG × University of Melbourne, Trust in AI global study — 48,000 people across 47 countries, 2025).
- 56% report making mistakes in their work because of AI (same study).
- And the cost shows up on the other side of the "time saved" ledger: 37% of the time AI saves is lost to rework — correcting, clarifying, or rewriting low-quality output. For every ~10 hours gained, nearly 4 go back to fixing it (Workday, released Jan 2026).
Put those together: fluent output, produced faster than anyone can check it, trusted by default by most of the people receiving it. That's a machine for manufacturing confident, unverified claims at scale.
Why this is worse for a PRD than for an email
A wrong sentence in a Slack message costs a follow-up. A wrong premise in a PRD costs a quarter. The document is the input to what gets built — a fabricated user need, an invented market stat, or a misremembered competitor fact doesn't get caught downstream; it gets implemented. This is the same failure that kills 43% of startups — building something nobody needed (CB Insights) — except now it arrives pre-formatted, well-organized, and sounding authoritative. The polish is the danger. A rough wrong draft invites scrutiny; a beautiful wrong draft gets approved.
The fix isn't "stop using AI." It's "make it show its work."
The same benchmarks that expose the problem point straight at the solution: grounding. When generation is tied to real, retrievable evidence — actual customer quotes, real usage data, cited sources — hallucination drops toward that ~1% floor, and, more importantly, every claim becomes checkable. The question stops being "do I trust the model?" and becomes "can I see the receipt for this line?"
So the standard for AI-generated product work in 2026 is simple, and it's not "use AI" or "don't":
- Every factual claim traces to a source you can open — a quote, a metric, a document — not the model's confidence.
- The draft is treated as a draft until the evidence is verified, not as a finished decision because it looks finished.
- Ungrounded generation is fine for phrasing, never for facts. Let AI write the sentences; make it prove the claims.
Generation got cheap. Being right didn't. The teams that win with AI-written docs won't be the ones producing the most of them — they'll be the ones who can point at any line and show where it came from.
The numbers, in one place
| What | Number | Source |
|---|---|---|
| Product pros using AI daily/often; PRD-writing a top use case | 94% | Productboard |
| Hallucination rate, grounded tasks (top models) | ~0.7–1.5% | 2025 grounded-summarization benchmarks |
| Hallucination rate, ungrounded, across 37 models | ~15–52% | 2026 hallucination benchmark |
| Fabricated-citation rate, adversarial tests | up to ~94% | hallucination benchmarks |
| Workers relying on AI output without assessing it | 58% | KPMG "Trust in AI" (2025) |
| Workers reporting AI-driven errors | 57% | KPMG (2025) |
| AI time-savings lost to rework | 37% | Workday (2025) |
| Startup failures from building the wrong thing | 43% | CB Insights |
The takeaway
AI is a genuinely great first-draft writer and a genuinely unreliable fact-checker of its own work. Those two things are both true, and the gap between them is where wrong products get built. The move isn't to stop letting AI draft your PRDs — it's to refuse to let a claim into one without a source you can check. Make the machine show its work, or you're not reviewing a spec. You're rubber-stamping a guess.
Related in this series: "Can AI replace product managers? What the 2026 data says" · "Building got cheap, knowing what to build didn't" · "Startups don't run out of money. They run out of evidence."
Sources
- Productboard — The New Reality of AI in Product Management (Oct 2025, n=379): AI daily usage; PRD-writing a top use case.
- 2025–2026 LLM hallucination benchmarks (e.g. HalluLens; grounded-summarization leaderboards; 37-model 2026 cohort study) — task-dependent hallucination ranges; grounding reduces rates toward ~1%; adversarial citation-fabrication rates.
- KPMG / University of Melbourne — Trust, attitudes and use of AI: a global study 2025 (47 countries): reliance on AI output without verification; AI-driven errors.
- Workday — 2025 AI-at-work research: share of AI time-savings lost to rework.
- CB Insights — Why Startups Fail (43% poor product-market fit).
Top comments (0)