DEV Community: Fathy Shalaby

AI is writing your PRDs. Who's checking if they're right?

Fathy Shalaby — Fri, 24 Jul 2026 08:58:29 +0000

By 2026, writing PRDs is one of the top things product managers hand to AI. That's the good news — first drafts are grunt work, and grunt work is exactly what AI should do. The problem is what a PRD is: not a summary you skim and forget, but the decision artifact that tells engineers what to build. If the doc is confidently wrong, the wrong thing gets built — and almost nobody is checking. AI generates fluent, finished-looking output at hallucination rates that, depending on the task, run anywhere from ~1% to over 50%; meanwhile most workers admit they don't verify it. This isn't an argument against using AI to draft. It's an argument for one rule: if a machine wrote it, the evidence under every claim has to be checkable — otherwise you're shipping confidence, not truth.

AI writes the artifacts now

The adoption data is unambiguous: 94% of product professionals use AI daily or often, and among the very top use cases is writing PRDs — alongside decks, competitive research, and roadmaps (Productboard, October 2025, n=379). Generating the product document is no longer the bottleneck. Which means the bottleneck moved to the thing after it: knowing whether the document is right.

AI is confidently wrong — at rates that matter

"Hallucination" isn't one number; it's a family of failure modes that spike or shrink with the task. But the ranges are sobering for anyone using AI on factual, decision-shaping work:

Top models on grounded tasks (where they must stick to provided sources) get hallucination rates down to ~0.7–1.5%.
Left to answer from their own weights, the same class of models ranges much higher — a 2026 benchmark across 37 models found rates between ~15% and 52%, depending on task and domain.
In adversarial citation tests, fabricated-citation rates as high as ~94% have been recorded — models inventing sources that look real.

The pattern is the tell: hallucination collapses when the model is grounded in real evidence, and balloons when it's free-associating. A PRD generated from nothing but a prompt is the second case wearing the formatting of the first.

And almost nobody is checking

Even where the output is wrong, verification is the exception, not the rule:

66% of people rely on AI output without evaluating its accuracy (KPMG × University of Melbourne, Trust in AI global study — 48,000 people across 47 countries, 2025).
56% report making mistakes in their work because of AI (same study).
And the cost shows up on the other side of the "time saved" ledger: 37% of the time AI saves is lost to rework — correcting, clarifying, or rewriting low-quality output. For every ~10 hours gained, nearly 4 go back to fixing it (Workday, released Jan 2026).

Put those together: fluent output, produced faster than anyone can check it, trusted by default by most of the people receiving it. That's a machine for manufacturing confident, unverified claims at scale.

Why this is worse for a PRD than for an email

A wrong sentence in a Slack message costs a follow-up. A wrong premise in a PRD costs a quarter. The document is the input to what gets built — a fabricated user need, an invented market stat, or a misremembered competitor fact doesn't get caught downstream; it gets implemented. This is the same failure that kills 43% of startups — building something nobody needed (CB Insights) — except now it arrives pre-formatted, well-organized, and sounding authoritative. The polish is the danger. A rough wrong draft invites scrutiny; a beautiful wrong draft gets approved.

The fix isn't "stop using AI." It's "make it show its work."

The same benchmarks that expose the problem point straight at the solution: grounding. When generation is tied to real, retrievable evidence — actual customer quotes, real usage data, cited sources — hallucination drops toward that ~1% floor, and, more importantly, every claim becomes checkable. The question stops being "do I trust the model?" and becomes "can I see the receipt for this line?"

So the standard for AI-generated product work in 2026 is simple, and it's not "use AI" or "don't":

Every factual claim traces to a source you can open — a quote, a metric, a document — not the model's confidence.
The draft is treated as a draft until the evidence is verified, not as a finished decision because it looks finished.
Ungrounded generation is fine for phrasing, never for facts. Let AI write the sentences; make it prove the claims.

Generation got cheap. Being right didn't. The teams that win with AI-written docs won't be the ones producing the most of them — they'll be the ones who can point at any line and show where it came from.

The numbers, in one place

What	Number	Source
Product pros using AI daily/often; PRD-writing a top use case	94%	Productboard
Hallucination rate, grounded tasks (top models)	~0.7–1.5%	2025 grounded-summarization benchmarks
Hallucination rate, ungrounded, across 37 models	~15–52%	2026 hallucination benchmark
Fabricated-citation rate, adversarial tests	up to ~94%	hallucination benchmarks
Workers relying on AI output without assessing it	58%	KPMG "Trust in AI" (2025)
Workers reporting AI-driven errors	57%	KPMG (2025)
AI time-savings lost to rework	37%	Workday (2025)
Startup failures from building the wrong thing	43%	CB Insights

The takeaway

AI is a genuinely great first-draft writer and a genuinely unreliable fact-checker of its own work. Those two things are both true, and the gap between them is where wrong products get built. The move isn't to stop letting AI draft your PRDs — it's to refuse to let a claim into one without a source you can check. Make the machine show its work, or you're not reviewing a spec. You're rubber-stamping a guess.

Related in this series: "Can AI replace product managers? What the 2026 data says" · "Building got cheap, knowing what to build didn't" · "Startups don't run out of money. They run out of evidence."

Sources

Productboard — The New Reality of AI in Product Management (Oct 2025, n=379): AI daily usage; PRD-writing a top use case.
2025–2026 LLM hallucination benchmarks (e.g. HalluLens; grounded-summarization leaderboards; 37-model 2026 cohort study) — task-dependent hallucination ranges; grounding reduces rates toward ~1%; adversarial citation-fabrication rates.
KPMG / University of Melbourne — Trust, attitudes and use of AI: a global study 2025 (47 countries): reliance on AI output without verification; AI-driven errors.
Workday — 2025 AI-at-work research: share of AI time-savings lost to rework.
CB Insights — Why Startups Fail (43% poor product-market fit).

Can AI replace product managers? What the 2026 data actually says

Fathy Shalaby — Fri, 24 Jul 2026 08:51:39 +0000

TL;DR

By 2026, using AI is no longer the question — 94% of product professionals use AI daily or often, and effectively every product team surveyed now uses AI tools. But look at what it took over: writing PRDs, building presentations, competitive research, synthesizing feedback — the coordination and grunt work that already ate ~60% of the week. What it did not take over is the part that decides whether a product lives: knowing what's worth building, and defending that call with evidence. Only ~6% of teams have made AI a "core, strategic capability" — because strategy isn't what it's good at. The data is blunt about which one matters: 43% of startups die from building something nobody needed, not from building it too slowly. So no, AI doesn't replace the product manager. It deletes the 80% of the job that was never the point and raises the price of the 20% that always was.

The adoption question is already settled

94% of product professionals use AI daily or often, and 100% of teams surveyed use AI tools — with nearly half calling it "deeply embedded" (Productboard, survey of 379 enterprise product professionals, October 2025).
62% report saving at least ~4 hours a week; the most-cited realized value is "time saved on repetitive tasks" (59.8%) and "faster insight synthesis" (50.4%) (State of Product Management 2026, Product-Led Alliance × ProductPlan).
The top use cases are writing PRDs, building presentations, competitive research, and roadmap creation.

Adoption isn't the story anymore. The story is what those hours went to — and, tellingly, what they didn't.

The adoption is deep — but it's all tactical

Here's the number that reframes the rest: only ~6% of teams have made AI "a core, strategic capability." The rest use it for limited workflows (~37%) or early experimentation (~32%) (State of Product Management 2026). Near-total adoption, concentrated almost entirely in tactical, mechanical work. That's not a gap teams are failing to close — it's a description of what the technology is actually good at.

What AI absorbed: the 80%

McKinsey estimates current generative AI and related tech can automate activities that absorb 60–70% of employees' time — with the biggest impact on knowledge work. For a product manager, that maps almost exactly onto the part of the week that was never the actual job:

Synthesizing scattered feedback into themes.
Drafting the first version of a spec, a brief, a status update.
Assembling the inputs for a prioritization call.
Chasing and summarizing "what's the status of X."

This is the "work about work" that Asana's Anatomy of Work pegged at ~60% of the workday, leaving only ~13% for strategic planning. AI is very good at this layer precisely because it's mechanical. Offloading it is a genuine, measured win — the hours saved are real.

But automating the overhead doesn't make the overhead the job. It just clears the desk.

What it didn't absorb — and why that's the part that counts

The activity AI is worst at automating is the one that determines outcomes. McKinsey's own analysis notes that decision-making and collaboration — judgment work — historically had the lowest potential for automation, and remain the hardest even as everything around them gets automated. And the cost of getting that judgment wrong is not marginal:

43% of failed startups shut down because of poor product-market fit — they built something the market didn't need (CB Insights, 2024 analysis of 431 shutdowns). The earlier study of 110+ post-mortems put "no market need" at the top at 42% — the number barely moved with 4× the data.
Those 431 companies had raised a combined $17.5 billion. Capital wasn't the root cause; CB Insights calls "ran out of money" the final symptom. What they lacked was evidence that anyone wanted what they were building.
And it compounds downstream: across shipped software, a large majority of features are rarely or never used — effort spent building the wrong 80%.

Read those together and the role inverts cleanly against the automation curve. AI is cheapest and best at exactly the tasks that don't decide success (drafting, summarizing, formatting) and useless-on-its-own at the one that does (deciding what's worth building, and proving it). You cannot prompt your way to product-market fit, because the missing ingredient isn't generation — it's grounded judgment about a specific market that no general model has seen.

The trap of the confident first draft

There's a second-order risk that makes the 20% more important in the AI era, not less. AI will happily generate a polished PRD for the wrong feature — fluently, instantly, and with total confidence. A draft that looks finished but rests on no evidence is indistinguishable, at a glance, from one that's right. The faster and cheaper generation gets, the easier it is to mass-produce confident artifacts for things nobody asked for — which is the 43%-failure mode with better formatting.

So the scarce, human, un-automatable work in 2026 isn't writing the spec. It's the chain underneath it: what did customers actually say, which signal is real, what's the evidence this is worth building, and how do we know if we were right after we shipped? That's judgment with receipts — the opposite of a one-shot generation.

What the job actually becomes

"Can AI replace product managers?" is the wrong frame. The right one: AI collapses the 80% of the PM week that was overhead, and in doing so makes the remaining 20% — judgment, grounded in evidence — the entire job. The winning product manager in 2026 isn't the one who generates the most documents. It's the one who spends the reclaimed hours on the decision that determines whether the next thing shipped is worth shipping at all.

The bottleneck moved. For a decade it was can we build this. Now that building is cheap and fast, it's do we know this is worth building — and that's the one question AI can't answer for you. It can only make sure you have the evidence in front of you when you answer it yourself.

The numbers, in one place

What	Number	Source
Product pros using AI daily or often (2025)	94%	Productboard
Product teams using AI tools	~100%	Productboard
PMs saving ≥4 hrs/week	62%	State of Product Management 2026
Teams with AI as a "core, strategic capability"	~6%	State of Product Management 2026
Employee time in activities AI can automate	60–70%	McKinsey
Workday spent on "work about work"	~60%	Asana
Workday left for strategic planning	~13%	Asana
Startup failures from poor product-market fit	43% (was 42% "no market need")	CB Insights
Combined capital raised by 431 failed startups	$17.5B	CB Insights

The takeaway

AI didn't come for the product manager's job. It came for the busywork wrapped around it — and handed back the hours. The teams that win with that gift won't be the ones generating specs faster. They'll be the ones spending the reclaimed time on the one thing no model can do for them: deciding what's worth building, and being able to show why.

Related in this series: "Building got cheap, knowing what to build didn't" · "Where a product manager's week actually goes" · "Why 80% of software features are never used."

Sources

Productboard — The New Reality of AI in Product Management (Oct 2025, n=379 enterprise product professionals): 94% use AI daily/often; 100% of teams use AI tools; top use cases writing PRDs, presentations, competitive research, roadmaps.
State of Product Management 2026 — Product-Led Alliance × ProductPlan (Q4 2025, ~250 professionals): AI-adoption stages (~6% "core strategic capability", ~37% limited workflows, ~32% experimentation); 62% save ≥4 hrs/week; top realized value time-saved (59.8%) and faster insight synthesis (50.4%).
McKinsey — The economic potential of generative AI (60–70% of work-time activities automatable; decision-making/collaboration lowest automation potential).
CB Insights — Why Startups Fail (poor product-market fit 43% / "no market need" 42%; capital as final symptom; 431 shutdowns, $17.5B raised).
Asana — Anatomy of Work (~60% "work about work"; ~13% strategic planning).