Originally published on CoreProse KB-incidents
A nonfiction book about artificial intelligence and truth has just failed its own reality test.
Steven Rosenbaum’s The Future of Truth: How AI Reshapes Reality includes multiple quotes that never happened—synthetic lines generated by large language models but presented as if real people had said them.[2][3]
For readers already using LLMs, this is not a minor copy error. It is a test of whether “AI‑assisted nonfiction” can be trusted at all.
This scandal sits at the intersection of:
- Generative models that create plausible but unverifiable language
- Editorial processes built for pre‑LLM research
- Emerging AI ethics frameworks demanding transparency and accountability
The goal here is less to attack one author than to use this as a case study in how nonfiction must evolve if it wants to keep its receipts in the age of AI.
1. The ‘Future of Truth’ Controversy: What Actually Happened
The New York Times found that The Future of Truth contained more than half a dozen misattributed or fully fabricated quotations.[3] They looked like sharp insights from well‑known people—but the individuals cited had never said them.[2]
Key facts:
- Rosenbaum admitted the book contained “improperly attributed or synthetic quotes” and called them accidental.[1][3]
- He disclosed in the acknowledgments that he used ChatGPT and Claude during research, drafting, and editing.[2][3]
- He now says he takes “full responsibility” and is working with editors to identify and correct all affected passages in future editions.[2][3]
- The BenBella imprint has not publicly explained its internal review process.[3]
The story went wide after the Times framed it as: a book about truth was caught using AI‑made quotes.[4][5] That framing escalated what might have looked like sloppy copyediting into a reputational crisis for author and publisher.
Context that raised the stakes:
- Rosenbaum runs the Sustainable Media Center and promotes an NYU “Master’s Degree in Truth.”[2][3]
- The book carried blurbs from Nicholas Thompson, Ian Bremmer, and Nobel laureate Maria Ressa, who wrote the foreword.[2][3]
⚠️ Why this matters: A book marketed as a serious guide to AI and reality smuggled hallucinated quotes into print. It shows how uncritical LLM use can quietly corrupt the factual record before anyone notices.
2. When LLMs Hallucinate History: Why Fabricated Quotes Are So Dangerous
Underneath the scandal is the familiar problem of AI hallucination. LLMs produced quotations and attributions that sounded right but had no verifiable source.[2] Those hallucinations were then treated as facts and fixed in print.
Core properties of LLMs like ChatGPT and Claude:[2][3]
- Optimized to predict the next token, not to retrieve ground‑truth citations
- Likely to synthesize “fitting” quotes in a public figure’s style when prompted
- Indifferent to whether a sentence has ever existed in the real world
📊 Design reality: These systems are probability engines, not evidence engines. Without external retrieval and human checking, fabrication is a feature, not a bug.
Policy context:
- U.S. “AI Bill of Rights” discussions stress that people should know when content is AI‑generated or when a system simulates a person, especially around consent and labeling.[7]
- Synthetic quotations—fake statements attributed to real people—are exactly the kind of simulation that demands clear boundaries.
Why direct quotes are high‑risk:
- They claim a specific person
- Used those exact words
- In a particular context
Inventing that combination:
- Misrepresents the speaker’s views
- Pollutes the record for journalists, scholars, and policymakers who may later cite the passage as evidence[6]
- Risks reputational harm to the person being simulated
One invented line was attributed to tech journalist Kara Swisher, who told the Times she “never said that.”[2] That goes beyond embarrassment to potential defamation and trust erosion.
⚡ The deeper irony: A book about how AI reshapes reality reshaped reality by putting AI‑written words in real people’s mouths.[2][3] It validates fears that careless generative‑AI use will erode trust in media and scholarship.
As LLMs become routine writing tools, this case will help define where publishers, regulators, and courts draw lines between assistive drafting and unmarked fabrication about identifiable individuals.[7]
3. Editorial Safeguards That Failed—and How AI Could Have Helped Catch AI
In traditional nonfiction workflows, synthetic quotes should never reach print. Rosenbaum’s case reveals at least two failures:[2][3]
- The author did not systematically verify every direct quotation.
- The editorial team did not either.
Professional copyeditors are trained to flag and verify:[6]
- Names and spellings (including accents and diacritics)
- Exact wording of quotations
- Source titles, subtitles, and publication details
If even minor details like a missing accent in Brené Brown’s name normally get checked, multiple fully synthetic quotations suggest that standard verification was patchy at best.
⚠️ Process gap: Once AI enters research and drafting, “trust but skim” becomes untenable. You must assume some sentences are fabricated until proven otherwise.
AI could also have been part of the solution. Editor Kristen Tate has shown that AI tools can assist fact‑checking by:[6]
- Comparing quote variants across sources
- Suggesting likely origins
- Flagging quotations that lack corroboration in accessible databases
Her work underlines a key rule: AI can help verify only when it is paired with external search and human judgment—not when it is both generator and sole checker.[6]
A safer AI‑assisted nonfiction workflow might include:
- Use LLMs for brainstorming, outlining, and polish—not for final factual claims.
- Keep a log of all AI‑suggested quotes, facts, and attributions.
- Run a separate fact‑checking pass using search, citation databases, and possibly a differently configured AI tool focused on validation.
- Require human sign‑off on every direct quote and key factual statement.
💡 Transparency layer: Federal AI policy work emphasizes that people should know when AI is used and have enough information to interpret outputs.[7] For books, that implies:
- Clear AI‑use disclosures in the front matter
- A visible distinction between AI‑drafted prose and verified citations
At the publisher level, the Future of Truth episode suggests concrete standards:[2][3]
- Mandatory AI‑use statements for any LLM‑assisted manuscript
- Documented quote‑checking procedures when AI is involved
- Incident‑response playbooks for AI‑related errors, including public corrections and updated editions
Handled this way, a scandal can become a driver for stronger editorial infrastructure.
4. Toward Ethical AI‑Assisted Nonfiction: Frameworks, Stakeholders, and Industry Trajectory
Publishing does not need to invent an ethics framework from zero. Investors and AI practitioners are proposing models like E.T.H.I.C.S., which emphasizes asking “What could possibly go wrong?” and centering explainability, transparency, human oversight, impact, consent, and safety.[8][10]
Applied to AI‑assisted nonfiction, those principles translate into:
- Explainable: The author can describe where AI entered the process and why specific outputs were accepted or rejected.[8]
- Transparent: Readers know that AI was used and in what roles.
- Human‑overseen: Humans have authority—and time—to override or discard AI suggestions, especially around quotes and attribution.[8][10]
- Impact‑aware: Teams anticipate who could be harmed if fabricated quotes or misattributed views slip through.[10]
💼 Real‑world practice: One manager at a small media nonprofit uses LLMs for research summaries with a standing rule: “No AI‑generated sentence goes to print without a human tracing it back to a primary source.” That kind of norm was missing in The Future of Truth workflow.
Major institutions are also weighing in. The Vatican has formed an in‑house AI study group and is preparing an encyclical framing AI ethics around human dignity, justice, and peace.[9] Church leaders explicitly compare today’s AI revolution to the moral upheavals of the Industrial Revolution.[9]
For nonfiction about AI, that creates a double obligation:
- Accurately describe AI’s risks and benefits.
- Model responsible AI use in its own production—or risk undermining trust in journalism, academia, and regulation.[7][9]
Stakeholders face different incentives:
- Authors: Want faster research and drafting, but must own verification.[2]
- Editors/publishers: Need scalable quality control and clear liability boundaries when AI is used.[3]
- Readers: Increasingly expect transparency and reliability in AI‑touched content.[7]
- Institutions: From nonprofits to religious bodies, are framing AI governance around trust, accountability, and human‑centered values.[8][9]
⚡ Likely trajectory: Expect movement toward:
- Contract clauses mandating AI‑use disclosure
- AI‑audit tools that scan manuscripts for likely hallucinations
- Ethics guidelines treating fabricated quotations as a predictable LLM failure mode—not a rare personal lapse[7][10]
Conclusion: Rebuilding Trust Before the Next “Future of Truth”
The Future of Truth scandal is not just one author’s misstep. It exposes a structural collision between probabilistic language models and the traditional trust signals of nonfiction—footnotes, blurbs, institutional credentials—that many readers still take at face value.[2][3]
We have seen how LLM hallucinations can manufacture quotations, how conventional editorial safeguards can fail to catch them, and how emerging frameworks—from copyediting practices to federal policy to Vatican statements—converge on transparency, verification, and human oversight as non‑negotiables.[6][7][9]
Trustworthy AI‑assisted nonfiction is possible, but only with:
- Explicit standards for AI use
- Auditable workflows that separate generation from verification
- Honest disclosure about where and how LLMs are involved
If you are an author, editor, or tech‑savvy professional, treat this episode as a cue to audit your own AI practices. Map where LLMs enter your workflow, design verification steps for anything presented as fact or quotation, and press publishers, platforms, and policymakers to adopt clear AI‑disclosure and fact‑checking norms—before the next book on “truth” has to retract its own footnotes.
About CoreProse: Research-first AI content generation with verified citations. Zero hallucinations.
Top comments (0)