Originally published on The Searchless Journal
The Irony Writes Itself — Except It Didn't
Steven Rosenbaum wrote a book about how AI threatens the concept of truth. He called it The Future of Truth. He used ChatGPT and Claude to help research and write it. The AI tools fabricated quotes that ended up in the published book. One of those fake quotes was attributed to Kara Swisher, who never said it.
You could not design a more perfect allegory for the problem AI poses to information reliability if you tried.
The New York Times broke the story on May 19, 2026. The Atlantic followed with a deeper analysis of what it called a "wave" of AI writing scandals hitting the literary world simultaneously. By the time the dust settled, the Rosenbaum incident was no longer an isolated embarrassment — it was a data point in a systemic pattern that has direct implications for every brand, publisher, and information source that depends on AI search engines for visibility.
The thesis of this piece is not that AI makes mistakes. That is well-established. The thesis is that the compounding nature of AI-generated errors — AI generates flawed content, the content gets published, AI search engines cite the published content as authoritative — represents a structural degradation of the information ecosystem that most GEO strategies are not accounting for.
This is the part that should worry you even if you have never written a book and never plan to.
What Actually Happened
Rosenbaum is not a fringe figure. He is an established media entrepreneur and documentary filmmaker. His book, published by a mainstream house, examined how AI is reshaping our relationship with truth and factual accuracy. The premise was serious. The execution undermined it.
According to the Times investigation, multiple quotes in the book could not be verified against any original source. The fabricated Kara Swisher quote was the most prominent example, but it was not alone. When confronted, Rosenbaum acknowledged using Claude and ChatGPT as research and writing tools. He told The Atlantic: "ChatGPT fucked up the book." He also described AI as "often a delightful writing companion... and then it betrays you in ways that are just really quite horrible."
Pangram, an AI-detection tool, flagged a 146-word passage in the book as 100% AI-generated. Rosenbaum denied that AI wrote sections of the book, insisting the tools were used for research and editing assistance, not composition. The distinction matters less than the outcome: fabricated quotes from an AI tool made it through the entire editorial process — author research, drafting, editing, fact-checking (or its absence), and publication — and into a printed book about truth itself.
The irony is the headline. The systemic problem is the story.
The Same Week, a Pattern
Rosenbaum was not alone. The same week his story broke, multiple literary scandals involving AI converged:
- Commonwealth Short Story Prize winners were accused of submitting AI-generated work. The Commonwealth Foundation issued statements acknowledging the allegations and reviewing submissions.
- A Nobel-winning novelist faced public accusations of using AI in recent work.
- Granta, one of the most prestigious literary magazines in the English language, found itself caught in the controversy as questions about AI-authored submissions spread across the literary prize circuit.
- A working paper from researchers studying Amazon's book catalog estimated that over half of all new books published on the platform now contain at least some AI-generated text.
None of these incidents occurred in isolation. They formed a pattern: AI writing tools have reached the mainstream of professional content creation, and the editorial infrastructure to verify AI-assisted work has not kept pace.
This is a cultural problem. It is also, less obviously, an infrastructure problem for every business that cares about how AI search engines find, evaluate, and cite information.
Why This Matters for AI Search — Not Just Literature
Here is the connection most coverage has missed.
The BuzzStream/XOFU citation study, released the same week, analyzed 4 million AI citations across 3,600 prompts spanning 10 industries. The study's most striking finding: editorial content — reported, written, and published by human journalists — accounted for 81% of all news citations in AI search answers. Press releases accounted for 0.04%.
Let me state that again because the asymmetry is staggering: 81% versus 0.04%.
AI search engines, from ChatGPT to Google AI Overviews to Perplexity to Gemini, depend overwhelmingly on editorial content as their citation base. They rely on the assumption that the articles, books, and reports they cite were produced through some version of journalistic or editorial process — research, fact-checking, editorial review. That assumption is the foundation of the citation chain.
When that editorial content is itself AI-generated and unverified, the citation chain degrades from both ends. Here is how the compounding loop works:
- A human researcher uses AI (ChatGPT, Claude, Gemini) to generate content or research.
- The AI produces plausible but fabricated or inaccurate information.
- The human publishes that information without adequate verification.
- The published content enters the web corpus.
- AI search engines crawl, index, and cite that published content as an authoritative source.
- The fabricated information is now cited as fact by AI answers.
- Those AI answers inform further content creation, completing the loop.
This is not hypothetical. It is already happening. And it is accelerating.
The Bigfoot Effect Compounds the Problem
Searchless reported on May 18 on what we called the Bigfoot Effect — the phenomenon where AI citation concentration narrows over time, with fewer and fewer sources receiving citations as AI engines build consensus around a shrinking pool of "authoritative" answers.
The Bigfoot Effect means that once fabricated content enters the citation chain, it has an outsized chance of being amplified. AI engines do not just cite one source — they build synthesized answers from multiple sources. When multiple sources contain the same fabrication (because they were all informed by the same AI output), the synthesis reinforces the error. The AI does not see five independent sources confirming a fact. It sees five sources confirming a fabrication. The error looks authoritative because it is corroborated.
Citation concentration data from the Searchless study showed a 20% reduction in the number of distinct brands cited in follow-up testing of the same queries. The citation pool is shrinking. The cost of a fabrication entering that pool is increasing.
This is the compounding hallucination problem, and it is the most under-discussed risk in generative engine optimization today.
The Editorial Content Crisis Is a Citation Infrastructure Crisis
Consider the numbers in context:
- BuzzStream found that ChatGPT cites company newsrooms at an 18% rate for news queries. Google's AI platforms cite newsrooms at approximately 3%.
- Evaluative and comparison prompts generate the highest citation rates for news content — 18% for evaluative queries versus 7% for brand awareness queries.
- Press releases are nearly invisible in AI citations at 0.04%.
This means AI search engines are making a bet: editorial content is more trustworthy, more substantive, and more worth citing than corporate communications. They are structurally biased toward citing journalism, books, and independent analysis over branded content.
That bet is only as good as the editorial content it is based on. When books about truth contain AI-fabricated quotes, when literary prizes are won with AI-generated prose, when Amazon's catalog is reportedly majority AI-assisted, the trust signal that editorial content carries degrades. And because AI engines overweight editorial citations, the degradation has an outsized impact on answer quality.
This is not just a problem for book publishers and literary magazines. It is a problem for every brand whose AI visibility strategy depends on being cited by AI search engines. If the citation infrastructure is degrading, the optimization strategies built on top of it need to account for that degradation.
What This Means for Brands and GEO Strategy
The practical implications are threefold.
First, citation authority is becoming a trust signal that needs active management. Brands that invest in producing genuinely researched, fact-checked, source-backed editorial content will stand out in an environment where the baseline quality of the web corpus is declining. The "editorial advantage" in AI citations — the 81% versus 0.04% gap — is not static. It will widen as AI-generated content floods the web, but only for brands that maintain genuine editorial standards. Content that looks editorial but is AI-generated will eventually be detected and deprioritized by AI citation algorithms.
Second, the citation chain audit is a new category of GEO work. Brands should not only track whether they are being cited by AI engines. They should track what content is being cited alongside them, and whether that co-cited content is reliable. If an AI answer cites your brand alongside three fabricated sources, your brand's association with those sources matters. You are known by the company your citations keep.
Third, the window for building citation authority on genuine editorial content is open but closing. Right now, the editorial citation advantage is massive. As AI-generated content proliferates, AI engines will need to develop better detection and filtering mechanisms. Those mechanisms will likely reward content with verifiable sourcing, clear authorship, editorial review processes, and provenance signals. Brands that invest in those signals now will have a compounding advantage.
The Measurement Gap
Here is what makes this problem hard to address: most brands have no systematic way to measure the quality of the citation ecosystem they depend on.
AI visibility audits — the kind Searchless provides — typically measure whether and how often a brand appears in AI answers. That is necessary but insufficient. A complete AI visibility strategy also needs to understand:
- What sources are being cited alongside your brand?
- Are those sources reliable, or do they carry fabrication risk?
- Is the citation pool for your industry expanding or contracting?
- Are AI engines citing editorial content or AI-generated content in your space?
This is the next frontier of GEO measurement. Citation quality auditing, not just citation presence auditing.
The Provenance Question
The content provenance infrastructure being built by OpenAI and Google — SynthID watermarking, C2PA metadata, content authenticity standards — is relevant here but insufficient. Searchless covered the SynthID and C2PA developments on May 22. Those tools address the detection side: they help identify whether content was AI-generated.
They do not address the citation side. Even if every AI-generated text were perfectly watermarked, AI search engines would still need to decide whether to cite watermarked content. And right now, they are not making that distinction consistently. BuzzStream's data shows AI engines citing content based on perceived authority and relevance, not based on whether the content was human-authored or machine-generated.
The provenance layer is being built. The citation filtering layer that would use it is not.
The Deeper Risk: Truth as a Service
Rosenbaum's book failed in a specific, detectable way: it contained fabricated quotes that could be checked against a living person's public record. Kara Swisher could deny saying something. The fabrication was falsifiable.
Most AI-generated content does not fail in ways that are so easily caught. The more insidious risk is content that is plausible, internally consistent, and difficult to falsify — but wrong in ways that matter. A synthetic statistic that is close to accurate. A paraphrase that captures the gist but inverts the emphasis. An attribution that points to a real person but fabricates the context of their statement.
This is the category of error that will compound fastest in AI citation chains, because it is the category of error that is hardest for both humans and AI systems to detect. AI search engines synthesizing answers from multiple sources will treat plausible-sounding content as corroborating evidence, not as a shared point of failure.
The Searchless AI visibility audit methodology accounts for this by testing citation behavior across multiple sessions and checking for volatility — the same query producing different cited sources across tests. Citation volatility is an indicator that the citation chain is unstable, which often correlates with the presence of unreliable or fabricated content in the citation pool.
The Bottom Line
A book about AI and truth contained AI-generated lies. Literary prizes were awarded to AI-generated work. More than half of new Amazon books reportedly contain AI-generated text. AI search engines depend on editorial content for 81% of their news citations. The editorial content pool is being diluted with AI-generated, unverified material.
This is not a literary scandal. It is an information infrastructure crisis that directly affects the reliability of every AI search answer. And it is compounding.
Brands that take AI visibility seriously need to start taking citation quality seriously — not just their own citations, but the health of the citation ecosystem they depend on.
Find out what AI search engines see when they look for your brand — and whether the sources they cite alongside you are reliable. Run a free AI visibility audit to measure your citation presence, citation quality, and competitive positioning across ChatGPT, Google AI Overviews, Perplexity, and Gemini.
Sources
- New York Times. "Future of Truth" AI quotes investigation. May 19, 2026.
- The Atlantic. "AI-Writing Scandals Are Getting Very Confusing." May 22-23, 2026.
- The Verge. "The Future of Truth has a problem in its fabricated present." May 19, 2026.
- Ars Technica. Rosenbaum interview on AI writing tools. May 23, 2026.
- BuzzStream/XOFU. 4 Million AI Citation Study. May 22-23, 2026.
- Searchless Journal. "The ChatGPT Bigfoot Effect: Citation Concentration and What It Means for Brands." May 18, 2026.
- Searchless Journal. "OpenAI and Google's SynthID Push: What Content Provenance Means for Brand Trust." May 22, 2026.
- Commonwealth Foundation. Statements on AI writing allegations. May 2026.
- Defector. Patrick Redford on AI writing scandals. May 2026.
FAQ
How can brands protect against the compounding AI hallucination problem in citations?
Brands should invest in producing genuinely researched, source-backed editorial content with clear authorship and editorial review processes. They should also audit the citation ecosystem around their brand — not just whether they are cited, but what sources are cited alongside them, and whether the overall citation pool in their industry is reliable.
What is the compounding hallucination problem?
The compounding hallucination problem occurs when AI generates inaccurate content, that content gets published, and then AI search engines cite the published content as an authoritative source. The fabricated information appears corroborated because multiple sources (all derived from the same AI output) agree, creating a self-reinforcing cycle of error.
How does AI-generated content in books affect AI search results?
AI search engines cite editorial content — including books — as authoritative sources. When those sources contain AI-generated, unverified content, the errors propagate into AI search answers. Because AI engines synthesize from multiple sources, a fabricated claim appearing in several books can appear well-sourced when it is actually the same error repeated.
What is the Bigfoot Effect in AI citations?
The Bigfoot Effect, identified by Searchless in May 2026, describes the phenomenon where AI citation concentration narrows over time, with fewer distinct sources receiving citations. This means errors that enter the shrinking citation pool are amplified more broadly than they would be in a diverse citation ecosystem.
How can I audit my brand's AI citation quality?
Use the Searchless AI visibility audit tool to measure your brand's citation presence and quality across major AI search engines. The audit checks citation volatility, source reliability, and competitive positioning.
Learn more about GEO services and pricing or explore our AI visibility resources.
Top comments (0)