Ask an AI model for sources and it will give you sources. Authors, year, journal, volume, pages — the full costume of scholarship. The trouble is that the costume is exactly what the model is best at producing. A citation can be 90% real and still be wrong in the one place that matters, and nothing in its formatting tells you which 10% to distrust.
The usual advice — "always check the sources" — is correct and useless, because nobody checks four references by hand every time. What works is a triage step: a prompt that tells you which citation to check, which detail in it is shaky, and the exact query that settles it in under a minute. Below is that prompt, a test run scored against the real bibliographic records, and an honest account of what it missed.
What does a fake ChatGPT citation actually look like?
Rarely a paper that doesn't exist at all. The common shapes, roughly from easiest to hardest to spot:
- Invented paper — plausible authors, plausible journal, no such article. The classic, and the one people already know to look for.
- Real paper, rewritten title — the authors, year, journal, and page numbers are all correct; the title has been paraphrased into something the paper never said. Passes a glance, fails a copy-paste search.
- Real paper, wrong job — the citation is accurate, but the text uses it for a claim the study didn't test.
- Real paper, stale finding — accurate citation, accurate summary, and the result has since failed to replicate.
The last three are the dangerous ones, because every surface check — "does this author exist, does this journal exist" — comes back clean.
The copy-paste citation-check prompt
Paste it into a fresh chat — not the one that produced the answer, so the model has no stake in defending its own references. It works the same in ChatGPT, Claude, or Gemini:
Audit every source cited in the text below. Do not assume any of them exist.
For each citation:
1. Copy it exactly as written.
2. Rate it: LIKELY REAL / UNSURE / LIKELY FABRICATED, with one line why.
3. Name the one detail most likely to be wrong (author, year, title, venue,
or the finding it is cited for).
4. Does the text use it for something the source actually studies? (yes/no/unsure + why)
5. Give the exact query a human should paste into Google Scholar to settle it.
Then list any claims in the text that sound sourced but cite nothing.
Text: [PASTE THE AI ANSWER WITH ITS SOURCES]
Step 4 is the one most citation checkers skip, and it's where a lot of the damage lives. A real paper used for the wrong claim is more persuasive than a fake one, because it survives the existence check.
What happens on a real AI answer?
I didn't plant errors this time. I asked a model, with no web access, a normal question: what does the research say about whether the Pomodoro technique improves focus and productivity — cite 4 academic sources. It returned a careful answer that admitted direct research on the technique is limited, plus four references. Then I ran the citation check on that answer in a fresh context (one pass, Claude Sonnet), and separately verified all four references against their publisher records and DOIs.
Ground truth first:
| # | Citation as given | Reality |
|---|---|---|
| 1 | Ariga & Lleras (2011), Cognition 118(3) | Real paper, correct authors, journal, and pages. Subtitle rewritten — the real one ends "…task goals preempt vigilance decrements", not "…mediate the effects of switching on vigilance". |
| 2 | Trougakos et al. (2014), Academy of Management Journal 57(2) | Real paper, correct metadata. Title rewritten — given as "Lunch breaks unpack: Individual choices influence recovery effects and end-of-day well-being"; the real one is "Lunch Breaks Unpacked: The Role of Autonomy as a Moderator of Recovery during Lunch". |
| 3 | Zacher, Brailsford & Parker (2014), Journal of Vocational Behavior 85(3) | Correct, word for word. |
| 4 | Baumeister et al. (1998), Journal of Personality and Social Psychology 74(5) | Correct, word for word. The effect it reports did not replicate in a 23-lab preregistered study (Hagger et al., 2016). |
No invented papers. Two of four titles quietly rewritten, with every number around them correct. That is what an honest-looking AI bibliography tends to look like, and it's why "the DOI-style details check out" proves very little.
What the checker caught:
- #2 flagged UNSURE, title named as the weak detail. It said the title did not match its recollection and that the real paper centers on autonomy as a moderator of recovery — which is exactly right.
- #2's misuse, too. The text used a study of the midday lunch break as evidence for Pomodoro-style short breaks. The checker called that "a scope mismatch even if the citation itself is real."
- #4's stale finding. Citation rated LIKELY REAL (correct), but step 4 came back "unsure/partially no", and the checker noted the 1998 study did not test whether breaks restore performance, and flagged the effect's "serious replication failures." The original answer mentioned neither.
- The unsourced summary. It singled out the closing "the consensus is…" sentence as the strongest claim in the passage with no citation behind it.
What did it miss?
#1. It rated Ariga & Lleras LIKELY REAL and guessed the page range was the detail most likely to drift. The pages were right; the subtitle was invented. The checker recognized the paper and never questioned the title wording.
That miss is the whole lesson. A model auditing citations from memory has the same memory that produced them. When it knows a paper well, it confirms the paper exists — and a rewritten subtitle sails through with it. The prompt narrowed four citations to one confirmed problem, one misuse, and one outdated finding; it did not make the fourth check unnecessary.
How do you settle the flagged ones fast?
The prompt's real value is step 5: a ready-made query for each source. The routine that caught the miss above takes about a minute per citation:
- Paste the title in quotes into Google Scholar. Not the authors — the exact title. A rewritten title returns nothing, or returns the real paper with a visibly different title. That single search would have caught #1 and #2.
- Open the publisher page, not a summary. Compare the title and the abstract with the claim the text makes. This is where "wrong job" citations show up.
- Search the finding plus "replication". For anything from psychology or nutrition before about 2015, this takes ten seconds and occasionally changes everything.
- Fix by replacing, never by softening. If a citation fails, swap in the real one or delete the claim. Rewording a bad citation into "research suggests…" launders it.
Can't I just tell the AI "only cite real sources"?
You can, and it will sincerely try. The four references above came from a model that hedged its answer and admitted the direct research was thin, and it still rewrote two titles. It wasn't lying; it was reconstructing from memory, and reconstruction is lossy in precisely the places that look authoritative. An instruction can't give a model access to a record it doesn't have. A separate audit pass, plus one exact-title search per flagged source, can.
Written with AI assistance. The test answer, the audit pass, and the ground-truth check were run as described — no web access for the first two, publisher records and DOIs for the third — and reported faithfully, including the miss. The input answer is summarized above; the two rewritten titles are quoted exactly as the model produced them.
This is part of the Verify AI Output series, alongside catching AI hallucinations and fact-checking ChatGPT. I keep my research prompts — literature review with a no-invented-citations rule, source compilation, fact verification — in one Notion workspace; if you want that library already organized, there's a free preview of the AI-Augmented Notion Workspace to try first.
Top comments (0)