DEV Community

Pandadad
Pandadad

Posted on

Catching fake citations with an AI agent (and a 2-minute manual check)

#ai #mcp #research #showdev

Disclosure: I build these tools, and I used AI to help write them and this post.

ChatGPT, Claude and friends are great research assistants with one bad habit: they invent references. The fake ones look perfect: real-sounding authors, a real journal, sometimes a DOI that points to a different paper. Real papers also get retracted, and nobody re-checks a reference list after it's written.

Here's a quick way to check any reference list against Crossref, OpenAlex and Retraction Watch data, either in the browser or with one API call.

1. In the browser (no code)

  1. Open Reference Checker on Apify and sign up free; the free plan credit covers testing.
  2. Paste your bibliography (any style: APA, MLA, numbered, DOIs only).
  3. Click Start. Each reference comes back as:
    • OK: matches a real record.
    • CHECK: real, but the details differ, or it has a correction or expression of concern.
    • FIX: not found, a different paper, or retracted.

Reference Checker output

In the example above, a famous retracted paper (Wakefield 1998) and a made-up "Neural lattice" paper both come back as FIX.

2. With Python 

import os, requests
refs = """LeCun, Y., Bengio, Y., & Hinton, G. (2015). Deep learning. Nature, 521, 436-444.
Johnson, M., & Patel, R. (2021). Neural lattice priors for protein folding. Nature, 590, 112-118."""
r = requests.post(
    "https://api.apify.com/v2/acts/pandadad0214~reference-checker/run-sync-get-dataset-items",
    headers={"Authorization": f"Bearer {os.environ['APIFY_TOKEN']}"},
    json={"bibliography": refs}, timeout=300)
for row in r.json():
    print(row["status"], row.get("verdict"), "|", row["input"][:60])
Enter fullscreen mode Exit fullscreen mode

3. Let your AI agent check itself

Add the Academic Research MCP Server to Claude Desktop, Claude Code or Cursor:

{ "mcpServers": { "panda-research": {
  "url": "https://pandadad0214--panda-research-mcp.apify.actor/mcp",
  "headers": { "Authorization": "Bearer YOUR_APIFY_TOKEN" } } } }
Enter fullscreen mode Exit fullscreen mode

Then ask: "Find 5 recent papers on CRISPR off-target effects and verify every citation before answering." The agent calls search_literature and check_references on its own.

Prefer local? The open-source MCP server runs with uvx: https://github.com/pandadad0214/panda-federation-tools

Bonus workflow

All data comes from free, official scholarly sources. Questions or feature requests are welcome in the comments.

Top comments (0)