DEV Community

yandong
yandong

Posted on Fully Autonomous

What an MCP Citation Checker Can Verify—and What It Cannot

A valid DOI proves that a record can be resolved. It does not prove that the paper supports the sentence in your manuscript. Research agents need to distinguish source existence, bibliographic identity, relevance and claim support before treating a citation as verified.

Disclosure: I am the builder of LitSource. This is a small, reproducible demonstration of its reference-verification MCP tool, including a deliberately incorrect input. It is not a benchmark against other products.

Four separate questions

  1. Existence: can a scholarly record be found?
  2. Identity: do the title, DOI, authors and year describe the same record?
  3. Relevance: is the study about the population, intervention and outcome in your sentence?
  4. Support: does the relevant passage actually justify that sentence, including its scope and uncertainty?

A metadata checker addresses the first two questions. Reading the source is still required for the last two. MCP makes tools available to an agent; it does not make every tool result a scientific conclusion.

The real run

On October 12, 2026, I submitted two references in one call to litsource_verify_references. The first is a real SELECT trial citation. The second repeats its bibliographic details but deliberately substitutes a different DOI. This seeded error was created for the demonstration; it was not found in a user's manuscript.

{
  "references": [
    "Lincoff AM et al. Semaglutide and Cardiovascular Outcomes in Obesity without Diabetes. N Engl J Med. 2023;389:2221-2232. doi:10.1056/NEJMoa2307563.",
    "Lincoff AM et al. Semaglutide and Cardiovascular Outcomes in Obesity without Diabetes. N Engl J Med. 2023;389:2221-2232. doi:10.1038/171737a0."
  ],
  "request_id": "litsource-dev-citation-demo-20261012-v1"
}
Enter fullscreen mode Exit fullscreen mode

Use your own new request ID for a new run. Reuse an ID only when retrying the exact same request. A fresh call can consume credits; this two-reference run charged 3 credits. Setup instructions are in the LitSource MCP guide.

The actual response, reduced to the decision fields, was:

[
  {
    "seq": 1,
    "decision": "verified",
    "mismatchFields": [],
    "evidenceSources": ["crossref"],
    "matchedDoi": "10.1056/nejmoa2307563"
  },
  {
    "seq": 2,
    "decision": "suspicious",
    "mismatchFields": ["doi"],
    "evidenceSources": ["openalex"],
    "matchedDoi": "10.1056/nejmoa2307563",
    "matchedPmid": "37952131"
  }
]
Enter fullscreen mode Exit fullscreen mode

For the seeded error, fallback title retrieval found a candidate with the correct DOI. That recovery does not make the input correct. An agent should retain both the supplied identifier and the candidate identifier, flag the conflict, and request source review before replacing it.

This run also illustrates a coverage limit: some candidate fields were null in the OpenAlex result. A strong title match is not a guarantee that every bibliographic field was checked. The response reported publication status as unknown; it should not be relabeled “not retracted.”

Source review and the final decision

I opened the PubMed record, PMID 37952131. It independently shows the SELECT title, publication year, journal, pages and DOI. The abstract describes participants aged at least 45, with existing cardiovascular disease and BMI at least 27, without a history of diabetes.

That matters when checking scope. A sentence claiming that this trial establishes the same benefit in every adult with obesity would go beyond the population described in the abstract. The correct DOI would still be a correct DOI.

The final decisions were therefore:

  • Reference 1: bibliographic identity accepted for this demonstration; no blanket claim-support approval.
  • Reference 2: reject the supplied DOI association; confirm the intended source before correcting it to the SELECT DOI.
  • A broader manuscript claim: leave it unapproved until the source and the exact wording have been reviewed.

The source check here used the public metadata and abstract, not a complete appraisal of the full paper or all supplementary materials. A researcher must review the exact manuscript claim before approving it.

A practical agent contract

Keep the submitted citation, retrieved record, provider, mismatch fields and human decision together. Preserve unknown and unresolved states. Treat scores as diagnostic signals rather than scientific truth probabilities. Do not silently replace a citation or transform verified into “the manuscript claim is supported.”

When a tool reports a DOI conflict, the useful output is a reviewable discrepancy with a source link. That is a better stopping point than a confident-looking bibliography with no audit trail.

AI disclosure: This article was prepared with AI assistance. The live tool response and linked public records were inspected for this example; no claim of independent expert review is made.

Top comments (0)