DEV Community

Cover image for How to Check Whether an AI Answer's Citations Back It Up
Mayur Agnani
Mayur Agnani

Posted on Originally published at theaicommit.com

How to Check Whether an AI Answer's Citations Back It Up

An AI answer can cite a real sentence from your own document and still get the fact wrong. You can check that the sentence exists with one string comparison. Checking that it backs the claim takes something else.

Engineer's view. This is a foreign key that points at a real row, just the wrong one. The database checks that the row exists, never that it is right. Citation APIs can guarantee the quoted passage exists in your document. None check that it backs the claim, so users see a wrong answer with a real source.

TL;DR. To verify an LLM citation, check two things separately: that the quoted passage exists in your source, and that it backs the claim. The first check can be exact. The second needs simple rules plus a judge, because counting shared words cannot tell a fair paraphrase from a quote about the wrong thing.

Explain Like I'm 5

A friend writes you a book report and puts a page number after every sentence. You can check each sentence in two ways.

First, open the book to that page. Are those words really there? That takes one look, and you are sure.

Second, read the words and ask whether they mean what your friend wrote. Maybe the page says the hero lost, and the report says the hero won. That check takes thinking. Words that look alike can still mean opposite things.

The first check never needs the second one. The second check can never be done with only the first.

The Problem

You have shipped this bug before, in a system that had nothing to do with AI.

Your docs site runs a link checker in CI. Any link that returns a 404 fails the build. Then a support ticket arrives. A page about rate limits links to the billing section. The link resolved fine, so the checker passed it. It only ever asked whether the target exists. It never asked whether it is the right target.

An AI answer with citations has the same blind spot, one level down. A model writes a marker like [2] the way it writes any other word: it predicts what usually comes next. So a citation can fail in three ways. It can point at a source that does not exist. It can point at a real passage that says something else. Or it can be right. The user sees the same tidy footnote in all three cases.

Citation features in model APIs close the first gap by different amounts. The strongest return the exact passage, cut from your own document. None of them check that the passage backs the claim.

The obvious cheap fix makes it worse. Teams score word overlap between the claim and the quote, and pass anything above a threshold. In the test built for this article, that rule passed 5 of 7 bad citations. It also rejected 3 of 9 good ones.

The fix: check the pointer exactly, check numbers and negations by rule, and send every other citation to a judge.

Read the full post on The AI Commit

How to Check Whether an AI Answer's Citations Back It Up carries the rest: the fix and why it works, the code to change for each role, how to tell it worked, when it is the wrong tool, and an interactive model you can play with.


Originally published at theaicommit.com, a daily AI engineering lab for software engineers. The original has an interactive version you can play with, and the full runnable code is on GitHub.

Top comments (1)

Collapse
 
__d03cc26214f3 profile image
איתמר וענונו •

The foreign-key analogy is spot on. One cheap middle step before the judge: ask the model to output the claim and the quote as a pair, and reject any pair where the claim has a number, date or name that doesn't appear in the quote. That catches a lot of "real source, wrong fact" cases with plain string checks, and leaves the judge only the paraphrase cases. Do you run the judge on every answer, or only on answers that fail the cheap checks?