DEV Community

Cover image for Your RAG Agent Cites Sources. Check That the Quotes Are Real.
Satyam Yadav
Satyam Yadav

Posted on Edited on

Your RAG Agent Cites Sources. Check That the Quotes Are Real.

A small Python quote check that catches made-up citations before a user sees them, and the one trap I hit while building it.

A citation makes an answer look trustworthy. But a model can attach a real document ID to a sentence that is not in that document. Or it can quote the document almost perfectly, with one number changed. Both look fine in the UI.

When I build agents that answer from documents, I don't trust a citation until code has checked it. Here is the small check I use, and a bug I found in my own first version while writing this post.

Ask for quotes, not just document IDs

A citation that only says "source: policy-7" cannot be checked. You would need another model call to decide whether the claim is supported. So I ask the model to return three things for every claim: the claim, a verbatim quote, and the source ID.

Most LLM frameworks can fill a Pydantic model through structured output (in LangChain it is with_structured_output). That gives me data I can test with plain code.

import re
import unicodedata

from pydantic import BaseModel
from rapidfuzz import fuzz

class Citation(BaseModel):
    claim: str
    quote: str
    source_id: str

def normalize(text: str) -> str:
    text = unicodedata.normalize("NFKC", text)
    text = text.replace("’", "'").replace("‘", "'")
    text = text.replace("“", '"').replace("”", '"')
    text = re.sub(r"\s+", " ", text)
    return text.strip().lower()
Enter fullscreen mode Exit fullscreen mode

The normalize step matters more than it looks. Models swap straight and curly quotes, collapse line breaks and change case. PDF extraction adds odd spacing. Without normalizing both sides, honest quotes fail.

Exact match first, fuzzy match second

The cheapest check is a substring test. If the normalized quote is inside the normalized source, we are done.

When it is not, I fall back to RapidFuzz. Its partial_ratio_alignment finds the best place for the shorter string inside the longer one and returns a score from 0 to 100, plus the start and end of the matched span. That span is useful: you can show it to a reviewer or highlight it in the UI.

NUM = re.compile(r"\d+(?:[.,]\d+)*")

def check_quote(cit, sources, min_score=95.0, min_words=5):
    doc = sources.get(cit.source_id)
    if doc is None:
        return {"ok": False, "reason": "unknown source"}
    quote, text = normalize(cit.quote), normalize(doc)
    if len(quote.split()) < min_words:
        return {"ok": False, "reason": "quote too short"}
    if quote in text:
        return {"ok": True, "score": 100.0}
    m = fuzz.partial_ratio_alignment(quote, text)
    if m.score < min_score:
        return {"ok": False, "reason": "not found", "score": m.score}
    span = text[m.dest_start:m.dest_end]
    if NUM.findall(quote) != NUM.findall(span):
        return {"ok": False, "reason": "numbers differ", "span": span}
    return {"ok": True, "score": m.score, "matched": span}
Enter fullscreen mode Exit fullscreen mode

Two small guards are in there. min_words stops tiny quotes like "30 days" that would match almost anywhere. And an unknown source_id fails straight away, because models do invent IDs.

One detail from the RapidFuzz docs: for a shorter string up to 64 characters it is guaranteed to find the best alignment. For longer strings it uses a faster method that may not find the optimal one. That is another reason to ask the model for short, focused quotes.

The trap: fuzzy matching forgives wrong numbers

My first version did not have the NUM line. I tested it with a source that says refunds are available "within 30 days" and a quote that says "within 60 days". It passed. The score was about 97.9, comfortably above my threshold of 95.

That makes sense once you think about it. One changed character in a 50-character sentence barely moves a similarity score. But a changed number is exactly the error that hurts most: a wrong amount, a wrong date, a wrong limit.

The fix is simple. Pull the numbers out of the quote and out of the matched span, and require them to be equal. Here is the demo after the fix:

sources = {
    "policy-7": (
        "Refunds are available within 30 days of purchase.\n"
        "After 30 days, customers may request store credit."
    )
}
wrong = Citation(
    claim="Refunds within 60 days",
    quote="Refunds are available within 60 days of purchase",
    source_id="policy-7",
)
typo = Citation(
    claim="Refunds within 30 days",
    quote="Refunds are availble within 30 days of purchase",
    source_id="policy-7",
)
print(check_quote(wrong, sources)["reason"])  # numbers differ
print(check_quote(typo, sources)["ok"])  # True
Enter fullscreen mode Exit fullscreen mode

The harmless typo still passes. The wrong number now fails. If your documents use other critical tokens, such as product codes or names of parties, the same idea applies: extract them and compare them exactly.

Make it a gate, not a log line

A check is only useful if something happens when it fails. I run it over every citation in an answer:

def verify_answer(citations, sources):
    results = [check_quote(c, sources) for c in citations]
    failed = [
        (c, r) for c, r in zip(citations, results) if not r["ok"]
    ]
    return {"passed": not failed, "failed": failed}
Enter fullscreen mode Exit fullscreen mode

In an agent graph (LangGraph or anything similar), this becomes a node with a conditional edge. If it passes, the answer moves on. If it fails, I send the failed quotes back to the model once, with the reason, and ask it to fix or drop those claims. If it fails again, the answer goes to a human reviewer instead of the user. Don't loop forever; one retry is usually enough to tell a sloppy quote from an unsupported claim.

This check does not prove the claim is true. It proves the quote exists. You still need the claim to follow from the quote, which is a harder problem. But it removes the most embarrassing failure, a citation pointing at text that is not there, for almost no cost.

Takeaway

Ask for verbatim quotes, normalize both sides, try an exact match before a fuzzy one, and compare numbers exactly. Then treat a failed check as a reason to retry or escalate, not just something to log.

Sources


Let's connect

I write about AI agents, RAG and .NET every week. If this helped, let's connect:

Follow me here for the next post.

Top comments (0)