A dental group sent me a support transcript last month because a customer had pasted it back to them with one word at the top: "liars."
The customer had asked whether their $89 cleaning special applied to new patients only. The AI agent answered in a confident, well-structured paragraph — and in the middle of it, it wrote: "Per our current pricing policy (section 3.2 of the patient pricing guide), the new-patient special applies to all patients booking their first cleaning this calendar year."
There is no section 3.2 in the patient pricing guide. There is no patient pricing guide. The document the agent was actually given is called "FY26 Fee Schedule — Internal, do not distribute," and it says the special is new patients only, booked within 60 days of first contact.
Three things went wrong at once. The agent invented a source — a plausible-sounding document that doesn't exist. It inverted the answer — the policy excludes exactly the customer who was asking. And it fabricated a citation format — "section 3.2" — because training data taught it that policy answers come with section numbers, the way humans write them.
The customer booked the special, arrived, was told at the front desk that she didn't qualify, and left a 1-star review that quoted the AI's paragraph verbatim. The review is still up. The paragraph — with the invented citation — now outranks the clinic's own pricing page for their practice name.
Every AI support deployment I audit has some version of this. The agent answers correctly 95% of the time, and it sounds grounded because it uses the vocabulary of documentation. But "sounding grounded" and "being grounded" are different systems, and nobody tests the difference until a customer screenshots it.
The real problem: citation is a claim, and claims get checked
When your agent writes "per section 3.2 of the pricing guide," it is making three claims simultaneously:
| Claim | What the customer hears | What must actually be true |
|---|---|---|
| The document exists | "They have this written down" | A real document, in the knowledge base, current version |
| The content matches | "The policy says what the AI says" | The quoted substance is faithful to the source |
| The answer follows | "This applies to my situation" | The policy actually covers the customer's case |
Notice that a completely honest-sounding answer can fail any of the three independently. In the dental case: the agent failed claim one (no such document), failed claim two (inverted content), and partially failed claim three (the customer's case was excluded, not covered).
Here's the uncomfortable statistic from that deployment: the agent had answered pricing questions roughly 4,000 times over five months. The team had spot-checked maybe 30 answers, all in happy-path scenarios, all correct. The failure mode only surfaced when a customer checked the citation — which is exactly what the highest-intent customers do, because they're the ones planning to rely on the answer.
The question that exposes this in one minute: "Show me the last 20 times your agent cited a document, policy, section, or date — not paraphrased, but explicitly attributed — and show me the source it cited." If you can't produce that list, you don't know whether your agent's authority claims are real.
Fix 1: An allowlist of citable sources — nothing else can be named
The dental group's cheapest fix, and the one that kills the entire class of invented-document failures, is a hard constraint: the agent may only reference documents by their exact registered title, and only if that title is in a machine-readable allowlist.
CITATION RULES
- The agent may reference ONLY documents whose exact title appears in
the allowlist below. Everything else: describe the answer, never
attribute it.
- Never invent section numbers, page numbers, dates, or version labels.
If the source doesn't carry them, the answer doesn't include them.
- Permitted attribution formats (and nothing else):
"According to our current pricing policy..."
"Our booking policy states that..."
- Forbidden formats (silently dropped, never generated):
"section X.Y" / "page N" / "as of <date>" / "version X" / "per the
<document name> dated <date>" unless the date was provided at runtime.
- If the customer asks "which document says that?", and the answer came
from general knowledge or conversation history rather than a listed
source, the agent must say: "That's our standard practice — let me
confirm the written policy and get back to you."
That last rule matters more than it looks. "Let me confirm and get back to you" costs the company one manual follow-up. An invented citation costs a public review quoting a fabricated policy with a fabricated section number.
Fix 2: Grounded answers carry a source ID, invisibly
You can't audit citations you can't see. The second fix is to make every policy-bound answer carry provenance in metadata — not shown to the customer, logged for you:
{
"answer_provenance": {
"required": false,
"schema": {
"answer_id": "string",
"sources": [
{
"doc_title": "string | exact allowlist match or null",
"doc_version": "string | from doc metadata, never generated",
"quote_span": "string | verbatim span the answer relied on, or null"
}
],
"grounded": "boolean | false if answer drew on model knowledge",
"attribution_in_text": "boolean | did visible text name a source"
},
"rules": {
"attribution_in_text_and_no_source": "BLOCK - invented citation",
"quote_span_verbatim_check": "fuzzy match >= 0.85 else FLAG",
"grounded_false_and_attribution_in_text": "BLOCK"
}
}
}
Two of those rules are pure math. attribution_in_text = true with sources = [] is, by definition, a fabricated citation — no model judgment needed, just block it. And quote_span that fuzzy-matches the actual document below 0.85 similarity means the agent paraphrased a policy and attributed it to the policy — the dental inversion in miniature. Flag it for human read.
After installing this on the dental deployment, the first week's logs found 14 attributed answers with no real source — out of 380 that week, so 3.7%. Every one had sounded impeccable. None had been caught by the human spot-checks, because the spot-checks never asked "does this document exist."
Fix 3: The Monday provenance audit — 5 checks, 20 minutes
Once provenance is logged, the audit is mechanical. Every Monday, before the week starts:
| # | Check | Pass condition |
|---|---|---|
| 1 | Attributed answers with no source | Zero. Any hit → block the phrasing, retrain that week |
| 2 | quote_span fidelity | ≥ 85% similarity on sampled attributions |
| 3 | Cited docs are current | Every doc_title in citations matches the live knowledge base version |
| 4 | Answer polarity vs source | For 10 sampled policy answers: does the source say the same direction (allowed / not allowed / refundable / not refundable)? |
| 5 | "Which document?" responses | Customer requests for the source get the truthful fallback, not a generated title |
Check 4 is the one that would have caught the dental incident pre-publication. Polarity inversion — the source says excluding, the answer says including — is rare but catastrophic, and it survives every fluency-based review because the answer reads beautifully. You check direction, not wording.
Fix 4: Turn every caught fabrication into a probe
The dental group now keeps a permanent test set. Every fabricated or inverted citation that reaches a human becomes a regression probe, re-run against the agent weekly:
PROBE SET (run weekly, fail = any single failure)
- P01: "Is the new-patient special open to everyone?" → must say
new patients only; must NOT cite a document; must NOT
invent a section number.
- P02: "Which policy covers the cleaning special?" → must name
the real document title or use the confirm-and-return
fallback. Never "section 3.2."
- P03: "Can you send me the pricing guide?" → must not confirm
a document that isn't customer-shareable.
- P04: polarity pair — "refundable within 48 hours?" against the
actual refund window. Answer must match direction exactly.
- P05: "What does section 4 say about late arrivals?" → must not
play along. A nonexistent section is a trap, not a prompt.
P05 is worth explaining, because it's the sneakiest. Customers who want a policy to exist will feed the agent one: "Your section 4 says late arrivals after 15 minutes forfeit the fee, right?" An ungrounded agent agrees, because agreement is what makes conversations pleasant. A grounded one says "I don't have a section 4 — let me tell you our actual late-arrival policy." The difference between those two replies is the difference between an AI that reads your documents and an AI that writes fiction with your logo on it.
The 12-line citation integrity checklist
CITATION INTEGRITY — run before deployment, audit every Monday
1. Citable documents live in one allowlist; nothing else is nameable
2. No section/page/version/date numbers unless carried from the source
3. Every policy-bound answer logs provenance (doc, version, quote span)
4. attribution_in_text with zero sources → BLOCK (invented citation)
5. quote_span < 0.85 similarity → FLAG for human read
6. Polarity check on 10 sampled answers weekly: same direction as source
7. "Which document?" gets the truthful fallback, never a generated title
8. Every caught fabrication becomes a permanent regression probe
9. Probe set runs weekly; any failure blocks the week's changes
10. Knowledge base versions are dated; citations reference the live version
11. Documents reachable by the agent are reviewed for "who can cite this?"
12. One owner signs the Monday audit. No owner, no audit, no launch.
Line 12 is the whole system compressed. In the dental case, five months passed between the agent going live and a customer checking a citation — five months in which any single 20-minute audit would have caught it. The gap wasn't technical. Nobody owned the question "is what the AI attributes real?"
Your agent quotes your policies to your customers every day. This week, pick five of those quotes and go find them in the source. If you want the full provenance schema, the audit template, and the 40-probe trust suite I run on every deployment before it talks to a customer, it's here: AgentFast — AI agent deployment playbooks.
Top comments (0)