Every team that deploys an AI agent eventually has the same conversation:
"It told a customer we open at 8. We open at 9."
The instinct is to blame the model, or to go shopping for a better one. In practice, that is almost never where the problem is. There are three causes, and you can tell them apart in about ten minutes.
Cause 1: the knowledge was never bound
An agent that has been configured but not connected to your documents will still answer. It answers from general knowledge, which sounds confident and is frequently wrong about anything specific to your business.
The tell: it is wrong about facts you never wrote down anywhere. If nobody has ever written "we open at 9", the agent has nothing to answer from and no reason to stay silent.
This is the most common cause and the cheapest to fix, which is also why it gets missed — the system works well enough in a demo that nobody checks whether it was ever given the corpus.
How to test it: ask a question you know the answer to, from a document you know exists. If the agent hedges or gives a plausible-but-wrong answer, the knowledge is not bound.
Cause 2: the question was never covered
Binding a corpus is necessary, not sufficient. The agent answers from what is in the corpus, and your business has facts that live outside it: the thing the front desk knows, the exception the manager handles, the policy that only exists in a conversation.
The tell: it is right about most questions and wrong about a specific class of question — usually the ones involving edge cases, exceptions, or anything that changed recently.
This is the cause people misdiagnose as "the model is not smart enough", because the failure looks like a comprehension problem. It is a coverage problem.
How to test it: ask something that is plausibly asked but is not in your documents. The correct behaviour is not a guess — it is an escalation or an honest "I do not have that, let me get someone". If it invents an answer, you have a coverage gap and no safety net.
Cause 3: the content went stale
Your knowledge base is a snapshot. Opening hours change for a holiday. Prices change. A policy is updated in the document but not in the corpus.
The tell: it is confidently wrong about something that used to be true. That is the signature of stale content, and it is worse than having no agent, because the answer sounds authoritative and is now false.
Stale content is not a technical failure. It is a maintenance failure, and it is the one most likely to be blamed on the vendor.
How to test it: pick the three facts most likely to change in your business — a price, an hour, a policy — and check them against what the agent says.
The fourth cause people forget: overlapping documents
If you have a current price list and an old one, and both are in the corpus, the agent has no way to prefer the new one. It will retrieve whichever matches better, which is effectively random from your point of view.
The fix is operational, not technical: retire old documents instead of leaving them next to new ones, and label which is authoritative. A knowledge base with contradictions in it will produce contradictory answers, and no model can resolve an ambiguity the business has not resolved.
A diagnostic order that saves time
When an answer is wrong, check in this order:
1. Is the fact in the corpus at all? If not, it is a binding or coverage problem. 2. Is the fact in the corpus, but stale? Compare the document to the current truth. 3. Is there more than one version of the fact? Look for duplicates and retired documents. 4. Only then consider whether the retrieval or the model is at fault.
In most deployments I have seen, the problem dies at step 1, 2, or 3.
What to build before you go live
Three things, none of them technical:
- A written list of the twenty questions you are actually asked. Not the questions you wish people asked. If you cannot write twenty, you do not yet know your own call patterns.
- Explicit answers for each. Including the ones where the answer is "I need to check".
- A named owner and a review cadence. Weekly for prices and hours, monthly for policies, and immediately after any change.
If those three exist, the agent's quality is mostly a function of your maintenance. If they do not, no model upgrade will help.
The honest version
An AI agent is not a knowledge worker who learns your business by osmosis. It is a system that answers from the material you gave it, in the moment you ask.
That is a limitation, and it is also the reason it can be trusted: you can inspect exactly what it knows. A human receptionist who has been there two weeks also gets things wrong — you just cannot audit why.
Disclosure: I work on RingBot, an Agent-as-a-Service platform where agents answer from a knowledge base you write or upload, with version history so a bad edit can be rolled back. The diagnostic order above applies to any agent platform — including the ones that claim you do not need a knowledge base at all.
Related: proving a knowledge change did not break anything, the update process, what to do when a flow change breaks calls, and the cost comparison of the four options.
Top comments (0)