An assistant that answers questions about your documents will be wrong sometimes. What destroys trust is not the wrong answer, it is the confident wrong answer to a question the documents never covered. So the first design question is not "how good is retrieval?" but "when does the system refuse?"
Here is a small demo that treats refusal as a first-class result. The data is public and the numbers are test numbers, not a client result.
What the demo does
One function, KB.answer(question), returns three things: the best passage as the citation, a confidence that it really holds the answer, and a decision: cite it, or refuse.
It deliberately returns the passage, not a generated sentence. Generating prose on top of a cited passage is the step that needs a language model; the part that decides whether to answer at all does not.
- Data: SQuAD 2.0 dev (Wikipedia, CC BY-SA 4.0), 35 articles. Wikipedia stands in for a company's documentation.
- Retrieval: BM25.
- Decision: a small model over retrieval features: the top score, its margin over the runner-up, IDF-weighted coverage of the question's terms, and the share of terms the passage doesn't know.
- Threshold: tuned on separate articles for a 95% precision target.
- No LLM, no GPU.
Evaluating without leaking
Articles are split into a tuning set and a test set. A third of the test articles are deliberately left out of the knowledge base, and their questions must be refused. Without that group you only measure how well the system answers questions it was built for, which is the easy case.
Results (test KB: 406 passages, 4,904 questions)
| Case | Result |
|---|---|
| Question about a topic NOT in the base | refused correctly in 96.1% |
| Answerable question: right passage in top-3 | 87.8% (top-1: 75.3%) |
| Answerable question: cited with confidence | 61.2% of questions, 93.2% of those cited point to the right passage |
| Answerable plus out-of-base, overall | 42.5% cited, 90.4% of cited are correct |
The refusal rate on out-of-base questions is the number I would put in an acceptance document. It says the system stays quiet when it has nothing.
Honest limits
- The target was 95% precision. On the test articles the cited answers reached 93% (answerable only) and 90% (with out-of-base questions mixed in). The threshold is a sampled estimate, and a pilot re-tunes it on the client's own documents and questions.
- Questions that are close to a document but not answered by it (SQuAD's "unanswerable" ones) are refused only 39% of the time. Lexical signals cannot see that difference. This is where a reranker or an LLM belongs in a real build, and I report the number instead of hiding it.
- Wikipedia is not your wiki. Expect to measure again on real material.
Takeaways
- Give refusal its own metric and its own test set.
- Always include questions about things that are not in the base.
- Return the citation, so a person can verify in one click.
- Report the failure mode you have not solved.
Code and numbers: ai-document-qa-demo. Portfolio and contacts: @Kotenev01_bot.
I write about practical, measurable AI adoption for business: demos on test data, no invented case studies. Telegram channel: @kotenev01_en
Top comments (0)