Designing retrieval evidence, stable page anchors and verifiable answers around a practical request flow.
A document assistant’s most useful output is often the small page reference beside a sentence. That reference is where a reader can inspect the evidence, resolve an ambiguity and decide whether to act. If it opens the wrong version, lands several pages away or supports only half the claim, fluent prose does little to repair the interface.
This article was drafted with AI assistance, then fact-checked and edited by the developer behind DocBento.
For an experienced software builder, this changes the design problem. The answer is one representation of an evidence trail. Retrieval, document reading, answer generation and page navigation must agree on what that trail contains. The architecture below is a proposed design, using an illustrative request about an approval deadline; it does not describe verified internals of any particular application.
Begin with a claim the reader can check
Consider this request: “When does the supplier renewal need approval, and who can approve it?” A useful answer needs at least two claims: a deadline and an authorized approver. Those claims may come from different pages, and they may have different levels of support.
The request flow begins by establishing the authorized search scope, retrieving candidate passages and reading the relevant document content. Only then should the assistant decide what it can say. A search hit is a pointer to possible evidence. A passage that contains “approval” and “renewal” might describe an unrelated process, an obsolete policy or a historical example.
I would make evidence classification explicit before composing the answer. For each requested claim, use four states:
- Sufficient: readable source material directly supports the claim within the requested scope.
- Partial: the material supports some of the claim, but leaves a necessary detail unresolved.
- Conflicting: relevant sources disagree and the available context does not resolve the disagreement.
- No evidence: a completed search and reading process found no usable support within the authorized scope.
These states are branches, not stages. A claim does not advance from partial to conflicting on its way to sufficient. The deadline might be sufficient while the approver remains partial. Classifying the entire answer with one status would hide that distinction.
Sufficient evidence leads to a supported answer. The other three states lead to an answer with limits: identify the supported portion, expose the disagreement or say that the requested fact could not be established. Page verification is available when there is readable cited evidence. An answer with no evidence has no supporting page to open.
Retrieval should preserve evidence identity
Hybrid retrieval can combine exact terms with semantic similarity. Exact matching helps with identifiers, names and distinctive clauses; semantic matching helps when the question and document use different language. Neither method establishes that a passage supports a claim. Ranking produces candidates for inspection.
Preserve identity through that inspection. At minimum, an evidence record should identify the document, its version, the physical page position and the relevant passage. A chunk identifier helps debugging, but it should not be the reader’s only route to the source. Rebuilding an index may change chunks while leaving the underlying document unchanged.
Here is a proposed interface schema, expressed as JSON, for one evidence record:
{
"evidence_id": "ev_17",
"document_id": "doc_42",
"version_id": "ver_3",
"page_index": 6,
"page_label": "5",
"span_text": "Renewals require approval by the procurement lead.",
"claim_ids": ["authorized_approver"],
"support": "direct"
}
The values and passage are illustrative. page_index is zero-based, so it identifies the seventh physical page. page_label records the printed label, which can differ because of cover sheets or front matter. claim_ids connects evidence to the particular assertion it supports. Here, the passage supports an approver claim but says nothing about the deadline.
This record is a contract between retrieval, generation and navigation. It is not proof by itself. The application still needs to validate that the version exists, the page position is valid and the passage matches the source representation used for reading. If span_text comes from OCR, preserve that provenance elsewhere in the record or its associated extraction metadata.
An evidence identifier also gives the answer generator a constrained reference vocabulary. The model can cite ev_17; application code resolves it into a navigable reference. Reject unknown evidence identifiers instead of allowing generated document names and page numbers to become authoritative links.
A page anchor must survive the next document version
A citation such as “Policy, page 5” is readable but underspecified. Which policy? Which version? Does “5” refer to the fifth page in the viewer or the number printed on the page?
Use a version-specific target with an explicit physical page position. The visible label can remain friendly: “Renewal policy · version 3 · printed page 5.” The viewer should navigate using the physical position. If a newer version becomes available, the old answer should continue pointing to the version it actually used.
There is a retention tradeoff here. Keeping cited versions makes historical answers easier to inspect, but consumes storage and may conflict with a document deletion policy. When a cited version is no longer available, show that condition explicitly. Quietly redirecting to the latest version replaces the evidence beneath an unchanged claim.
A text highlight improves navigation when it is reliable. For scans, bounding boxes can help readers locate the relevant line. But coordinates depend on a rendering convention: page rotation, cropping and scale must agree between extraction and preview. A misleading highlight is worse than opening the correct page without one.
Access control remains part of resolution. Check permission when retrieving evidence and again when opening a citation. A reader’s access can change after an answer is created. If access is revoked, the citation should report that the source is unavailable to that reader; it should not reveal a passage through a tooltip or cached preview.
Compose the answer around the evidence boundary
Once the assistant has read candidate sources, answer generation should preserve the distinction between source text and interpretation. “The document names the procurement lead as approver” is a different assertion from “the procurement lead is available to approve this renewal.” The second needs additional evidence.
Attach citations close to the claims they support. A single reference at the end of a paragraph containing a deadline, an approver and an exception makes verification unnecessarily difficult. Splitting the paragraph into two sentences may be the most effective interface improvement available.
For partial evidence, state the established fact and the missing fact separately. For conflicting evidence, cite both sources and explain the disagreement in ordinary language. Do not turn retrieval rank into a policy precedence rule. A newer upload timestamp, for example, does not necessarily establish which document governs.
The following trace illustrates a conflict in the deadline claim. It is a hypothetical workflow, not a reported incident.
If the application has a documented rule for resolving precedence, apply it explicitly and show the reason. Otherwise, retain the conflicting state. The assistant can still provide value by locating the exact disagreement and reducing the reader’s search work.
Treat verification failures as interface outcomes
Clicking a citation is part of the request flow. It should open the cited version at the expected page, with enough document context to assess the passage. The reader may need the heading, neighboring paragraph or footnote to understand a condition omitted from the extracted text.
Several failures require distinct treatment. An extraction problem may leave a readable scanned page with unusable text. A source may disappear after retrieval. A viewer may fail to load. None of these proves that the document lacks the requested fact.
Keep execution failures separate from the four evidence states. “No evidence” means the intended search and reading work completed without usable support. A search timeout means that work did not complete. Likewise, a citation that cannot be opened is a verification failure, even if the earlier answer had sufficient evidence when generated.
This distinction should reach the reader. “The search could not complete” invites a retry. “The cited version is unavailable” explains why inspection failed. “No supporting passage was found in the searched documents” describes a bounded evidence result. Collapsing all three into “I don’t know” hides the action that could resolve the problem.
Tests should exercise these boundaries. Check that an approver passage cannot support a deadline claim, that printed page labels resolve to the right physical pages and that a version replacement leaves earlier citations intact. Include permission changes and missing versions. These checks assess whether the reader can inspect the promised evidence; they do not require an invented universal score for answer quality.
A concrete implementation: DocBento
DocBento is my own self-hosted document management application, built with FastAPI and React and using PostgreSQL with pgvector or MariaDB. Its hybrid keyword and semantic search returns filters, snippets and page numbers. Its optional assistant searches and reads documents before answering and provides page citations.
Document versions and previews are relevant to source inspection, while roles, groups and restricted folders define access boundaries. Those are verified product capabilities. The proposed evidence schema, per-claim state model and version-specific citation contract in this article are architectural recommendations; the listed capabilities do not establish that DocBento implements every detail of that design.
For a document assistant, the useful acceptance question is concrete: can a reader follow this claim to the evidence that justified it, understand its limits and recognize when verification fails? That question belongs in the interface specification alongside the answer endpoint.
Top comments (0)