The expensive part of a healthcare appointment assistant is not generating a fluent reply. It is proving which scheduling record justified that reply, while keeping the index affordable as the record set grows. My default design is a two-stage retriever with immutable evidence IDs, a visible citation map, and a hard rule that the model may only use returned passages. That shape also works for a logistics team detecting semantically near-duplicate delivery records, where every extra indexed field has a carrying cost.
Short answer: store a stable source ID beside every chunk, retrieve candidates with metadata filters plus semantic search, rerank a small set, and return citations that resolve to the original record and revision. If the trace cannot be replayed, the answer is not production-ready.
The incident lesson: a match is not an explanation
I've been paged for missed cron jobs and duplicate queue deliveries. The recurring failure was not a dramatic outage; it was an answer that looked plausible after the underlying row had changed. A patient saw an available slot, the assistant cited a vector result, and an operator could not tell whether the result came from yesterday's schedule or the current revision. The same shape appears in logistics when two nearly identical delivery notes get merged and the wrong one is treated as canonical.
The invariant is simple: retrieval metadata is part of the answer. Keep source_id, revision, ingested_at, document_hash, and the authorization scope with the text. A citation should point to those fields, not to a transient rank in an approximate nearest-neighbor index. Replaying a query then means rebuilding the candidate set from a known snapshot, recording the scoring inputs, and checking that the cited revision was eligible for that user.
Three design choices are easy to confuse:
| Choice | Strength | Cost or risk |
|---|---|---|
| Keyword or inverted index | Predictable filters and cheap exact matching | Misses paraphrases such as “move my visit” |
| Dense vector index | Finds semantic variants and near-duplicates | Memory, rebuild, and drift costs rise with corpus size |
| Hybrid retrieval with reranking | Uses exact constraints before semantic similarity | More moving parts and latency to observe |
For appointment workflows, date, clinic, clinician, and patient scope are constraints, not hints. Apply them before semantic ranking. A high cosine score must never outrank an authorization filter.
How should a healthcare appointment assistant trace retrieval evidence?
Make the trace an explicit data structure. One record can produce several chunks, so a citation must identify both the parent and the chunk boundaries. Store a hash of the normalized text; it lets an auditor detect silent edits without exposing unnecessary protected health information in logs. Log query intent, filter values, retriever version, candidate IDs, reranker scores, selected IDs, and the model response ID. Keep the content itself behind the same access controls as the source system.
The response contract can stay small:
package trace
type Evidence struct {
SourceID string `json:"source_id"`
Revision string `json:"revision"`
Chunk string `json:"chunk"`
TextHash string `json:"text_hash"`
Score float64 `json:"score"`
PolicyScope string `json:"policy_scope"`
}
type Answer struct {
Text string `json:"text"`
Citations []Evidence `json:"citations"`
}
Before generation, reject any candidate whose scope, revision, or freshness policy fails. After generation, verify that every citation ID appears in the selected set and that every material claim has at least one citation. This post-check is a guardrail, not a substitute for good retrieval. Your mileage may vary on the exact claim granularity; test it with real appointment intents and with adversarial near-duplicates.
Keeping index cost under control at scale
Index cost is a systems decision, not a model preference. Start by measuring bytes per source record, vector dimensionality, replica count, and rebuild frequency. In the logistics duplicate-detection job, I would index the normalized description and a compact set of routing attributes, while retaining the full payload in object storage or the transactional database. That avoids paying vector storage for fields that filters can handle exactly.
Use a coarse candidate budget. Retrieve perhaps 50 IDs, rerank 10, and cite only the passages that survive policy checks. The numbers are tuning knobs, not universal facts; record them in configuration so an evaluation run can reproduce them. Batch ingestion, tombstone deletions, and periodic compaction keep stale revisions from quietly inflating the index.
A useful cost report has four lines: storage per million chunks, ingestion CPU, p95 retrieval latency, and human review rate for incorrect citations. If a smaller index increases review work, its apparent savings are fictional. If a larger index adds recall but causes timeout retries, the queue will turn that gain into duplicate work.
Failure modes and the runbook response
The first failure mode is stale evidence. Alert when a citation revision is older than the source's freshness window, then quarantine the answer rather than silently substituting a newer row. The second is duplicate delivery from an at-least-once ingestion queue. Make the ingestion key a deterministic pair of source_id and revision; acknowledge the message only after the write and its idempotency record commit.
The third is a retrieval-policy mismatch: a tenant filter is present in the application but absent from the index query. Treat that as a test failure. Generate fixtures containing identical appointment text under two scopes and assert that only the authorized citation can be selected. Keep a replayable sample of every blocked response so an on-call engineer can inspect the decision without guessing.
A fourth failure is citation drift after chunking changes. Version the chunker and embedding configuration, and retain enough metadata to explain why an old answer differs from a new one. I am not sure one universal similarity threshold exists across specialties; calibration data from your own appointment language is what resolves that uncertainty.
When this architecture is not the right fit
The catch is operational weight. A full hybrid stack is not suitable when the corpus is a few thousand stable, hand-curated entries and a relational query can return the exact answer with a simpler audit trail. In that case, stick with the transactional search plus deterministic templates. Choose a vector stage when paraphrase recall or near-duplicate detection is a measured problem, and choose a separate policy engine when authorization rules outgrow index filters.
Do not rank systems by a single recall number. Compare replayability, deletion behavior, access-control tests, p95 latency, index bytes, and operator effort. Run the same corpus through each candidate, inspect false citations, and keep the decision record with the configuration used. That is the part that survives a page at 03:00.
Sources
The retrieval-augmented generation architecture and its grounding rationale are described in the original research paper:
Top comments (0)