Use an immutable (document_id, content_digest) pair as the identity of every chunk you put in a retrieval index, keep exactly one revision of each claim document answerable at a time, and make the change from one revision to the next a single commit that either lands or doesn't. A claims intake system that skips any of those three will eventually read a superseded damage estimate back to an adjuster with total confidence, and no reranker will save you, because the stale chunk was already in the candidate set before the reranker ever ran.
That's the entire recommendation. Everything below is why the failure happens and how to land the fix without rebuilding your pipeline.
The signal that says your index is answering from a dead revision
A claim folder is not a folder. It's a slow-moving stream of corrections: first notice of loss, then the adjuster's report, then a corrected loss run, then a re-issued endorsement, then an amended medical bill that arrives six weeks after the first one and contradicts it. Same claim number, same document role, different bytes. If your ingest job treats each arriving PDF as a new document, both revisions sit in the index forever; if it treats them as the same document and overwrites by filename, you lose the audit trail that a disputed claim will eventually demand. Neither default is safe, and the second one is worse, because it fails quietly.
The symptom I'd watch for shows up as non-determinism. Ask the same question twice, get two different dollar figures, because the amended bill and the original bill embed to nearly the same vector and the ANN search returns whichever one wins by a rounding difference in the distance calculation. Chunk-level near-duplicates are exactly the case where similarity scores stop discriminating.
So instrument the thing that actually matters: for every answer you return, record whether each cited chunk belongs to the revision your control plane currently considers live. That ratio is your service level indicator. Anything below it is a compliance conversation, not a relevance-tuning exercise — an answer that quotes a withdrawn figure in an insurance workflow is a defect with a paper trail attached.
Latency, not quality, is usually what pushes teams into the broken design. Filtering costs milliseconds, so somebody drops the filter.
How should document versioning work in a claims intake retrieval architecture?
Three stores, with one authority each. An object store holds immutable blobs keyed by their content digest and never mutates anything. A control table maps (claim_id, document_role) to the digest that is currently live, plus the full ordered history of superseded digests. The vector or lexical index holds chunks tagged with both doc_id and digest, and the query path constrains retrieval to the live set.
Three patterns get you there, and they trade retrieval quality against latency differently.
| Approach | How a new revision goes live | Read-path cost | Where it hurts |
|---|---|---|---|
| Index every revision, filter at query time | Control row flips; nothing moves in the index | A predicate on a high-cardinality field; recall falls as the live fraction shrinks | Index grows with every amendment, and ANN filtering may have to widen the search to fill k
|
| Delete-and-insert per document | Old digest's chunks are removed in the same commit that adds the new ones | None | Write amplification on hot claims; a half-applied commit leaves one document unanswerable |
| Rebuild and swap the whole collection | New collection is built, then an alias is repointed | None | Rebuild cost scales with the entire corpus, not the one amended file, and storage roughly doubles during the swap |
For a B2B SaaS product answering questions over a folder of claim PDFs, the middle row is usually the right default, and the reason is capacity, not elegance. A mid-size book of business might hold a few hundred thousand documents averaging maybe 40 chunks each; letting every amendment accumulate in the hot index means the index grows monotonically while the answerable set stays flat, and you end up paying memory for vectors nobody is allowed to cite. The catch is that per-document deletes are not free everywhere — some engines tombstone and defer compaction, so a claim that gets amended eleven times during litigation will carry dead space until the segment merges.
Index aliases exist in Elasticsearch and OpenSearch, and Qdrant has collection aliases, so the third row is mostly a configuration question on those engines. Postgres with pgvector has no alias primitive, so the same atomicity comes from a transaction plus a view or table swap; pgvector 0.5.0 added HNSW indexes alongside IVFFlat, which changes rebuild cost considerably but doesn't change the swap story at all.
Keep the first row available anyway, because point-in-time retrieval is a real requirement in claims: "what did the file say on the day we denied it" is a question your legal team will ask. Serve it from cold storage and a replay job, on a latency budget measured in minutes, and keep it out of the interactive path.
Writing the ingest path so a re-issued PDF is never half-live
Derive identity from bytes. Filenames lie, upload timestamps lie, and adjusters re-send the same attachment three times when an email bounces.
package intake
import (
"crypto/sha256"
"encoding/hex"
"time"
)
// Revision identifies one immutable version of one document in one claim.
// Digest is the identity; ClaimID and Role are the business key that the
// control plane flips when a correction arrives.
type Revision struct {
ClaimID string // e.g. "CLM-2026-118842"
Role string // "adjuster_report", "loss_run", "endorsement"
Digest string // sha256 over the raw PDF bytes
Supersedes string // digest this revision replaces; empty for the first
IngestedAt time.Time
}
func NewRevision(claimID, role string, pdf []byte, supersedes string) Revision {
sum := sha256.Sum256(pdf)
return Revision{
ClaimID: claimID,
Role: role,
Digest: hex.EncodeToString(sum[:]),
Supersedes: supersedes,
IngestedAt: time.Now().UTC(),
}
}
The commit is where most implementations get sloppy. Embedding a 90-page adjuster report takes long enough that you cannot hold a database transaction open across it, so split the work: embed into a staging namespace outside the live filter, prove the new chunks answer a small set of canary questions, and only then flip the control row and delete the superseded chunks. If the process dies between the flip and the delete, the query path is still correct, since it reads the control row; a sweeper reconciles the orphans later.
// Index and Control are the two seams worth keeping vendor-neutral: every
// engine can satisfy them, and swapping one out stays a one-file change.
type Index interface {
Upsert(ctx context.Context, digest string, chunks []Chunk) error
Search(ctx context.Context, q string, f Filter, k int) ([]Hit, error)
DeleteByDigest(ctx context.Context, digest string) error
}
type Control interface {
// Promote is a compare-and-set: it fails if the current head is not `from`.
Promote(ctx context.Context, claimID, role, to, from string) error
}
// Commit publishes a revision. The index write happens first and is
// idempotent on Digest, so a retried job re-embeds nothing.
func Commit(ctx context.Context, ix Index, ctl Control, rev Revision, chunks []Chunk, canaries []string) error {
if err := ix.Upsert(ctx, rev.Digest, chunks); err != nil {
return fmt.Errorf("upsert %s/%s: %w", rev.ClaimID, rev.Role, err)
}
for _, q := range canaries {
hits, err := ix.Search(ctx, q, Filter{Digests: []string{rev.Digest}}, 5)
if err != nil {
return fmt.Errorf("canary search %q: %w", q, err)
}
if len(hits) == 0 {
return fmt.Errorf("canary %q retrieved nothing from %s: refusing to publish", q, rev.Digest[:12])
}
}
// Single row, compare-and-set on Supersedes: two workers racing on the same
// claim cannot both win, and the loser retries against the new head.
if err := ctl.Promote(ctx, rev.ClaimID, rev.Role, rev.Digest, rev.Supersedes); err != nil {
return fmt.Errorf("promote %s: %w", rev.Digest[:12], err)
}
return ix.DeleteByDigest(ctx, rev.Supersedes)
}
Two details in there are load-bearing. The compare-and-set on Supersedes is what makes concurrent ingest safe, which matters more than it sounds, because claim documents arrive in bursts when a carrier batch-forwards a mailbox. And the canary check refuses to publish rather than publishing something unretrievable, which is the correct bias for an intake pipeline: a document that is visibly missing gets escalated, while a document that is present but silently unretrievable does not.
Extraction failures deserve the same treatment. A scanned fax that OCRs into 40 characters of noise should fail the canary and stay unpublished, with the previous revision still live and a work item on someone's queue.
Verification, rollback, and the latency you're paying for the guarantee
Rollback is a control-plane operation: point the row back at the previous digest and re-embed from the object store if the chunks were already swept. That's why the sweeper needs a retention window rather than an immediate delete — 14 days is a reasonable starting point, long enough to cover a bad extractor deploy that nobody noticed over a holiday weekend, short enough that storage stays boring.
Alert on two things.
# SLI: share of cited chunks that belong to the live revision.
sum(rate(citations_total{revision_state="live"}[30m]))
/ sum(rate(citations_total[30m])) >= 0.999 # page below
# Staleness of the control snapshot the query path reads.
max(control_snapshot_age_seconds) < 60 # warn above
The second one catches the sneaky failure. If the query path caches the live-digest map to avoid a database round trip per search — and at 200 queries per second you will want that cache — then the cache is now a correctness dependency, and a stuck refresh loop means you are serving withdrawn documents while every index-level metric stays green.
On budget: a 400 ms p95 for an interactive answer leaves you roughly 250 ms after generation overhead, and a digest filter over a live set that is 95% of the corpus is cheap, while the same filter over a live set that is 30% of the corpus is not, because the ANN graph traversal has to visit far more candidates to fill k. That ratio, live chunks over total chunks, is the number that decides whether filtering stays viable or you move to the delete-and-insert commit above. Measure it per tenant. Litigation-heavy books of business skew badly.
Buy versus build: where the versioning logic actually belongs
Build the control plane. It's a table, a compare-and-set, and a sweeper — a few hundred lines that encode your legal and business rules about which document supersedes which, and no vendor can encode those for you. The retrieval engine underneath is a genuine buy decision, and the deciding question is narrow: does it give you atomic visibility changes, filtered search that doesn't collapse recall, and deletes that reclaim space on a schedule you can predict?
Stick with a managed engine when your on-call rotation is three people and your corpus fits comfortably in memory. Self-hosting a vector store to save infrastructure cost is a poor trade at that size, because the operational load lands on the same team that owns intake correctness.
Where this whole design is a bad fit: high-churn document sets with no meaningful revision semantics, where every upload is genuinely new and you're just doing search. Then the control plane is overhead. It earns its keep exactly when a document's meaning changes while its identity stays the same — claims, contracts, regulatory filings, medical records.
I'm not certain the 14-day retention default holds for every jurisdiction, and I'd check that against your own record-retention policy before shipping it. The rest of this generalizes.
Top comments (0)