Short answer: keep every uploaded claim document immutable, assign a monotonically increasing revision, and let retrieval filter by an explicit as-of policy before ranking chunks. That ordering matters more than the choice of vector index: a fast search over the wrong policy wording is a correctness failure, not a latency win.
In an insurance claims intake system, one claim can accumulate a police report, a repair estimate, a revised estimate, correspondence, and an amended policy schedule. Treating those files as one mutable blob makes a RAG answer impossible to defend. The adjuster needs to know which wording was in force when a decision was made, while the intake service still has to return useful evidence quickly.
Dates lie. Hashes don't.
The constraint is temporal, not merely semantic
The retrieval unit should carry two identities: a stable document_id for the logical artifact and a revision_id for each accepted byte sequence. Store the original object under a content-addressed key, then write a metadata row that records claim, document type, revision number, effective interval, ingestion timestamp, and a hash of the bytes. The vector index gets chunk records that point back to that row; it never becomes the source of truth.
This arrangement separates three clocks that otherwise get mixed together: when a claimant uploaded a file, when an operator approved it, and when the policy text became effective. A query for “coverage on the loss date” should use the effective interval. A query for “what did the reviewer see yesterday?” should use ingestion time. Make the policy explicit in the request and in logs.
One small rule prevents a large class of incidents: a revision is append-only. If OCR improves, write a new derived revision and preserve the input hash. Deleting an old vector is an indexing operation, not document deletion; retention and legal hold belong to the object and metadata layers.
How should retrieval architecture handle insurance claims intake document versioning?
Start with candidate generation constrained by claim and revision policy, then apply semantic ranking. Do not ask an embedding model to infer chronology from prose. For each candidate chunk, carry revision_id, page coordinates, and a citation range into the prompt context. If the policy selects one revision, a superseded chunk must be ineligible, even when its cosine score is higher.
Here is a deliberately plain Python sketch. The important part is the filter that runs before ranking; the vector store is an interchangeable implementation detail.
from dataclasses import dataclass
from datetime import date
@dataclass(frozen=True)
class Chunk:
claim_id: str
document_id: str
revision: int
effective_from: date
effective_to: date | None
text: str
vector: list[float]
def eligible(chunks: list[Chunk], claim_id: str, as_of: date) -> list[Chunk]:
return [
chunk for chunk in chunks
if chunk.claim_id == claim_id
and chunk.effective_from <= as_of
and (chunk.effective_to is None or as_of < chunk.effective_to)
]
def retrieve(chunks: list[Chunk], claim_id: str, as_of: date, embed_query):
candidates = eligible(chunks, claim_id, as_of)
query_vector = embed_query("deductible and loss coverage")
return rank_by_similarity(query_vector, candidates)[:8]
The production version also checks a revision manifest before returning results. A manifest can require that all chunks for a revision are present, that OCR status is accepted, and that the byte hash matches the object metadata. If any check is false, fail closed for that revision and surface a review state rather than quietly mixing old and new chunks.
Failure modes worth testing before launch
The most common failure is a partial re-index: page one comes from revision 4 and page two remains revision 3. A second is an “update” that changes text while retaining the same revision number, defeating audit joins. The third is a time-zone bug around an effective-at midnight boundary. These are ordinary data bugs, and a relevance benchmark will miss them.
The partial re-index deserves a concrete test. Upload estimate revision 4, let its first page finish embedding, interrupt the worker, and issue a coverage query while revision 3 is still searchable. The expected response is either a complete revision 3 result or an explicit “revision 4 pending” state; a hybrid answer is forbidden. Then resume the worker, verify that every chunk points to the same manifest generation, and repeat the query at the exact midnight boundary in both UTC and the business timezone. This test catches the quiet failures that look like plausible language to a reviewer, especially when the changed deductible appears on a page that was not re-embedded.
I test with a small adversarial corpus: two estimates that differ only in a deductible, a policy endorsement effective one day after the loss, and a duplicate upload with a different filename but identical bytes. Expected results assert revision IDs and citation spans, not just answer strings. A 409-style conflict from the metadata writer should be retried only when the idempotency key is unchanged; otherwise it indicates a competing revision and needs operator visibility.
| Decision | Favors | Cost or limitation |
|---|---|---|
| Filter by effective date first | Correct historical answers | Smaller candidate pool can reduce recall when dates are missing |
| Keep all revisions in one index | Simple operations and broad recall | Every query needs strict metadata predicates |
| Separate index per revision | Strong isolation and easy rollback | More indexes, slower promotion, and harder cross-document search |
| Re-embed on every OCR change | Consistent text/vector pairs | Embedding spend and a longer publish window |
Your mileage may vary when claims arrive in dozens of languages or when policy endorsements are represented as structured schedules rather than prose. In those cases, add deterministic field extraction and language-aware evaluation; do not loosen the revision predicate to compensate for weak chunking.
Rollout and observability
Publish a revision in stages: write the immutable object, commit metadata, chunk and embed, validate the manifest, then atomically mark the revision searchable. Keep the previous searchable revision until the new one passes checks. A reader that sees a pending revision should receive the last accepted revision plus a visible freshness timestamp, never an unlabelled mixture.
Track retrieval latency separately from freshness lag. Useful counters include eligible-candidate count, filtered-out superseded chunks, manifest failures, citation coverage, and the percentage of answers whose cited revision matches the requested policy. Sample traces should include claim ID, revision policy, and index generation, with personal data redacted.
Ship less. Verify more.
The catch is operational: immutable history increases storage and retention work. It is not suitable for teams that cannot enforce legal-hold and deletion rules across both objects and derived vectors. Stick with a simpler current-state index for a low-stakes internal search, or choose a relational audit log first when every answer must be reconstructed years later.
For an incremental migration, dual-write new uploads to the revision table while continuing to serve the current index. Backfill hashes and effective intervals, replay a fixed evaluation corpus, and compare citation-level results. Switch reads only after the mismatch rate is understood; a green latency graph alone says nothing about historical correctness.
Top comments (0)