The trade-off is easy to state: indexing every revision gives broad recall, while serving only the latest text keeps latency and answer clarity under control. For internal engineering documentation, I would keep immutable document versions, attach a validity interval to each chunk, and filter retrieval by the reader's requested release before reranking. Short answer: make version a first-class retrieval field, then measure recall and latency separately.
Property-management teams see this sharply. A maintenance runbook may describe a boiler model from 2024, while a 2026 replacement uses different safety steps. A vector match that ignores version can return a plausible paragraph and still cause a bad work order.
The field guide: choose a version policy before a chunking policy
| Policy | Pick it when | Main risk |
|---|---|---|
| Latest-only index | Readers almost always need current operating procedure | Historical incident analysis loses context |
| Immutable versions with metadata filters | Releases, tenants, or audit reviews must reproduce an answer | More storage and filter logic |
| Dual index (current plus archive) | Fast interactive search and occasional audits have different SLOs | Two ingestion and monitoring paths |
The second option is the useful default for engineering documentation. Store doc_id, version, effective_from, effective_to, and a content hash beside every vector. A tombstone can mark a withdrawn revision without deleting evidence. Do not infer version from a filename; hvac-guide-final-v3.md is not a contract.
How should retrieval architecture handle internal engineering documentation document versioning?
Think of the pipeline as a diagram in words: source commit flows to a canonical document, the canonical document splits into structure-aware chunks, each chunk receives version metadata, embeddings enter a vector index, and a query carries a version constraint into both lexical and vector search. Reranking happens after that constraint, never before it.
Chunk boundaries should follow headings, procedures, and tables. Keep a stable chunk_key derived from the document id, version, and section path. That lets an ingest retry upsert the same logical chunk instead of creating duplicates. For a long inspection checklist, overlap should be small enough to avoid repeating every warning, but large enough to retain the prerequisite sentence. Your mileage may vary; validate overlap on real questions rather than adopting a magic token count.
Here is a compact ingestion shape. It is an interface, not a vendor SDK, so a worker can target a hosted or self-managed index.
type Chunk = {
chunkKey: string;
docId: string;
version: string;
effectiveFrom: string;
effectiveTo?: string;
sectionPath: string[];
text: string;
contentHash: string;
};
async function ingest(chunks: Chunk[], upsert: (c: Chunk) => Promise<void>) {
for (const chunk of chunks) {
await upsert(chunk);
}
}
A query for a 2026 work order should pass version=2026 or an effective date, not merely ask for “boiler reset.” If no version is supplied, return the current version and label it clearly; silently mixing revisions is the dangerous behavior.
Failure modes that look like good search
The first trap is stale dominance: old chunks have stronger wording and win nearest-neighbor search. The second is split-brain metadata, where the vector row says 2026 but the source payload still contains 2024 text. Hash the normalized chunk and compare it during indexing. The third is delete drift: a removed procedure remains searchable because deletion was implemented only in the source connector. Track tombstone counts and sample queries after every release. Keep a small set of property records that changed during the last release and run them through the pipeline after each deploy. Compare the returned version and contentHash with the manifest, then inspect the raw source when they disagree. That join catches parser changes, connector deletes, and accidental reuse of a chunk key in one pass, giving on-call engineers an artifact they can attach to an incident instead of arguing from screenshots.
I once expected a simple “latest” filter to solve this class of issue. It did not. A query log showed that users asked for “the steps used during the January outage,” which is inherently historical. The fix was a visible time selector plus an archive policy, not a bigger embedding model.
Observability needs three separate views: retrieval quality (recall at k, citation-version match, and answer refusal rate), latency (p50/p95 from query receipt through reranking), and freshness (source commit to searchable timestamp). Alert on a rising citation-version mismatch even when p95 is green. A green latency chart can hide a wrong answer.
Build a labeled set with questions that name a release, questions that omit a release, and questions whose answer changed. Test lexical-only, vector-only, and hybrid retrieval against the same set. Keep a canary index for a new parser or embedding model; compare version-conditioned recall before promoting it.
For deployment, make ingestion idempotent, queue retries with a bounded backoff, and publish a manifest containing source revision, chunk count, and hash count. The manifest is the audit join key between a source repository and a search result.
A small TypeScript check can block accidental cross-version joins:
function sameRevision(resultVersion: string, requestedVersion?: string) {
return requestedVersion === undefined || resultVersion === requestedVersion;
}
Measure it.
Limits and decision rule
Version-aware retrieval is not suitable when documents are disposable, answers never need historical context, and the team cannot operate metadata validation; a latest-only index is simpler there. It is also a poor fit for unstructured scans with no reliable source revision, until a canonicalization step is added.
Choose immutable versions when auditability matters, dual indexes when interactive latency and archive search have different budgets, and latest-only only after measuring the cost of stale answers. Retrieval quality is the decision axis; storage and latency are constraints to instrument, not reasons to guess.
Top comments (0)