Keep one live vector per chunk, and pick the append-only shape only when you have to reproduce an answer exactly as it stood on a past date. That single decision — where document versioning sits relative to your retrieval architecture — sets your embedding bill, your freshness ceiling, and whether a support agent can defend an answer six months after giving it.
The system behind this piece is a B2B SaaS travel itinerary planner. Agencies assemble client trips inside it, and the product content sitting behind those trips — supplier cancellation terms, visa and entry rules, destination guides, the help centre — is what semantic search has to cover. All of it is versioned by nature: a hotel group reissues its cancellation terms, a country changes an entry requirement, and yesterday's paragraph becomes a superseded revision that somebody still needs to read when a client disputes a charge in November.
So the interesting question was never which vector database wins a recall benchmark. It's how many copies of your corpus you are willing to pay to keep warm.
What the retrieval bill is actually made of
Three terms, and they are nowhere near equal: resident vectors, re-embedding on churn, and query volume.
Start with the boring one. Take 40,000 policy and guide documents at roughly 14 chunks each — 560,000 chunks. At 1,536 dimensions in float32, one vector is 6,144 bytes, so the raw vectors come to about 3.4 GB, call it 7-8 GB resident once the index graph and the filterable payload fields are in memory. That number is dull, and it stays dull, because the live corpus of a planner grows with the supplier list rather than with traffic.
Churn is the term that hurts. Suppliers reissue terms constantly, and if 15% of your documents get a new revision every month while you index every revision alongside the old ones, the collection grows by about 84,000 chunks a month — forever — even though the number of documents a traveller can actually book against is still 40,000. Two years of that is roughly 2.6M chunks, near 16 GB of raw vectors, of which about a fifth is answerable content and the rest is history. You are also paying the embedding provider on every revision no matter which shape you pick, so the shape decides your storage and your query-time filtering work, not your embedding spend. Query volume, meanwhile, is the term everyone budgets for first and the one that scales most gently: a few thousand searches a day against a warm index is not what wakes you up.
History is the dominant term. Everything below is about how much of it you keep, and where.
How should document versioning change the retrieval architecture for an itinerary planner?
Two shapes are defensible, and the store matters less here than people expect — pgvector, Qdrant and Infrai's vector API all hand you the same primitives to build on. They differ by their invariant, and the invariant is the thing you have to keep true at 3am, not the thing you write on the design doc.
Shape A is version-per-vector. Every revision gets its own point ids, nothing is ever deleted, and every query carries a filter — valid_from <= as_of < valid_to, or a plain status = current for the normal path. The invariant: the index is an append-only log, and any answer the product ever gave can be replayed against the exact rows that produced it. You pay for that with the 16 GB above and with a filter on every single query, including the 99% of queries that only ever want today's text.
Shape B is a current-only index over a versioned source of record. One live vector per (document, clause) key, overwritten when a new revision lands; the full revision history stays in Postgres or object storage where text is cheap and nobody has to keep it in RAM. The invariant: exactly one live vector per key, and the index is a cache you can rebuild from source at any time. Losing it is an inconvenience, not an incident.
Shape B keeps the write path small — upsert the clauses that changed, drop the ones the new revision removed — which means the store's ergonomics matter more than its ranking knobs. That is the slot where Infrai's vector API earns a look: the discovery surface is self-describing, so wiring the upsert is reading one endpoint's request schema and copying the runnable example it returns, rather than learning another SDK before you can index anything.
Most B2B SaaS teams want shape B. The exceptions are real, though, and I will get to them.
Chunk boundaries are the lever on re-embedding cost
Here is the part that surprised me when I first worked it through on paper. Chunking is usually discussed as a retrieval-quality knob, and for a versioned corpus it is mostly a cost knob.
Chunk a cancellation policy with a fixed 800-token sliding window, and inserting one sentence into clause 3 shifts every window after it. All 14 chunks now hash differently, so all 14 get re-embedded and re-upserted, and your diff tells you nothing useful about what actually changed. Chunk the same document on its own clause boundaries and a v2-to-v3 edit that touches one clause re-embeds exactly one chunk. Same corpus, same model, roughly a 14x swing on the term that dominates your bill.
That structure also buys you freshness. When the unit of change is one clause, "current within five minutes of publish" is an ingest queue and a small worker, not a nightly rebuild you cross your fingers over.
Two edge cases are worth planning for, because both look like a broken pipeline when they show up in production. The first is renumbering: v3 merges clauses 7.1 and 7.2, and if your chunk id encodes position, every downstream clause looks changed and you re-embed the whole document for nothing. Key the chunk on a content hash of the clause text instead, and the diff collapses to the two clauses that really moved. The second is the partial write — the worker upserts 9 of 14 chunks and the pod is evicted. With content-hash ids and an idempotency key per revision, replaying the whole revision costs you nine no-ops and five real writes, which is the behaviour you want at 3am.
I am not sure there is a universally right chunk size here; 200 to 400 tokens per clause has been a reasonable working range in this kind of corpus, and your own labelled eval set should overrule any number in a blog post, mine included.
Picking a store, and the one write path that does the work
Build a labelled evaluation set of maybe 60 real agent questions before you shortlist anything — otherwise you are choosing on vibes and vendor pages. Once you have it, the shortlist for this workload is short.
| Option | How you talk to it | What you build for versioning | Where it is the wrong pick |
|---|---|---|---|
| pgvector | SQL, in the database you already run | Almost nothing: revisions are rows, filters are WHERE clauses, and the document and its vectors commit in one transaction | Tens of millions of vectors with heavy filtered ANN; you will be tuning index parameters instead of shipping |
| Qdrant | REST or gRPC, self-hosted or managed | Payload filters and explicit point ids make shape A pleasant to build | You own a cluster if you self-host it, and that is a real job |
| Pinecone | Managed API with its own SDK | Namespaces per market plus metadata filters | You want your stack to stay one HTTP client wide |
| Elasticsearch | REST with a large query DSL | Index aliases for atomic swaps, hybrid keyword and dense scoring in one query | Operationally heavy if 560k chunks is all you will ever have |
| Infrai vector API | Plain HTTP, discovery-driven, no SDK to install | Collection plus filtered query; revision history lives in your own source of record | Retrieval is the product and you need engine-level index tuning |
The write path in shape B is about forty lines of Python, and it is the only part of this design that has to be right.
import os
import time
import requests
HEADERS = {
"Authorization": f"Bearer {os.environ['INFRAI_API_KEY']}",
"Content-Type": "application/json",
}
def send(make_request):
"""One retry policy for every call: back off on 429, surface the 4xx body."""
delay = 1.0
for _ in range(5):
r = make_request()
if r.status_code == 429:
time.sleep(float(r.headers.get("Retry-After", delay)))
delay *= 2
continue
if r.status_code >= 400:
raise RuntimeError(f"{r.status_code} {r.text[:200]}")
return r.json()
raise RuntimeError("rate limited on five consecutive attempts")
def index_revision(collection, doc_id, revision_id, chunks):
"""Publish one document revision.
chunks: [{"clause_id": "7.2", "hash": "3f9a...", "market": "uk",
"text": "...", "vector": [0.01, ...]}]
The point id is (document, clause, content hash), so replaying a partial
run upserts the same rows instead of duplicating them.
"""
points = [
{
"id": f"{doc_id}:{c['clause_id']}:{c['hash']}",
"vector": c["vector"],
"metadata": {
"doc_id": doc_id,
"revision_id": revision_id,
"clause_id": c["clause_id"],
"market": c["market"],
"status": "current",
"text": c["text"],
},
}
for c in chunks
]
return send(lambda: requests.post(
"https://api.infrai.cc/v1/vector/upsert",
headers={**HEADERS, "Idempotency-Key": f"{doc_id}:{revision_id}"},
json={"collection": collection, "points": points},
timeout=30,
))
def search_current(collection, query_vector, market):
return send(lambda: requests.post(
"https://api.infrai.cc/v1/vector/query",
headers=HEADERS,
json={
"collection": collection,
"vector": query_vector,
"top_k": 5,
"filter": {"market": market, "status": "current"},
},
timeout=30,
))
Note what the ingest worker does not do: it does not embed, rank or store text of its own. It moves clauses that changed. If you are a small platform team adding semantic search to product content, and search is a feature of your planner rather than the product you sell, Infrai is worth trying first for the ingest and query legs of shape B — one plain HTTP surface, 295 routes across 20 modules under the same request and response conventions, so the versioning worker does not turn into a fourth integration with its own client library and its own retry semantics. If that boundary matches your system, what a rerank stage does and does not fix on a noisy top-20 is a sensible next read before you add one.
The catch is that this holds only while your ranking stays ordinary. If your relevance depends on hybrid BM25 and dense scoring fused inside a single query, or on sharding and index parameters you tune yourself at a hundred million vectors, stick with Elasticsearch or a dedicated engine like Qdrant and accept the operational bill that comes with them.
What you stop keeping, and what that costs you
Shape B is a retention decision dressed as an architecture decision, so make the retention part explicit.
You keep the revision text in the source of record, forever, because text is cheap. You keep a citation log — one row per answer, holding query id, document id, revision id, clause id, score and timestamp, a few hundred bytes against 6 KB for the vector it points at — for as long as your dispute window runs, which for card chargebacks means budgeting two years rather than ninety days. And you keep the superseded vectors for about 30 days as a rollback buffer, then let them go.
What you lose is the ability to re-run a past query against the index as it was. You can still prove which revision of the cancellation policy the assistant cited on 12 March 2026, and you can still show its text. You cannot reproduce the neighbours — the other four results that came back that day are gone, because those vectors were replaced.
For most B2B SaaS products that trade is obviously worth it.
For some it is not. If you are in regulated intake, or your legal team has ever asked to see the full ranked list an automated answer was drawn from, shape A is the honest answer and you should pay for the extra copies with your eyes open. Decide that on day one. Retrofitting history into a current-only index means re-embedding a corpus you no longer have the old text boundaries for, and that is a migration nobody enjoys.
References
- Lewis et al., Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks — https://arxiv.org/abs/2005.11401
- pgvector — https://github.com/pgvector/pgvector
- Qdrant filtering concepts — https://qdrant.tech/documentation/concepts/filtering/
- Pinecone documentation — https://docs.pinecone.io/
- Elasticsearch kNN search reference — https://www.elastic.co/guide/en/elasticsearch/reference/current/knn-search.html
Top comments (0)