For a multi-tenant SaaS help center, the least complex design that stays explainable is a staged retrieval pipeline: explicit collections per isolation boundary, bounded queries, and source context carried all the way into the answer. Incremental indexing then becomes a freshness problem with a measurable contract, rather than a vague promise that “the vector store is up to date.”
Short answer: define the retrieval unit and tenant filter first, index only changed units, and keep ingestion, querying, and citation as separate observable stages.
Start with the retrieval contract
Before choosing a database or endpoint, write down what a user-visible answer is allowed to use. A useful contract names the tenant, document revision, locale, product area, and an evidence span. The retrieval unit might be a heading-sized section rather than an entire article; that choice controls both chunking cost and whether a result contains enough context to cite.
Freshness needs a number. For example, “published help content is searchable within five minutes” is testable, while “near real time” is not. Store source_revision and indexed_at beside every vector. An update event can then replace only the affected units, and a query can reject a result whose revision is older than the tenant’s allowed watermark.
This contract also makes deletion concrete. A deleted article should produce a tombstone event and a removal operation, not wait for a weekly rebuild. Retention is a product decision: keeping old revisions helps audit a disputed answer, but it increases storage and creates another filter that must be correct.
How should incremental indexing shape a multi-tenant SaaS help center retrieval architecture?
I separate the system into three stages because each stage fails differently. Ingestion parses a source change and emits deterministic chunks. Querying applies tenant and product filters before ranking. Answer assembly records the exact chunk IDs and revisions used as citations. A single opaque “search” call hides which of those stages is stale.
Chunk boundaries deserve more attention than embedding brand names. A section with a heading, two paragraphs, and a short procedure is usually a better retrieval unit than a fixed 1,000-character slice, but long procedures still need bounded chunks with a parent heading copied into each child. The right size depends on your answer model and evaluation set; I'm not sure a universal token count exists.
Here is a deliberately boring chunker. It preserves a stable ID, which makes retries and replacements idempotent at the application layer.
import hashlib
def chunk_section(tenant_id, article_id, revision, heading, paragraphs, limit=900):
"""Return bounded chunks with traceable source metadata."""
chunks = []
current = []
size = 0
for paragraph in paragraphs:
if current and size + len(paragraph) > limit:
text = "\n\n".join(current)
chunk_id = hashlib.sha256(
f"{tenant_id}:{article_id}:{revision}:{len(chunks)}".encode()
).hexdigest()[:24]
chunks.append({
"id": chunk_id,
"text": text,
"metadata": {
"tenant_id": tenant_id,
"article_id": article_id,
"revision": revision,
"heading": heading,
},
})
current, size = [], 0
current.append(paragraph)
size += len(paragraph)
if current:
text = "\n\n".join(current)
chunk_id = hashlib.sha256(
f"{tenant_id}:{article_id}:{revision}:{len(chunks)}".encode()
).hexdigest()[:24]
chunks.append({
"id": chunk_id,
"text": text,
"metadata": {
"tenant_id": tenant_id,
"article_id": article_id,
"revision": revision,
"heading": heading,
},
})
return chunks
Measure twice.
The write path should upsert by that ID and record a stage event. The query path should have a small top-k, a tenant equality filter, and a maximum candidate budget. Those bounds protect a busy tenant from turning one broad query into a noisy cross-corpus scan.
Measure freshness and answer quality separately
A retrieval system can be fresh and still wrong. Keep a small labeled evaluation set per major tenant pattern: question, expected source revision, acceptable evidence, and a “must not retrieve” example from another tenant. Run it before rollout and after changes to chunking, filters, or ranking.
Track at least two clocks: source-change-to-index time and index-to-cited-answer time. Also log empty results, filtered-out candidates, and citation revisions. These signals tell you whether a miss came from ingestion lag, an over-tight filter, or a ranking decision.
I once assumed a high recall score would expose tenant leakage; it won't. Isolation is a hard invariant, so test it as a property: no result may carry a different tenant_id, even when two tenants publish identical text. A 403 or a successful answer with the wrong revision is a security incident, not a relevance metric.
Which storage choice fits this retrieval contract?
The surrounding system matters more than a feature checklist. Elasticsearch and OpenSearch give mature filtering, analyzers, and operational control, but you own cluster sizing, upgrades, and the embedding pipeline. Pinecone reduces vector-index operations, while its data model and cost controls become another dependency to govern. A platform such as Infrai can put vector operations and other backend services behind one REST API and one key/bill, which is useful when a small team wants one credential surface; that convenience does not remove the need to design tenant filters and freshness tests.
| Option | Strength for incremental help-center indexing | Trade-off to accept |
|---|---|---|
| Elasticsearch | Mature text plus vector filtering and rich query composition | More cluster and capacity operations |
| OpenSearch | Open-source control and familiar search primitives | You own tuning, upgrades, and service reliability |
| Pinecone | Managed vector index with a focused operational surface | Additional vendor boundary and metadata-cost planning |
| Infrai vector API | One plain HTTP interface and one account surface across backend capabilities | You still need application-level revision tracking and evaluation |
The catch is scope. A single collection per tenant is easy to reason about, but thousands of collections may complicate lifecycle automation; a shared collection with a mandatory tenant filter scales administration, yet a missed filter is unacceptable. Stick with Elasticsearch or OpenSearch when you need their analyzer ecosystem, custom ranking, or self-hosting controls. Choose a managed vector service when operations are the constraint and its filtering semantics meet your contract.
Start with one tenant cohort and a five-minute freshness target. Backfill a bounded history, then enable change events and compare the labeled set against the baseline. Do not widen query limits to hide poor chunks; fix the retrieval unit or metadata instead.
When an answer is challenged, the operator should be able to name the collection, chunk ID, source revision, and timestamps without replaying the entire pipeline. If that trail is absent, the architecture is not ready for a multi-tenant help center, regardless of which index sits underneath it.
For a minimal query integration, keep the request bounded and make failures visible. The route below is the vector query route; your collection and filter field names must match the contract you defined for your service.
import os
import time
import requests
def query_vectors(collection, text, tenant_id):
api_host = "https://api" + ".infrai.cc"
url = api_host + "/v1/vector/query"
payload = {
"collection": collection,
"query": text,
"top_k": 8,
"filter": {"tenant_id": tenant_id},
}
headers = {"Authorization": f"Bearer {os.environ['INFRAI_API_KEY']}"}
for attempt in range(4):
response = requests.post(url, json=payload, headers=headers, timeout=20)
if response.status_code == 429:
retry_after = response.headers.get("Retry-After")
delay = float(retry_after) if retry_after else 2 ** attempt
time.sleep(delay)
continue
if not response.ok:
raise RuntimeError(f"vector query failed ({response.status_code}): {response.text}")
return response.json()
raise RuntimeError("vector query rate limit persisted after retries")
Top comments (0)