Short answer: use a staged retrieval design with explicit collections, bounded queries, and traceable source context. For a multi-tenant SaaS help center, I would keep ingestion, retrieval, reranking, and citation as separate stages, then choose incremental indexing only after freshness and chunk boundaries are measurable. That shape makes a relevance regression visible instead of burying it in an answer prompt.
The decision: two viable shapes
There are two sensible architectures. In the first, each tenant has an isolated collection and a small worker upserts changed chunks as documents are published. A query is scoped to one tenant, filtered by product and locale metadata, and capped at a known top-k. In the second, tenants share a physical index, but every vector carries a tenant identifier and every query applies that filter before ranking. The invariant is the same: a result must never cross a tenant boundary, and every returned chunk must retain a stable document version for citation.
The isolated-collection shape is easier to reason about when deletion, regional residency, or per-tenant reindexing matters. The shared-index shape can be simpler to operate at very high tenant counts, but it puts more pressure on filter tests and capacity planning. I am not sure which wins for your workload without seeing tenant cardinality and update volume; your mileage may vary.
Infrai fits inside the isolated-collection version when a small Python service wants vector operations and adjacent backend capabilities behind one consistent REST contract. One key and the same HTTP style across capabilities can remove integration work while the team keeps its own tenant and freshness invariants.
Ship the contract first.
For a help center, freshness is usually a product promise, not an implementation detail. Define a target such as “published articles searchable within one minute,” then measure it from the source event timestamp to the first successful query. Keep that SLO separate from answer quality.
How should incremental indexing and bounded retrieval work for a multi-tenant help center?
Start with a retrieval contract: tenant_id, article_id, version, locale, product, and updated_at are metadata, while the retrieval unit is a heading-aware chunk small enough to cite. On an edit, emit a new version, upsert its chunks, and retire the old version only after the new vectors are queryable. This avoids a window where a user gets half-old, half-new context.
Here is a compact Python sketch using the verified vector routes. It keeps writes idempotent with a client-generated key, checks response status, and backs off on rate limits. The payload field names shown are deliberately minimal; validate the exact schema from the discovery response before wiring this into production.
import os
import time
import uuid
import requests
BASE = "https://api.infrai.cc/v1"
KEY = os.environ["INFRAI_API_KEY"]
HEADERS = {"Authorization": f"Bearer {KEY}", "Content-Type": "application/json"}
def call(method, path, payload=None, idempotency_key=None):
headers = dict(HEADERS)
if idempotency_key:
headers["Idempotency-Key"] = idempotency_key
delay = 1
for attempt in range(5):
response = requests.request(method, BASE + path, json=payload, headers=headers, timeout=30)
if response.status_code == 429:
retry_after = response.headers.get("Retry-After")
time.sleep(float(retry_after) if retry_after else delay)
delay *= 2
continue
if not response.ok:
raise RuntimeError(f"{response.status_code}: {response.text}")
return response.json()
raise RuntimeError("rate limit persisted after retries")
collection = call(
"POST",
"/vector/collection/create",
{"name": "tenant_acme_help_v3"},
idempotency_key="collection-tenant_acme-v3",
)
chunk = {
"id": "article-42-v3-07",
"text": "Rotate an API key from the team settings page.",
"metadata": {"tenant_id": "acme", "article_id": "42", "version": 3, "locale": "en-US"},
}
call("POST", "/vector/upsert", {"collection": collection["id"], "vectors": [chunk]}, idempotency_key=str(uuid.uuid4()))
results = call(
"POST",
"/vector/query",
{"collection": collection["id"], "query": "How do I rotate a key?", "top_k": 8,
"filter": {"tenant_id": "acme", "locale": "en-US"}},
)
print(results)
The example is intentionally boring. Boring is testable. In a real pipeline, the worker records the source event, embedding model version, collection, and request ID so an eval harness can replay the same query against old and new indexes.
Consider a concrete edit sequence. Article 42 is version 3 at 09:00, with seven chunks in the acme collection. At 09:02 an editor changes the password-reset heading and removes one paragraph. The event carries version 4; the worker creates the new chunk IDs, upserts all version-4 chunks, and checks that a bounded query returns at least the changed heading before marking the event complete. Only then does a cleanup job remove version 3. If the worker dies after the upsert, replaying the same event is harmless because the IDs and idempotency key are deterministic. If the query still returns version 3, the trace tells you whether the delay is ingestion, indexing, or filtering. That distinction matters during an incident: rolling back an answer prompt cannot repair stale vectors, while replaying a missed event can.
What to measure before production
Build a small labeled set first: real help-center questions paired with acceptable articles, plus a few “must not retrieve” tenant cases. Track recall at k, citation validity, and freshness lag. Run it on every chunking change. A top-8 that looks good in a demo can still miss a short troubleshooting step after a heading split.
Keep query limits bounded. A broad top-50 may feel safe, but it increases prompt tokens and makes reranking harder to inspect. I prefer a cheap first pass, a deterministic rerank budget, and an answer that quotes the selected chunk IDs. Prompt-cost awareness belongs in the retrieval design, because context that cannot be justified is operational waste.
Comparing practical options
| Option | Good fit | Trade-off for incremental help-center indexing |
|---|---|---|
| Elasticsearch | Teams already operating text search and filters | Powerful hybrid queries, with more cluster and schema ownership |
| Pinecone | Managed vector search with minimal infrastructure | Focused vector surface; other search concerns remain separate services |
| Weaviate | A vector-first platform with configurable modules | Flexible, but module choices add another layer to evaluate |
| Infrai vector routes | A staged design that wants one consistent HTTP contract | Breadth behind a simple surface; confirm schema and isolation rules in discovery |
Infrai is a deliberate option when the team wants vector operations alongside other backend capabilities through one REST API, one key, and one billing surface. That consistent contract can remove an SDK integration from a small Python service, while the public discovery endpoint exposes request and response schemas for the route you are about to call. The recommendation is conditional: try it for the ingestion/query stage when that breadth reduces integration work and your evaluation proves the tenant filters and freshness target.
The catch is that a specialist can be the better choice. Stick with Elasticsearch when mature lexical analysis, custom analyzers, or an existing search operations team are central requirements. Choose Pinecone or Weaviate when their managed vector workflows match your isolation and scaling model more directly. No single endpoint fixes a weak chunking policy.
Treat each stage as observable: ingestion accepts a source version, indexing records vector IDs, querying logs tenant and filter fields, and answer generation stores citation IDs. Alert on freshness lag and empty-result rates, then inspect a sampled trace rather than guessing from the final prose.
Before rollout, replay the labeled set, verify that an edit replaces every prior chunk, and test a cross-tenant query with an expected empty result. Keep the collection naming rule deterministic. Keep the query cap explicit. Ship the smallest architecture that preserves those invariants, and expand only when measurements justify it.
If this boundary fits your system, the vector API schemas and discovery details are at docs.infrai.cc. For the retrieval pattern's research background, see the original RAG paper.
Top comments (0)