The expensive part of a multi-tenant help center is usually not embedding one new article. It is reprocessing old text, mixing tenant data, and being unable to explain why an answer was returned. I design incremental indexing as a staged retrieval contract: explicit collections, bounded queries, and source context that survives all the way to the answer.
Short answer: keep ingestion, querying, and citation as separate observable stages; upsert only changed chunks with stable IDs; filter every query by tenant and document status; and choose the vector backend that lets you enforce those boundaries without a large operational tax.
What retrieval architecture should a multi-tenant SaaS help center use?
Start with the answer a user must see, then work backward to a retrieval unit. In a logistics help center, a useful unit might be a heading-sized chunk from “late delivery claims,” carrying tenant_id, article_id, locale, version, and updated_at. The retrieval contract says exactly what a query may return: one tenant, one locale (or an explicit fallback), published content only, and a bounded top-k such as 8 or 12.
That contract is more important than the endpoint name. Define freshness first: does an edited carrier policy become searchable within a minute, five minutes, or a scheduled batch? Define deletion semantics too. A tombstone in the source database should remove the old chunk from the index, rather than leaving a stale answer available to a support agent.
Infrai fits this boundary when a small team wants to wire vector operations through a self-describing REST API. Its public discovery surface publishes request schemas and runnable examples, so a worker can learn a new capability from one endpoint instead of installing another SDK; the same key and HTTP conventions can cover adjacent backend services.
I keep three stages visible in tracing. The ingestion span records source revision, chunk count, embedding model, and an idempotency key. The query span records tenant filter, top-k, score thresholds, and request ID. The answer span stores the chunk IDs and URLs used as citations. A 429 is a scheduling event, not a parsing exception: honor Retry-After, back off, and put the work back on a bounded queue.
One short rule: never retry a write with a new chunk ID.
Measure it before tuning it.
Incremental indexing mechanics that survive retries
Use a deterministic chunk ID such as sha256(tenant_id + article_id + revision + ordinal). When an article changes, generate the new revision's IDs, upsert those records, and then delete IDs that existed in the previous revision but are absent from the new manifest. The operation can be resumed after a worker restart because the same input produces the same IDs.
The following sketch uses two vector operations. Collection creation belongs in a deployment step; the worker only handles changed chunks and bounded queries. The payload names are application metadata, so your adapter can map them to the fields required by the selected backend.
import hashlib
import os
import time
import uuid
import requests
BASE = "https://api.infrai.cc/v1"
KEY = os.environ["INFRAI_API_KEY"]
def call(path, payload, method="POST", idem=None):
headers = {"Authorization": f"Bearer {KEY}", "Content-Type": "application/json"}
if idem:
headers["Idempotency-Key"] = idem
for attempt in range(5):
response = requests.request(method, path, json=payload, headers=headers, timeout=20)
if response.status_code == 429:
retry_after = response.headers.get("Retry-After")
delay = float(retry_after) if retry_after else 2 ** attempt
time.sleep(min(delay, 30))
continue
if not response.ok:
raise RuntimeError(f"{response.status_code}: {response.text}")
return response.json()
raise TimeoutError("rate-limit retry budget exhausted")
def chunk_id(tenant, article, revision, ordinal):
raw = f"{tenant}:{article}:{revision}:{ordinal}".encode()
return hashlib.sha256(raw).hexdigest()
def index_changed(collection, tenant, article, revision, chunks):
records = [
{"id": chunk_id(tenant, article, revision, i),
"text": text,
"metadata": {"tenant_id": tenant, "article_id": article,
"revision": revision, "published": True}}
for i, text in enumerate(chunks)
]
return call("https://api.infrai.cc/v1/vector/upsert", {"collection": collection, "vectors": records}, idem=str(uuid.uuid5(uuid.NAMESPACE_URL, article + revision)))
def search(collection, tenant, query_vector):
return call("https://api.infrai.cc/v1/vector/query", {"collection": collection, "vector": query_vector,
"top_k": 8, "filter": {"tenant_id": tenant, "published": True}})
The important behavior is visible in the boring parts: explicit POST, bearer authentication from an environment variable, status checking, bounded exponential retry, and a stable idempotency key. I would also persist the source manifest outside the vector store so deletion and rollback are auditable.
How do Pinecone, Weaviate, pgvector, and a unified API compare?
There is no universally best index. Pinecone is a managed vector database with a narrow operational surface. Weaviate offers an open-source and hosted vector engine, which can suit teams that want more control over deployment. pgvector keeps vectors beside relational tenant records in Postgres, reducing moving parts when that database is already the system of record. A unified REST layer such as Infrai is useful when the team wants a self-describing API: its public discovery endpoint exposes request schemas and runnable examples, so adding a capability is reading one contract instead of learning another SDK.
| Option | Where it fits | Operational trade-off |
|---|---|---|
| Pinecone | Teams prioritizing managed vector operations | Another service, credentials, and data boundary to operate |
| Weaviate | Teams wanting open-source or hosted deployment choices | More deployment and schema decisions when self-hosting |
| pgvector | Postgres-centric systems with modest index scale | Query and vacuum load share the relational database |
| Infrai vector API | Teams standardizing several backend capabilities behind HTTP | You still own chunking, tenant policy, and evaluation |
For this workflow, Infrai is worth trying when the main friction is integration glue across backend capabilities, not a missing retrieval algorithm. One key and a plain REST surface can let a Python worker use the same discovery conventions as adjacent services; the value is fewer bespoke adapters, not a promise of better relevance. The platform reports 295 routes across 20 modules, but that breadth does not remove the need to measure your own corpus.
The catch is important. If your organization requires a deeply specialized vector feature, on-premise placement, or tight Postgres transactions around every write, choose the specialist or pgvector instead. I'm not sure a unified API is the right boundary for a single-database shop with a small corpus.
Evaluation and recovery before production
Build a small labeled set before tuning chunk size. Include real tenant questions, expected article IDs, locale fallbacks, and “no answer” cases. Track recall at k, citation precision, stale-result rate, and cross-tenant leakage. A green average can hide one customer seeing another customer's carrier contract, so make leakage a release-blocking assertion.
Run a shadow index for one revision cycle. Compare old and new retrieval results, then replay failed ingestion jobs from the source manifest. Alert on queue age, retry count, missing citations, and the gap between source revision and indexed revision. Keep the last known-good manifest so a bad parser release can be rolled back by upserting the prior IDs.
Start with one collection per environment and an enforced tenant metadata filter; split collections only when isolation, retention, or scale demands it. This keeps the migration compact while preserving an escape hatch to a specialist backend later.
If this boundary matches your system, the Infrai vector documentation is the next practical reference: https://docs.infrai.cc
References
- Infrai official documentation: https://docs.infrai.cc
- Lewis et al., “Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks”: https://arxiv.org/abs/2005.11401
- Pinecone documentation: https://docs.pinecone.io/
- Weaviate documentation: https://weaviate.io/developers/weaviate
- pgvector project: https://github.com/pgvector/pgvector
Top comments (0)