Short answer: use staged retrieval with explicit collections, bounded queries, and source context that survives into the cited answer. For a multi-tenant SaaS help center, this makes retrieval quality measurable while keeping latency predictable.
Start with the bill and the retention decision
The bill is shaped first by how much text enters ingestion and how often it is searched. In a help center, repeated boilerplate is an easy multiplier: navigation, legal footers, and a contact paragraph can appear in thousands of chunks. Deduplicate those fragments before embedding. Then set a maximum candidate count for every query. An unbounded top-k quietly turns a relevance experiment into a latency and storage problem.
Count the expensive units before choosing an index.
Keep it bounded.
Retention is part of schema design, not a housekeeping ticket. Keep the canonical URL, tenant id, product area, locale, document revision, and ingestion timestamp. Keep chunk text or a content hash long enough to reproduce a citation. I do not keep every scraped page forever; stale copies make answers sound authoritative when they are not. The catch is operational: when a customer disputes an answer, less history means less forensic context. Pick a retention window that support and compliance can defend, and record the policy next to the document.
How can a multi-tenant SaaS help center retrieval architecture control latency?
Treat a retrieval unit as a versioned chunk, not an entire article. A useful identity is (tenant_id, collection, document_id, revision, chunk_id). The collection is an explicit namespace for an isolation boundary, such as a product line or region. tenant_id remains a required filter even when the vector store offers namespaces. Defense in depth matters when a support agent changes teams.
Metadata should be typed and unexciting: product, locale, visibility, updated_at, and source_url. Do not encode these values in prose and hope an embedding learns them. Filter first, then run vector search with a bounded top_k; a small candidate set gives the answer stage a known latency budget. If a query is ambiguous, ask for the product or locale instead of widening the tenant scope silently.
Freshness needs its own field and policy. A release-note chunk may expire in days; a security guide may require review before it is eligible again. Record that rule with the document, and have the query reject expired revisions. Writing “latest” in a prompt is not a freshness policy.
I once treated a whole troubleshooting article as one retrieval unit. The first result had the right product name but the wrong version, so the generated answer repeated an obsolete OTP timeout. I traced ingestion, vector search, and citation logs; each stage reported success, yet the source revision was missing from the filter. That was a schema error, not a model failure. Smaller versioned chunks plus a revision filter made the failure visible, and a labeled question caught the regression before release. The extra fields feel fussy until someone has to explain a bad answer to a customer.
Keep ingestion, querying, and citation observable
Give the pipeline three stages and three log records. Ingestion records source, revision, chunk count, and an upsert request id. Querying records the normalized query, filters, candidate ids, scores, and elapsed time. Answer generation records which candidate spans became citations. Carry one request id through all three. A trace beats a dashboard full of averages when a tenant reports a wrong article.
Web-backed context is enrichment, not a bypass around tenant policy. Search public release notes, scrape only selected pages, normalize them into the same chunk schema, and tag their provenance. Do not merge those results into a private tenant collection without an allow-list and an ownership decision. In practice, that means the ingestion worker stores the fetched page separately, checks its tenant and product tags, and only then emits versioned chunks; the query service never reaches back to the web on a cache miss, because doing so would make both latency and data boundaries unpredictable. This separation also gives compliance a concrete record of which public source was admitted and why.
This minimal Python example shows the write and read stages with the verified vector paths. The wrapper in a real service should add status handling, exponential backoff for HTTP 429, honor Retry-After, and reuse the same idempotency key for a retried write.
import os
import time
import requests
BASE = os.environ["INFRAI_BASE_URL"] # set to the provider's v1 base URL
HEADERS = {"Authorization": f"Bearer {os.environ['INFRAI_API_KEY']}"}
record = {
"id": "acme:docs:billing:v7:chunk-03",
"text": "Refunds are available for annual plans within 30 days.",
"metadata": {
"tenant_id": "acme",
"collection": "help-center",
"document_id": "billing",
"revision": 7,
"locale": "en-US",
"visibility": "customer",
"source_url": "https://example.com/docs/billing",
},
}
upsert = requests.post(
f"{BASE}/vector/upsert",
headers={**HEADERS, "Idempotency-Key": "acme-billing-v7"},
json={"collection": "acme-help", "records": [record]},
timeout=15,
)
upsert.raise_for_status()
query = requests.post(
f"{BASE}/vector/query",
headers=HEADERS,
json={
"collection": "acme-help",
"query": "How do annual-plan refunds work?",
"top_k": 5,
"filter": {"tenant_id": "acme", "locale": "en-US", "visibility": "customer"},
},
timeout=15,
)
query.raise_for_status()
print(query.json())
The example sends the provider credential only to the API base URL. Citation code should retain the returned source metadata; it should never invent a URL from a document title. For writes, idempotency prevents a network retry from duplicating a chunk. For reads, surface non-2xx response bodies so an operator can distinguish a bad filter from an empty result.
Compare the choices on retrieval behavior
There is no universal winner. Elasticsearch brings mature lexical search and filtering, Pinecone offers a focused managed vector index, and Weaviate combines vector and schema concepts with hybrid options. A relational database with a vector extension can be the better fit when transactional ownership and joins dominate. Run the same labeled questions and tenant filters against each option.
| Option | Help-center strength | Trade-off to test |
|---|---|---|
| Elasticsearch | Strong lexical search, filters, and operational tooling | More tuning surface as vector and hybrid ranking grow |
| Pinecone | Focused managed vector workflow and straightforward scaling | Your application still owns metadata and text-search behavior |
| Weaviate | Schema-aware vector workflow with hybrid retrieval | Module and cluster choices add operational decisions |
| REST aggregation layer | One contract can cover web enrichment and vector stages | You still own evaluation, tenant policy, and retention |
Infrai fits the last row because it offers one REST API and one key across 295 routes in 20 modules, so adding web search or scraping beside vector operations is another HTTP call rather than another SDK integration. Its positioning is one key, one bill, and any language or runtime can call the plain API without installing an SDK. The API is genuinely self-describing, and its public discovery surface helps validate request shapes before rollout. That reduces integration surface; it does not prove higher relevance.
Stick with a dedicated search stack when you need bespoke ranking, strict data-residency controls, or workloads outside the aggregation layer's supported capabilities. Choose the common REST surface when consistent contracts across web and vector stages matter more than owning every ranking primitive.
Ship only after a small evaluation set
Before production, label a few dozen real tenant questions with acceptable source spans and an allowed freshness window. Track recall of the expected chunk, citation correctness, and p95 latency separately. One aggregate score hides the failure support will report: “the answer was fluent, but it cited the wrong revision.”
Roll out in stages. Shadow traffic first and compare candidates; enable answers for one collection next; expand by tenant only after reviewing misses. I am not sure any generic benchmark predicts your customers' vocabulary. Your mileage may vary; the labeled set and latency budget are the evidence.
Top comments (0)