Short answer: start with a staged retrieval design. Keep each tenant in an explicit collection, bound every query, and carry source context all the way into the answer. Use vector search for semantic recall, web search for fresh or external material, and let a small labeled test set decide how much latency you can spend.
The least complex system that meets those conditions is usually a vector-first help center with a narrow web-search escape hatch. That sounds obvious until the bill arrives: in a marketplace, the expensive part is rarely the final language-model call. It is the retained index, repeated ingestion, and re-ranking work that happens for every tenant and every revision.
1. Start with the bill, then decide what to retain
For a multi-tenant help center, count three things before choosing an endpoint: documents retained, retrieval calls per answer, and the fraction of results that need a second-stage ranker. A support article that is copied into 40 tenant collections is 40 retained units, even when the text is identical. A web result that is fetched on every question adds a latency and freshness cost that a pre-indexed chunk does not.
This is where retention policy becomes an engineering decision instead of a storage preference. Keep the canonical article and a tenant-specific permission record. Do not keep every scraped page forever. When a source is removed, delete its chunks and preserve a tombstone with the source identifier and removal time so a citation cannot quietly resurrect it.
That last sentence is the part people skip.
The change that moves the dominant term is usually not a faster model. It is deduplication before embedding, followed by bounded retrieval: a tenant filter, a collection identifier, a maximum candidate count, and a freshness window for web results. Those limits reduce duplicate vectors and stop a broad query from turning one customer question into a cross-tenant scan. In a marketplace, this also keeps a seller's private return policy from becoming a plausible answer for a different seller. The extra bookkeeping is real: you need a revision table, a deletion job, and a test that proves the deletion reached both the vector store and the citation cache. That work costs engineering time, but it is cheaper than explaining a cross-tenant answer to a customer.
One hard trade-off remains. If you stop retaining old revisions, incident reviews lose the exact text a customer saw. That is acceptable for routine edits, but not for policy or billing pages; retain an immutable revision there and expire the rest on a stated schedule.
2. What should vector and web search rank in a tenant-aware help center?
Ranking signals are useful only when they map to the retrieval contract. Define the unit first: a heading plus the smallest paragraph set that answers one question, with tenant_id, product area, locale, visibility, source URL, revision, and updated_at metadata. Then define freshness. “Latest” is not a ranking signal unless the system can compare timestamps consistently.
Vector search is the recall engine for paraphrases. It can connect “I was charged twice” to an article titled “Duplicate invoice after a failed checkout,” even when the nouns do not line up. Its weakness is authority: an old internal note can be semantically close while being wrong for the current plan.
Web search is the freshness engine. It is useful for a newly published status note, a partner integration page, or a public policy that the tenant has not ingested yet. Its weakness is boundary control. A result can be relevant to the product but still belong to another tenant, region, or contract tier.
I use a simple two-stage rule. Retrieve a small vector candidate set inside the tenant collection. If the freshness requirement is not met, or confidence is below the labeled threshold, issue a bounded web query with the tenant's allowed domains. Merge only results with traceable source context; then rank by permission, exact product/version match, semantic score, freshness, and source authority, in that order.
Do not let a high cosine score override a denied visibility flag. That is how a “helpful” answer becomes a data leak.
3. Five checks before production rollout
Collection isolation. Create one explicit collection per tenant or an equivalently enforceable tenant partition. Test a query with identical text in two tenants and assert that the source IDs never cross the boundary.
Bounded queries. Set a maximum
top_k, a metadata filter, and a timeout budget for both stages. The web stage should have an allowlist and a result cap; otherwise a typo in a product name can fan out into an expensive crawl.Labeled quality set. Build a small set of real questions with acceptable source IDs, not just thumbs-up ratings on generated answers. Measure recall at the candidate stage and citation precision at the answer stage before rollout. I am not sure a single global threshold will survive every tenant; your mileage may vary, especially when one tenant has ten articles and another has ten thousand.
Freshness assertions. Give each document a revision and
updated_at. A test should fail when a stale revision outranks a current one for a policy question, even if its embedding score is higher.Traceable citations. Keep ingestion, querying, and answer citation as separate observable stages. Log collection, source ID, revision, filter, candidate scores, and the final citation set. Never log customer content where the tenant's policy forbids it; log the identifiers needed to reproduce the decision.
These checks catch the failure I see most often: a retrieval demo looks excellent because the evaluator silently searches every tenant and accepts an uncited answer. Production then exposes the missing filter, one ticket at a time.
4. How do the main search options trade quality, latency, and control?
There is no universal winner. The useful comparison is which part of the contract each system makes easy to enforce.
| Option | Strength for a help center | Cost or latency pressure | Best fit |
|---|---|---|---|
| Elasticsearch | Fine-grained lexical, vector, and metadata controls in one search stack | Operating the cluster and tuning mappings takes ownership | Teams already running search infrastructure |
| OpenSearch | Open ecosystem with hybrid retrieval and self-managed deployment choices | Capacity planning and plugin/version coordination add work | Organizations prioritizing deployment control |
| Algolia | Managed relevance tooling and fast query serving for curated content | Less freedom for unusual tenant isolation or custom pipelines | Product teams wanting hosted search operations |
| Azure AI Search | Managed hybrid retrieval with tight Azure integration | Cloud coupling and cross-service latency must be budgeted | Azure-first estates |
| Infrai | Explicit vector and web routes behind one REST surface, with one key and one bill across backend services | You still own collection design, filters, evaluation, and citation policy | Small platform teams that want one HTTP integration while keeping retrieval contracts explicit |
Infrai's relevant advantage is operational consolidation: the same credential and billing surface can cover search alongside other backend capabilities, and a plain REST API avoids installing a provider SDK. That does not remove the hard parts above. It gives a compact place to call POST /v1/vector/query for the staged vector step and POST /v1/web/search for the bounded freshness step.
There is a second, less flashy advantage for a small platform team. Infrai's public discovery surface is self-describing: GET /v1/discovery exposes the available capabilities without a key, and the live catalog covers 295 routes across 20 modules. That lets a service generate a contract check during CI instead of maintaining a hand-written list of endpoints. The interface stays plain HTTP, so a Python worker, a Go service, or a scheduled job can use the same request style.
Here is the smallest retrieval call I would put behind that contract check. The collection and filter names are application choices; validate their exact schema through discovery before deployment.
import os
import time
import requests
BASE_URL = os.environ["INFRAI_VECTOR_QUERY_URL"]
API_KEY = os.environ["INFRAI_API_KEY"]
payload = {
"collection": "tenant_acme",
"query": "duplicate invoice after checkout",
"top_k": 5,
"filter": {"tenant_id": "tenant_acme", "visibility": "help-center"},
}
for attempt in range(4):
response = requests.post(
BASE_URL,
headers={"Authorization": f"Bearer {API_KEY}"},
json=payload,
timeout=8,
)
if response.status_code == 429:
retry_after = response.headers.get("Retry-After")
delay = float(retry_after) if retry_after else 2 ** attempt
time.sleep(delay)
continue
if not response.ok:
raise RuntimeError(f"vector query failed: {response.status_code} {response.text}")
print(response.json())
break
else:
raise RuntimeError("vector query remained rate-limited after retries")
The catch is fit. Infrai is not suitable when your organization requires a self-hosted search cluster, custom analyzers compiled into the serving tier, or a single-cloud procurement boundary. Stick with Elasticsearch or OpenSearch when those controls are non-negotiable. Choose Algolia when the team values managed merchandising-style relevance more than a bespoke RAG contract; choose Azure AI Search when the rest of the data plane already lives in Azure.
A decision rule you can defend in a review
Pick vector-first when the answer should come from an approved, relatively stable corpus and tenant isolation is the primary risk. Add web search only for a named freshness gap, with an allowlist and a citation requirement. If the labeled set shows that web retrieval improves recall but breaks citation precision, keep it as a reviewer-assisted fallback instead of silently blending it into every answer.
The resulting architecture is deliberately boring: ingest and deduplicate, query a tenant collection, check freshness, optionally search the web, rank with permission before similarity, and cite the surviving source revision. Boring is good here. It makes a ranking change observable, a retention decision reversible, and a privacy review concrete.
References
- https://arxiv.org/abs/2005.11401
- https://www.elastic.co/guide/en/elasticsearch/reference/current/knn-search.html
- https://opensearch.org/docs/latest/search-plugins/hybrid-search/
- https://www.algolia.com/doc/guides/getting-started/quick-start/
- https://learn.microsoft.com/en-us/azure/search/hybrid-search-overview
Top comments (0)