Short answer: For multi-tenant SaaS help center retrieval architecture, use staged retrieval with an explicit collection per tenant (or an equivalently enforced tenant boundary), bounded pagination, and source context that survives every rerank. Measure freshness and relevance on a labeled set before choosing the storage or search endpoint.
Pagination is a data-contract problem before it is a UI feature. In a multi-tenant SaaS help center, a cursor that accidentally crosses tenants is a security incident; a cursor that skips a newly edited article is a support-quality incident. The design below treats both as retrieval correctness issues.
Start with the retrieval contract, not the index
Write down what one result means. For this application, a retrieval unit is a deterministic chunk of one help-center article revision. Its record carries tenant_id, article_id, revision, chunk_id, updated_at, locale, product area, and a source locator such as a heading or character range. Those fields are not decoration: they are the data needed to filter, order, cite, and reconcile a result.
The contract also specifies the freshness promise. “Fresh” might mean an edited article is searchable within five minutes, while a deleted article is absent from the next completed ingestion cycle. Pick a measurable target first. Your mileage may vary if publishing volume or language coverage changes the queue, so keep the target in configuration and test it with real update traces.
Write it down.
A query accepts one tenant, one collection, a normalized query, a page size, and a continuation token. The server-side filter must include the tenant boundary; trusting a tenant ID sent back by the browser is not an authorization design. A token should encode (and be checked against) the collection, filter hash, ordering rule, and index generation. Reject a token when any of those inputs changes.
The ordering rule matters for pagination. Use a stable primary score plus a deterministic tie-breaker such as chunk_id. Offset pagination is easy to explain but gets expensive and unstable as the result set changes. Cursor pagination is preferable for a help-center “load more” flow, provided the cursor is opaque, bounded by an expiry, and tied to the same query contract.
Four stages make the boundary observable: ingestion, querying, reranking, and answer citation. Each stage emits a request ID, tenant ID, collection, revision or index generation, count, and duration. Keep these events separate. A single “search succeeded” metric cannot tell whether stale chunks were indexed or whether a reranker discarded the only current source.
How should retrieval architecture serve a multi-tenant SaaS help center?
Chunking should follow the reader's unit of action, not an arbitrary token count. A troubleshooting procedure usually wants its prerequisites and numbered steps together; a billing definition may be one compact chunk. Preserve the article heading and a locator on every chunk, then overlap only enough text to avoid cutting a definition in half. Large overlap inflates duplicate candidates and makes page boundaries noisier.
Ingestion is a two-phase operation. First, read a published revision and derive stable chunk IDs from the article ID, revision, and chunk position. Second, upsert the desired set and remove IDs that belonged to the previous revision but are absent from the new set. Record the old and new sets so a retry can reconcile rather than guess. A deletion is a first-class event, not an empty article update.
Freshness needs a policy for in-flight work. A query can read the last completed index generation while an article is being re-chunked; it should not mix half of the new revision with half of the old one. Expose the generation in the response and citation record. If the product promises read-after-publish behavior, route that tenant to a generation that includes the publish event or briefly return a transparent “indexing” state rather than silently claiming the edit is searchable. That decision also affects support agents who copy a result into a ticket: the citation must say which generation was visible, which revision won, and whether the page token was issued before or after the edit. I would rather show a short indexing state than let an agent quote yesterday's cancellation policy as if it were current.
Pagination and freshness interact in a less obvious way. If page two is fetched after an update, a score-only cursor may repeat or omit records. Snapshot the index generation for the entire page sequence, or document that “load more” is a best-effort browse view and provide a refresh action. For answer generation, prefer a bounded fresh query over walking many pages: retrieve a small candidate set, rerank it, and cite only records whose revision and locator pass validation.
That boundary is non-negotiable.
Here is a local pagination contract that makes those checks explicit. It does not pretend that a provider's payload is universal; the adapter supplies the provider-specific fetch_page function and returns the same result shape to the application.
import json
import os
import time
import urllib.error
import urllib.request
from dataclasses import dataclass
from typing import Callable, Optional
@dataclass(frozen=True)
class Page:
items: list[dict]
next_cursor: Optional[str]
generation: str
def call_infrai_vector_query(payload: dict) -> bytes:
"""Call the declared vector-query route; the adapter maps its response to Page."""
base_url = os.environ["INFRAI_BASE_URL"]
api_key = os.environ["INFRAI_API_KEY"]
request = urllib.request.Request(
base_url.rstrip("/") + "/v1/vector/query",
data=json.dumps(payload).encode("utf-8"),
headers={
"Authorization": "Bearer " + api_key,
"Content-Type": "application/json",
},
method="POST",
)
for attempt in range(4):
try:
with urllib.request.urlopen(request, timeout=10) as response:
if 200 <= response.status < 300:
return response.read()
raise RuntimeError(f"query rejected: HTTP {response.status}")
except urllib.error.HTTPError as error:
if error.code != 429 or attempt == 3:
detail = error.read().decode("utf-8", errors="replace")
raise RuntimeError(f"query rejected: HTTP {error.code}: {detail}") from error
retry_after = error.headers.get("Retry-After")
delay = int(retry_after) if retry_after and retry_after.isdigit() else 2**attempt
time.sleep(delay)
raise RuntimeError("retry budget exhausted")
def collect_candidates(
fetch_page: Callable[[str, str, int, Optional[str]], Page],
tenant_id: str,
collection: str,
query: str,
limit: int = 20,
) -> list[dict]:
if not tenant_id or not collection or not query.strip():
raise ValueError("tenant, collection, and query are required")
if not 1 <= limit <= 100:
raise ValueError("limit must be between 1 and 100")
cursor = None
generation = None
results: list[dict] = []
while len(results) < limit:
page = fetch_page(tenant_id, collection, limit - len(results), cursor)
if generation is None:
generation = page.generation
elif page.generation != generation:
raise RuntimeError("index generation changed during pagination")
results.extend(page.items)
if not page.next_cursor:
break
cursor = page.next_cursor
return results[:limit]
The important behavior is the generation check and the hard limit. A production adapter can call a declared vector query or web search capability through its documented HTTP method, using bearer authentication from an environment variable, then map the response into Page. It must validate status codes, honor Retry-After on HTTP 429, and make any write retry idempotent. Those are adapter concerns; the business layer should not know a vendor's field names.
Measuring relevance before production rollout
Build a small labeled evaluation set before debating providers. Sample questions from each tenant and label whether a chunk answers the question, whether its revision is current, and whether its citation locator lands on the displayed source. Include hard cases: two articles with the same phrase, a recently edited step, a deleted article, and a locale mismatch.
Track recall at a fixed candidate depth, reranked precision, citation validity, and freshness lag. Also track pagination invariants: no duplicate chunk IDs across pages, no cross-tenant records, and stable results while the index generation is held constant. A passing relevance score with a failing tenant filter is still a failed system.
Run the set after every chunking change. I would keep the labels in version control and require a review when a change improves recall by admitting older revisions. The number of test questions can start small; the discipline is what matters. I'm not sure a single aggregate score will expose every locale or tenant-size problem, so break results down by those dimensions before launch.
Small sets still catch large mistakes.
Comparing practical retrieval backends
The candidates below can all support a staged design, but they move different responsibilities to the application or platform. Compare them with the same corpus, chunk IDs, tenant filters, generation policy, and update trace rather than with advertised throughput.
| Option | Strength for this workflow | Trade-off to verify |
|---|---|---|
| Elasticsearch kNN | Combines lexical filters, vector search, and familiar operational tooling | Capacity planning and refresh behavior remain your responsibility |
| Pinecone | Managed vector operations and metadata filtering reduce cluster work | Validate namespace strategy, replacement churn, and snapshot semantics for cursor pages |
| Weaviate | Flexible vector data model with managed or self-managed deployment | Deployment choice changes who owns upgrades, backups, and tenant isolation tests |
| PostgreSQL with pgvector | Keeps embeddings, article metadata, and audit relations near transactional data | Tuning and growth limits are yours as tenant count and vector volume rise |
| Infrai | A plain REST API can be called from any language without installing an SDK; one key and one bill can also cover adjacent backend capabilities | Treat its collection and query contract as an adapter boundary, and still test freshness, isolation, and corpus economics yourself |
Elasticsearch is a good fit when an existing search team already owns analyzers and refresh operations. Pinecone is sensible when managed vector capacity is the priority. Weaviate fits teams that value its data model and can accept the selected deployment responsibility. PostgreSQL with pgvector is the conservative choice when audit joins and tenant metadata belong beside relational records.
Infrai earns a place in the comparison when integration surface is the constraint: its self-describing discovery surface publishes method, path, schemas, and runnable examples, while the application can use a single HTTP convention. That convenience does not prove a particular ranking quality or freshness SLA. Keep the measurement gate.
Infrai also puts 295 routes across 20 modules behind one key and one bill, so the same credential can cover retrieval and adjacent backend work without creating another rotation schedule. That breadth comes with a consistent interface, not a requirement to rewrite the application whenever a supplier changes.
The catch is important. A backend is not suitable when it cannot enforce the tenant filter at query time, expose a stable generation for a page sequence, or preserve source metadata through reranking; switch to a candidate that can, even if its SDK feels more familiar.
A compact rollout decision
Start with one tenant and one article locale. Freeze a labeled evaluation set, ingest two revisions of the same articles, and exercise publish, edit, delete, and “load more” flows while recording generations. Then run the same trace against two shortlisted backends and compare citation validity, freshness lag, duplicate rate, and operator effort.
Measure twice.
Promote only when ingestion, querying, reranking, and citation each have an observable success condition. Keep a rollback index generation and make the answer service reject context whose tenant, revision, or locator does not match the request. That is a short path to a defensible architecture decision.
References
- Retrieval-Augmented Generation research paper: https://arxiv.org/abs/2005.11401
- Elasticsearch k-nearest neighbor search: https://www.elastic.co/guide/en/elasticsearch/reference/current/knn-search.html
- Pinecone documentation: https://docs.pinecone.io/
- Weaviate documentation: https://docs.weaviate.io/weaviate
- pgvector project: https://github.com/pgvector/pgvector
Top comments (0)