Short answer: enforce tenant scope before vector search, keep only the text needed for retrieval, and make every answer auditable. In a multi-tenant SaaS help center, privacy is a query-construction problem as much as an encryption problem. A fast index that can return another customer’s article is not a retrieval system; it is a data leak with good latency.
The application scenario here is product-content search: customers ask about billing, API limits, or workspace settings, and the service retrieves passages for a grounded answer. The main engineering tension is retrieval quality versus latency, but privacy controls decide which candidates are even eligible.
Scope first.
Start with the bill and the retention boundary
The bill is usually dominated by retained representation, not by the few milliseconds spent checking a tenant id. Every published article can exist as source HTML, normalized text, chunks, embedding vectors, metadata, query logs, cached prompts, and backups. Keeping all of those forever increases storage, backup, and deletion work. It also multiplies the places where a privacy review must look.
I model each stored item as a tuple: (tenant_id, document_id, revision, chunk_id, payload, retention_until). The tenant_id is immutable. A revision is append-only until its replacement is indexed and tested; then old chunks can expire. Query logs should keep timing, result ids, and policy decisions for a short audit window, while raw user questions need a separate, shorter retention class because they may contain account details.
Here is the retention decision I use for a help center:
| Data | Keep for | Reason | Delete action |
|---|---|---|---|
| Published chunk text | Until document deletion plus 30 days | Rebuild and audit window | Tombstone, then purge |
| Embedding vector | Same as chunk text | Re-index without source fetch | Delete by tenant and revision |
| Access decision log | 30 days | Investigate cross-tenant attempts | Aggregate, then erase ids |
| Raw query text | 7 days | Debug relevance | Hash or remove content |
| Backup snapshot | 35 days | Disaster recovery | Expire by snapshot policy |
The catch is that shorter retention makes incident reconstruction harder. A team that deletes vectors immediately may have to re-embed from a source system that has already changed. I would rather retain a signed document hash and revision metadata than keep a second copy of sensitive prose. That choice costs some recovery convenience, and it is deliberate.
Deletion is where the accounting becomes concrete. Suppose a tenant replaces a 40-page API guide after discovering that an example exposed an internal hostname. The live index may hold 120 chunks, a duplicate set in a warm cache, and three snapshots. A correct workflow marks the revision unavailable before deleting bytes, records the revision hash without retaining the prose, evicts cache entries keyed by that revision, and waits for each snapshot's expiry rule. During that window, retrieval returns the replacement revision or no result; it never falls back to the old text merely because the new embedding is still warming. The team can then prove what was removed by replaying a query against the post-deletion index and by checking that the old revision id is absent from logs after the audit window. This costs some storage while the expiry clocks run, but it prevents a “deleted” document from surviving in a less visible layer.
How should multi-tenant SaaS retrieval enforce privacy before ranking?
Use two gates. The first gate runs in the application before the retrieval request: authenticate the caller, resolve the tenant from a trusted session claim, and reject a request whose body tries to name another tenant. The second gate runs inside the index query as a hard metadata filter. Ranking, reranking, and prompt instructions happen after both gates.
Do not accept tenant_id from a browser as authority. Treat it as a display hint at most. The server should derive the scope from an access token and a membership lookup, then pass a normalized scope object to the adapter.
from dataclasses import dataclass
from typing import Protocol
@dataclass(frozen=True)
class Scope:
tenant_id: str
actor_id: str
allowed_collections: tuple[str, ...]
class VectorIndex(Protocol):
def search(self, *, text: str, filters: dict, limit: int) -> list[dict]: ...
def retrieve(index: VectorIndex, question: str, scope: Scope, limit: int = 8) -> list[dict]:
if not question.strip():
return []
if not scope.tenant_id or not scope.allowed_collections:
raise PermissionError("retrieval scope is empty")
# The filter is mandatory and constructed server-side.
filters = {
"tenant_id": {"equals": scope.tenant_id},
"collection": {"in": list(scope.allowed_collections)},
"status": {"equals": "published"},
}
hits = index.search(text=question, filters=filters, limit=limit)
return [h for h in hits if h.get("tenant_id") == scope.tenant_id]
That final list check is not a substitute for an index filter. It is a tripwire for adapter mistakes and should emit a security event if it ever removes a hit. A useful test uses two tenants with identical article titles and asserts that every returned id belongs to the caller’s scope. Run it against the real adapter, because a mock can quietly ignore filters.
Build the index around deletion, not just ingestion
Ingestion should be idempotent and revision-aware. Normalize HTML, split on semantic boundaries, attach tenant and collection metadata, and write a new revision. Only after the new revision passes retrieval tests should it become published. This avoids a half-updated help center where a title points at new text but an old vector remains searchable.
A deletion request is a workflow: mark the document as deleting, block it from new queries, remove chunks and vectors by the immutable document key, invalidate caches, and verify that a sample query returns no deleted ids. Backup expiry must be part of the same policy; deleting the live index while retaining an unbounded snapshot is not deletion.
Failure modes deserve names. The dangerous ones are filter omission, stale cache, revision collision, metadata truncation, and log over-collection. Metadata truncation is especially quiet: if a connector drops tenant_id because its schema allows only a fixed set of fields, the query may still return plausible results. Add a schema check at write time and fail closed when required metadata is absent.
For quality, measure recall within a tenant, not across the whole corpus. A global benchmark can look excellent while one tenant’s terminology is poorly represented. Keep a small, labeled question set per tenant, record the expected document ids, and compare recall at k=5 and k=10. Latency should be reported at the same filter cardinality, since a tenant with 20 documents is not comparable to one with 20 million.
Keep retrieval evidence separate from generation
Pass only selected chunks to the generation step, with source ids and revision numbers attached. The model should never receive neighboring tenants’ metadata “for context.” A citation is useful only if the citation resolver checks the same scope as retrieval. Otherwise a user can receive a valid-looking link to a document they cannot open.
Cache keys need the tenant and policy version: tenant_id:policy_hash:query_hash. A cache hit must still pass authorization, because memberships change faster than cache TTLs. I use a short TTL for query results and a longer TTL for immutable published chunks, then invalidate both on deletion.
I'm not sure a single latency target is meaningful here. A strict privacy filter, per-tenant reranking, and an authorization lookup may add work, but removing any one of them changes the risk model. Set a budget for each stage, expose p50 and p95 timings, and make timeout behavior return no answer rather than unscoped candidates.
Decide when this architecture is the wrong fit
This design suits a shared service with many small or medium help centers and a common retrieval pipeline. It is not suitable when regulations require physical separation, customer-managed keys, or tenant-specific residency that your index cannot provide. In those cases, use separate indexes or regions, even though operations become heavier.
It is also a poor fit for highly dynamic private data where every query must reflect a transactional row-level permission check. Keep an ordinary database query in the authorization path and use retrieval only for text that has already passed that check. Stick with lexical search when exact identifiers, version strings, and error codes dominate; semantic ranking can blur those tokens and add latency without improving answers.
The decision rule is simple: choose the smallest searchable representation that meets measured recall, and make scope enforcement independent of the ranking engine. If a team cannot demonstrate tenant-isolation tests, deletion verification, and bounded logs, it is not ready to trade latency for richer retrieval.
References
- https://arxiv.org/abs/2005.11401
- https://www.rfc-editor.org/rfc/rfc7519
- https://www.w3.org/TR/trace-context/
Top comments (0)