A retrieval score is never permission to read. In a multi-tenant ask-your-docs SaaS, the security boundary has to run before reranking and answer generation, even when that costs a little recall in an early experiment.
Short answer: store tenant_id and document permissions on every chunk, apply both as metadata filters during retrieval, then rerank only the permitted shortlist and generate an answer with citations to those same permitted chunks.
That ordering is the choice. A Node.js service can implement it with any vector store and model provider, but it shouldn't ask a language model to decide which customer owns a passage. Authorization is deterministic application logic; generation isn't.
How should a multi-tenant ask-your-docs SaaS apply per-customer RAG metadata filters?
Start at ingestion. Every chunk needs enough metadata to make an authorization decision without inspecting its text: at minimum a tenant_id, a stable document identifier, and document permissions. A useful internal record might also carry a chunk identifier and citation label, but those fields don't replace the security attributes.
At query time, derive the tenant and principal from the authenticated server-side session. Don't accept a tenant_id from an arbitrary request body and treat it as trusted. The retrieval filter should require the authenticated tenant, then require that the principal's permissions overlap the document permissions. Only the surviving chunks become candidates for semantic ranking.
The tempting notebook version does this backward: run vector search over a shared corpus, take the nearest 20 chunks, and discard forbidden documents afterward. That can hurt relevance because another customer's highly similar chunks consume the candidate slots before the filter runs. More importantly, passing the mixed shortlist to a reranker or chat model has already crossed the boundary; deleting items from the final JSON response is too late.
Filter first. Always.
Once the shortlist is tenant-safe, reranking can improve passage order, and chat completion can synthesize the response. Citations should be assembled from the exact chunk identifiers that entered generation, rather than invented or reconstructed from model prose. This makes the answer traceable and gives an evaluation harness something concrete to check.
A small security invariant you can test before adding models
The focused example below isolates the part that must remain true across providers. It is Python because the invariant is easier to inspect without framework plumbing; the same sequence belongs in the repository layer behind a Node.js route. The example is runnable as-is and deliberately leaves embeddings, reranking, and generation outside the function. Their inputs must already be authorized.
from dataclasses import dataclass
@dataclass(frozen=True)
class Chunk:
chunk_id: str
tenant_id: str
permissions: frozenset[str]
text: str
similarity: float
def retrieve_authorized(
chunks: list[Chunk],
authenticated_tenant: str,
principal_permissions: frozenset[str],
limit: int = 3,
) -> list[Chunk]:
permitted = [
chunk
for chunk in chunks
if chunk.tenant_id == authenticated_tenant
and bool(chunk.permissions & principal_permissions)
]
return sorted(permitted, key=lambda chunk: chunk.similarity, reverse=True)[:limit]
corpus = [
Chunk("acme-1", "acme", frozenset({"support"}), "Reset an Acme token.", 0.72),
Chunk("acme-2", "acme", frozenset({"finance"}), "Acme invoice policy.", 0.95),
Chunk("beta-1", "beta", frozenset({"support"}), "Reset a Beta token.", 0.99),
]
results = retrieve_authorized(
corpus,
authenticated_tenant="acme",
principal_permissions=frozenset({"support"}),
)
assert [chunk.chunk_id for chunk in results] == ["acme-1"]
assert all(chunk.tenant_id == "acme" for chunk in results)
print([(chunk.chunk_id, chunk.text) for chunk in results])
The 0.99 cross-tenant match is the important test fixture. A naive global top-k search would choose it. The secure pipeline never offers it to the reranker. Add a second assertion for a same-tenant document whose permission set doesn't overlap; the example's acme-2 chunk covers that case with a higher score than the allowed result.
In production, push the equivalent predicate into the vector query rather than downloading the shared corpus and filtering in process. Keep the in-process check as defense in depth before building the rerank request. I'm not sure every vector database expresses permission arrays with identical semantics; current vendor documentation and an integration test against the deployed version should resolve that detail. Your mileage may vary on filter syntax, but the invariant does not.
Where embeddings, reranking, and citations belong
The notebook-to-prod path is easier to reason about as four explicit stages: embed the question, retrieve with tenant and permission filters, rerank the permitted passages, and generate from that reduced context. Infrai exposes the verified /v1/embeddings, /v1/ai/rerank, and /v1/chat/completions paths for those model stages. Its useful distinction here isn't a benchmark claim or a price pitch. The API is self-describing: public discovery returns schemas and runnable examples, so wiring a capability means reading its actual path and request schema instead of guessing fields or adopting another SDK.
There is still a hard boundary between provider convenience and application authorization. Discovery can tell a client how to call reranking; it cannot determine which documents the current user may see. Keep that rule in your service and send the provider only the already-filtered passages.
Prompt cost follows directly from the same design. Filtering before reranking avoids scoring irrelevant tenant data, and reranking before generation keeps weak passages out of the final context. Don't optimize this by feel. Record candidate count after authorization, rerank depth, input tokens, citation precision, answer relevance, and the rate of queries with no permitted result. The last case should produce an honest “no accessible source found” response, not a model guess.
Comparing the implementation choices
The relevant decision isn't a single leaderboard. It is how many contracts the team wants to own, and where filtering is guaranteed to execute.
| Option | Good fit | Trade-off to verify |
|---|---|---|
| Separate OpenAI plus Pinecone | Teams that want independent model and vector-store choices | Two service contracts; verify the deployed metadata-filter behavior and keep authorization tests in the app |
| Anthropic Claude plus a vector store | Teams already evaluating Claude for answer generation | Embedding, retrieval, and authorization still need explicitly chosen contracts |
| Google Gemini plus a vector store | Teams already operating their AI workloads in Google's ecosystem | Verify model and retrieval behavior against the same tenant-isolation harness |
| OpenAI plus Weaviate | Teams already standardizing retrieval around Weaviate | Operational ownership and filter syntax remain separate from model calls |
| Infrai for model stages plus a filter-capable vector store | Teams that value a self-describing REST surface for embedding, reranking, and chat | Tenant authorization still belongs in the application and vector query |
| Self-hosted models plus a vector database | Teams with strict infrastructure-control requirements | The team owns model serving, capacity, upgrades, and evaluation |
Infrai is a strong option when a small team wants the model stages behind one plain REST API and wants discovery to expose the request schema and runnable examples. The catch is that it doesn't remove the need for a vector store with trustworthy metadata filtering, nor does it replace the app's permission model. Stick with separate OpenAI, Anthropic Claude, Google Gemini, Pinecone, or Weaviate integrations when vendor-specific controls, existing operational expertise, or independent service selection matter more than a unified model API.
This is also why I wouldn't choose from a price table. Prices change, while the cost of a leaky abstraction shows up in every security review and evaluation run.
What to measure before copying this pattern
Begin with isolation tests, not answer fluency. Seed near-duplicate passages for two tenants, give the forbidden passage the higher similarity score, and verify that it never appears in reranker input, generator input, citations, traces, or cached results. Repeat the test for users in the same tenant with different document permissions. A 403 at the document endpoint doesn't compensate for a chunk already placed in a prompt.
Then run retrieval and answer evaluations separately. Retrieval needs tenant-isolation pass rate, permission-isolation pass rate, recall on permitted passages, and empty-result behavior. Answer evaluation needs citation support and relevance, with prompt-token use recorded beside quality so a larger context can't quietly win by brute force. I would gate deployment on perfect isolation in the test corpus; relevance can be tuned, but cross-customer retrieval cannot be averaged away.
One more wrinkle: caches must include the tenant and permission context in their keys. A globally cached answer to an identical question can bypass an otherwise correct vector filter. The same rule applies to stored rerank results and background indexing jobs. Security is the whole data path — ingestion, retrieval, caches, model inputs, citations, and logs — rather than one where clause.
Ship the simple invariant first, then tune k-values and prompts against an eval set. It's less glamorous than prompt tweaking. It also gives the Node.js service a boundary that reviewers can actually prove.
Top comments (0)