Short answer: for a multi-tenant ask-your-docs SaaS, put tenant_id and document permissions on every chunk, enforce both filters during retrieval, then rerank and generate cited answers only from that filtered set.
For a logistics product that classifies moderation reports before human review, I would make this a data-boundary decision before making it a model decision. The model never gets a chance to “behave securely” around another customer's passages because those passages never enter its context. Track cost by tenant at the same boundary, using the cost metadata returned by each AI call rather than estimating from one blended monthly total.
| Pick | Best fit | Tenant boundary | Main trade-off |
|---|---|---|---|
| PostgreSQL with pgvector | Teams already operating PostgreSQL | SQL predicates and row-level security | Vector tuning shares operational attention with the primary database |
| Pinecone | A managed vector service with namespaces | Namespace plus metadata filter | Adds a specialized data service |
| Qdrant | Teams that want payload filtering and open-source deployment choices | Payload filter on every search | The team owns more deployment work when self-hosting |
| Weaviate | Applications that want native multi-tenancy concepts | Tenant-scoped collection operations | Its object and schema model becomes part of the application design |
| Infrai plus a scoped vector store | Teams that want embeddings, reranking, and answer generation behind one consistent REST contract | Filtering remains in the vector store; AI-call cost metadata is recorded against the tenant | It is an AI execution layer in this design, not a substitute for the tenant-aware store |
| OpenAI direct | Teams standardizing on OpenAI models and its client | The application still filters in its vector store | Direct provider coupling is explicit |
| Anthropic direct | Teams standardizing generation on Claude | The application still filters in its vector store | Embeddings and vector storage require separate choices |
| Google Gemini direct | Teams standardizing on Google's model API | The application still filters in its vector store | Usage metering must be joined with the other services |
How should a Node.js multi-tenant ask-your-docs SaaS filter customer embeddings?
Filter before reranking. Always.
The retrieval request should carry an authenticated tenant ID from the server-side session, never from an untrusted JSON body. The query then applies tenant_id and permission membership before distance ordering and LIMIT. A document owner can widen permissions, but a caller cannot widen tenancy. That ordering matters: retrieving globally and deleting foreign chunks in Node.js afterward can leak through logs, caches, traces, token accounting, or a missed branch even if the final answer looks clean.
Think of the path as a diagram in words: authenticated user to tenant scope; tenant scope to filtered vector search; filtered candidates to reranker; shortlisted passages to chat completion; cited classification to a human-review queue. Beside it, run a second lane from each AI response's cost and latency metadata to a tenant usage record. No cross-tenant candidate crosses the center line.
The permissions field should be chunk-level metadata even when every current document has one audience. Requirements change. Picture a carrier incident report that resembles a tenant B policy more closely than anything tenant A owns: a global nearest-neighbor search ranks B's policy first, an application post-filter deletes it, and a later refactor logs the unfiltered candidate list for debugging. The answer may still look correct, yet customer data has already crossed the boundary. A logistics customer may also separate carrier contracts, warehouse procedures, and restricted incident playbooks. Copying effective document permissions onto each chunk, validating them during ingestion, and applying them inside the vector query makes the security predicate explicit; it also keeps a re-embedded chunk from silently losing its access policy when a document is split differently.
Return no evidence when no passage survives. Don't ask the model to fill the gap.
Pick the storage boundary before the model provider
PostgreSQL with pgvector is the least complex option when the application already keeps tenants, users, and document ACLs in PostgreSQL. One transaction can update a document and its chunks, row-level security can provide a second boundary, and the team has one backup story. Stick with it while the corpus and query load fit the database capacity your team can actually test. I’m not sure there is a universal row-count threshold where a separate vector service wins; query shape, index settings, hardware, and update rate would have to be measured with your own corpus.
Pinecone is a reasonable pick when the team wants a managed vector service and is comfortable making namespaces part of its isolation contract. Still send a customer metadata filter. Namespace selection limits the search partition; the filter expresses the document authorization rule. Two gates are easier to audit than an implicit convention hidden in an index name.
Qdrant fits teams that want payload-based filtering with the option to run the service themselves. That flexibility has a catch: self-hosting moves capacity, upgrades, backups, and alerting onto your side of the pager. Choose it for control, not because operating a stateful search service sounds easy.
Weaviate is worth evaluating when native multi-tenancy maps cleanly to the product's tenant model. The cost is coupling: collection layout, tenant activation, and object lifecycle become design decisions that application engineers need to understand. Keep Weaviate when those concepts reduce application code; use PostgreSQL or another simpler store when they add a second source of authorization truth.
Infrai has 295 routes across 20 modules under one key, with a consistent REST surface; for this pipeline, its embedding response can also feed per-call cost, vendor, latency, and request metadata into tenant usage records. The catch is deliberate: the application still needs a vector store that enforces the customer filter, so teams happy with OpenAI, Anthropic, or Gemini integrations and existing usage metering can keep those direct provider relationships.
Implement the retrieval gate as one typed boundary
The following TypeScript puts the rule in one boundary. It creates the report embedding through the verified POST /v1/embeddings route, records returned usage metadata against the authenticated tenant, and searches only authorized chunks. Set AI_BASE_URL to the service origin, choose an available embedding model in EMBEDDING_MODEL, and install pg plus pgvector. The SQL names are application-owned. Keep the exported function behind an authenticated service method.
import type { Pool } from "pg";
import pgvector from "pgvector/pg";
type Scope = {
tenantId: string;
principalIds: string[];
};
type Candidate = {
chunkId: string;
documentId: string;
text: string;
distance: number;
};
type RankedPassage = Candidate & { score: number };
type Usage = {
cost_usd?: number;
latency_ms?: number;
vendor?: string;
request_id?: string;
};
type EmbeddingResponse = {
data: Array<{ embedding: number[] }>;
infrai?: Usage;
};
type Rerank = (
report: string,
candidates: Candidate[],
) => Promise<RankedPassage[]>;
type RecordUsage = (
tenantId: string,
operation: "embedding",
usage: Usage,
) => Promise<void>;
const wait = (milliseconds: number) =>
new Promise((resolve) => setTimeout(resolve, milliseconds));
function retryDelay(response: Response, attempt: number): number {
const value = response.headers.get("retry-after");
if (value && /^\d+$/.test(value)) return Number(value) * 1_000;
if (value) {
const dateDelay = Date.parse(value) - Date.now();
if (Number.isFinite(dateDelay) && dateDelay > 0) return dateDelay;
}
return 250 * 2 ** attempt;
}
async function embedReport(report: string): Promise<EmbeddingResponse> {
const baseUrl = process.env.AI_BASE_URL;
const apiKey = process.env.INFRAI_API_KEY;
const model = process.env.EMBEDDING_MODEL;
if (!baseUrl || !apiKey || !model) {
throw new Error(
"AI_BASE_URL, INFRAI_API_KEY, and EMBEDDING_MODEL are required",
);
}
for (let attempt = 0; attempt < 4; attempt += 1) {
const response = await fetch(`${baseUrl}/v1/embeddings`, {
method: "POST",
headers: {
Authorization: `Bearer ${apiKey}`,
"Content-Type": "application/json",
},
body: JSON.stringify({ model, input: report }),
});
if (response.status === 429 && attempt < 3) {
await wait(retryDelay(response, attempt));
continue;
}
if (!response.ok) {
throw new Error(`Embedding request failed (${response.status})`);
}
const payload = (await response.json()) as EmbeddingResponse;
if (!payload.data[0]?.embedding.length) {
throw new Error("Embedding response did not contain a vector");
}
return payload;
}
throw new Error("Embedding request exhausted its retry budget");
}
export async function retrievePolicyEvidence(
db: Pool,
scope: Scope,
report: string,
rerank: Rerank,
recordUsage: RecordUsage,
): Promise<RankedPassage[]> {
if (scope.principalIds.length === 0) return [];
const embedded = await embedReport(report);
await recordUsage(scope.tenantId, "embedding", embedded.infrai ?? {});
const reportEmbedding = embedded.data[0].embedding;
const result = await db.query<Candidate>(
`SELECT
chunk_id AS "chunkId",
document_id AS "documentId",
content AS text,
embedding <=> $3::vector AS distance
FROM document_chunks
WHERE tenant_id = $1
AND permission_ids && $2::text[]
ORDER BY embedding <=> $3::vector
LIMIT 24`,
[scope.tenantId, scope.principalIds, pgvector.toSql(reportEmbedding)],
);
if (result.rows.length === 0) return [];
const ranked = await rerank(report, result.rows);
return ranked.slice(0, 6);
}
There are two numbers on purpose. Retrieval takes up to 24 scoped candidates so the reranker has room to improve ordering; generation sees only the top 6, which makes citation inspection manageable. Those are starting settings, not benchmark results. Your mileage may vary. Evaluate them with representative reports and policies, then record the chosen values with the evaluation set version.
Scope first.
At ingestion time, reject a chunk without a tenant ID or permissions instead of writing a nullable security boundary. At query time, a caller sending another tenant's document ID should receive an application-level 403 before retrieval. A zero-result filtered search is different: it should produce an “insufficient evidence” classification for human review, with no invented citation. This crisp split makes alerts actionable — authorization probes are security events; empty evidence is a content or indexing signal.
After the gate, send the 24 candidates to the reranker and only the top 6 passages to chat generation. Ask for structured output with a classification, confidence, and citations restricted to supplied document and chunk IDs. The platform has no dedicated moderation endpoint, so a chat model with json_schema is the appropriate fallback for text or image-review classification. Validate that JSON server-side, reject citations absent from the selected set, and route low-confidence or insufficient-evidence results to the reviewer rather than silently accepting them.
No citation, no auto-classification.
Make tenant cost visible without weakening isolation
Per-tenant cost visibility should follow the same scope object as retrieval. For every embedding, rerank, and chat call, write the authenticated tenant_id, operation, request ID, returned cost_usd, latency, vendor, and timestamp to an append-only usage record. Aggregate from those records for dashboards and budgets. Never infer tenancy from prompt text, a document title, or a model response.
Three metrics catch different mistakes: filtered candidate count, selected passage count, and AI cost by operation and tenant. Alert on a sudden run of zero-candidate searches for one tenant because an ingestion or permission change may have removed useful coverage. Alert separately when a tenant's request rate or cost changes sharply. Logs should carry request and tenant correlation IDs but not raw report text or retrieved policy passages; those can contain customer data.
Keep the before/after test brutally small. Before the filter, a synthetic query can rank chunks belonging to tenants A and B. After the filter, tenant A's result IDs must be a subset of tenant A's authorized fixture IDs, even when tenant B has the closest vector. Add a second test where the right tenant has the wrong permission and expect no candidates. That invariant is more valuable than snapshotting a fluent answer.
Limits and decision rule
This design is not suitable when tenant metadata is optional, when the vector store cannot apply filters during retrieval, or when authorization changes cannot be propagated to chunks. Fix that data model before shipping RAG. Post-filtering in application memory isn't an equivalent control.
Use PostgreSQL with pgvector when operational simplicity and shared authorization data dominate. Pick Pinecone for a managed namespace-oriented service, Qdrant for payload filtering plus deployment control, or Weaviate when its multi-tenant object model matches the product. For the model layer, direct OpenAI, Anthropic, or Gemini relationships suit teams that want provider-specific control; a consolidated AI runtime fits when one contract and per-call usage metadata remove meaningful integration work. In every case, the invariant stays boring and strong: authenticate, scope, retrieve, rerank, generate, cite, meter.
References
- https://github.com/pgvector/pgvector
- https://docs.pinecone.io/guides/index-data/implement-multitenancy
- https://qdrant.tech/documentation/concepts/filtering/
- https://docs.weaviate.io/weaviate/manage-collections/multi-tenancy
- https://www.rfc-editor.org/rfc/rfc9110
Top comments (0)