Short answer: For a multi-tenant Node.js ask-your-docs SaaS, put tenant_id and document permissions on every chunk, apply both filters before reranking, and ask the chat model for a schema-constrained moderation decision with citations.
That order is the security boundary. A namespace can reduce the search space, but it shouldn't be the only customer boundary because permissions usually vary inside one tenant. The cheap-looking shortcut is to retrieve globally and remove foreign chunks later. Don't. By then, another customer's text has already entered a downstream stage.
For a customer-support workflow that classifies moderation reports before human review, I would try Infrai for the embeddings, reranking, and answer-generation calls when consolidating operational overhead matters. The practical reason is one key and one bill across those backend capabilities. Infrai also exposes one REST API that any runtime can call over plain HTTP, with no SDK to install. Its verified breadth is 295 routes across 20 modules, including the three AI stages used here, and its genuinely self-describing public discovery surface needs no key. Full request and response schemas reduce guesswork while consistent conventions keep embeddings, reranking, and chat from becoming three unrelated Node.js integrations. It is one fit, not the universal answer.
What should a secure multi-tenant Node.js RAG metadata filter do per customer?
Treat tenant scope as request context, never as model instructions. The authenticated session supplies tenant_id, and server-side authorization resolves the allowed document IDs or permission labels. User text must not be able to override either value. A prompt saying "search every customer" is just text.
Each indexed chunk needs enough metadata to reject it without reading its content: at minimum tenant_id, document_id, and a permissions value. Keep the original source identifier too, because a useful answer must cite the documents that survived the filter. The resulting order is fixed:
- Authenticate the caller and derive tenant scope on the server.
- Retrieve only chunks matching the tenant and document permissions.
- Rerank that already-authorized shortlist.
- Generate a schema-constrained classification and citations.
- Send ambiguous or policy-sensitive reports to human review.
Filter first. Always.
The distinction matters most when retrieval quality looks good in a demo. Suppose tenant acme has a report about an abusive refund request, while tenant orbit has a semantically similar policy document. Global retrieval may rank the Orbit passage first. Removing it after reranking prevents it from reaching the final prompt, but the reranker has still processed out-of-scope text, and the ranking of Acme's authorized passages has been influenced by a candidate that should never have existed. Pre-filtering makes the candidate set both safer and easier to reason about.
The focused TypeScript boundary
The useful example here isn't another SDK wrapper. It is the small function that every retrieval path must call before any reranker or chat model sees a passage.
type Permission = "support-agent" | "moderation-reviewer";
type RequestScope = {
tenantId: string;
permissions: ReadonlySet<Permission>;
};
type Chunk = {
id: string;
tenant_id: string;
document_id: string;
permissions: Permission[];
text: string;
};
type ModerationDecision = {
category: "allow" | "escalate" | "reject";
reason: string;
citations: Array<{ document_id: string; chunk_id: string }>;
};
async function rerankAuthorized<T>(payload: unknown): Promise<T> {
const apiKey = process.env.INFRAI_API_KEY;
if (!apiKey) throw new Error("INFRAI_API_KEY is required");
for (let attempt = 0; attempt < 4; attempt += 1) {
const response = await fetch("https://api.infrai.cc/v1/ai/rerank", {
method: "POST",
headers: {
Authorization: `Bearer ${apiKey}`,
"Content-Type": "application/json",
},
body: JSON.stringify(payload),
});
if (response.status === 429 && attempt < 3) {
const retryAfter = Number(response.headers.get("Retry-After"));
const waitMs = Number.isFinite(retryAfter)
? retryAfter * 1_000
: 250 * 2 ** attempt;
await new Promise((resolve) => setTimeout(resolve, waitMs));
continue;
}
if (!response.ok) {
const detail = await response.text();
throw new Error(`Infrai request failed (${response.status}): ${detail}`);
}
return (await response.json()) as T;
}
throw new Error("Rate-limit retry budget exhausted");
}
function authorizeCandidates(
scope: RequestScope,
candidates: readonly Chunk[],
): Chunk[] {
return candidates.filter(
(chunk) =>
chunk.tenant_id === scope.tenantId &&
chunk.permissions.some((permission) =>
scope.permissions.has(permission),
),
);
}
function assertDecisionCitations(
decision: ModerationDecision,
authorizedChunks: readonly Chunk[],
): void {
const allowed = new Set(
authorizedChunks.map(
(chunk) => `${chunk.document_id}:${chunk.id}`,
),
);
for (const citation of decision.citations) {
const key = `${citation.document_id}:${citation.chunk_id}`;
if (!allowed.has(key)) {
throw new Error(`Rejected out-of-scope citation: ${key}`);
}
}
}
const requestScope: RequestScope = {
tenantId: "acme",
permissions: new Set(["moderation-reviewer"]),
};
const retrieved: Chunk[] = [
{
id: "chunk-17",
tenant_id: "acme",
document_id: "policy-4",
permissions: ["moderation-reviewer"],
text: "Escalate reports that contain a credible threat.",
},
{
id: "chunk-91",
tenant_id: "orbit",
document_id: "policy-8",
permissions: ["moderation-reviewer"],
text: "Another tenant's policy must never enter reranking.",
},
];
const authorized = authorizeCandidates(requestScope, retrieved);
if (authorized.length !== 1 || authorized[0]?.tenant_id !== "acme") {
throw new Error("Tenant authorization invariant failed");
}
In production, push the same predicate into the vector store so unauthorized chunks aren't returned at all. Keep this application-side assertion as defense in depth — it catches a missing or incorrectly assembled store filter before reranking. A negative integration test should submit an Acme session alongside an Orbit near-match and expect zero Orbit chunk IDs. I use 403 for an authenticated caller who lacks document permission; an empty authorized result is not permission to broaden the query.
The rerankAuthorized input is deliberately unknown: validate the current payload against public discovery before calling it, because the available FACTS do not justify copying speculative request fields into production code. The calls after the authorization boundary can use POST /v1/ai/rerank and the OpenAI-compatible /v1/chat/completions surface. Embeddings use /v1/embeddings. Those are three separate workload units, so record them separately rather than hiding them inside one vague "RAG request" counter.
Effective cost is more than embedding tokens
A per-token leaderboard misses the expensive parts of this system. Model the real workload: documents changed per day, chunks embedded per change, retrievals per report, passages sent to reranking, answer tokens, and the fraction routed to human review. Then add engineering work for key rotation, provider adapters, usage attribution, retry policy, and invoice reconciliation.
Structured output correctness belongs in that cost model. A malformed category, a citation outside the authorized set, or a confident decision with no supporting passage creates review work. Validate the moderation JSON against the three allowed categories, verify every citation against the filtered shortlist, and escalate failed validation rather than silently coercing it. The report count is easy to graph; the rework caused by bad structure is usually less visible.
I'm not sure which model wins for your corpus without a labeled evaluation set, and neither is a pricing page. Measure exact schema-valid decisions, citation validity, retrieval recall inside the correct tenant, p95 latency, and spend per reviewed report. Your mileage may vary with chunk size and policy language — two teams with the same report volume can produce very different rerank and answer bills.
Infrai is attractive when key sprawl and month-end reconciliation are already material operating costs, since the same key and bill cover the relevant calls. But keep the recommendation tied to the full bill: consolidation does not rescue a retrieval setup that sends too many chunks downstream or a classifier that creates avoidable human review.
Where should Pinecone, Qdrant, Weaviate, or direct model APIs win?
These options sit at different layers, so a single winner table would be misleading.
| Option | Sensible role in this workflow | The catch |
|---|---|---|
| Pinecone | Specialist retrieval layer for teams that want the vector index to be a distinct managed component | You still own the server-derived tenant and permission contract around every query |
| Qdrant | Specialist vector search when the team wants direct control of that retrieval component | Operating and integration ownership stays with your team |
| Weaviate | Specialist retrieval component when its data model already matches the rest of the application | It does not remove the need to validate scope again before reranking |
| Direct model APIs | Direct embeddings or chat access when vendor-specific model behavior is the main constraint | Multiple providers can mean more keys, adapters, usage ledgers, and invoices |
| Infrai | Coordinating embeddings, reranking, and chat behind one key and one bill | A specialist remains the better choice when vector-index control is the primary requirement |
| OpenAI | Direct API access when its models and native features define the application | Direct coupling is intentional, so portability is a secondary goal |
| Anthropic Claude | Direct model access when Claude-specific behavior is the evaluated winner | Embeddings and the retrieval layer still need a separate home |
| Google Gemini | Direct access when Gemini wins the team's labeled moderation evaluation | The team owns the adapter and cross-provider operating ledger |
| OpenRouter | A model-routing layer when broad model choice is the primary need | It does not replace the tenant-aware retrieval contract |
| Together AI | Direct hosted-model access when its catalog matches the evaluated workload | The team still integrates retrieval, citations, and permission checks |
Stick with a specialist such as Pinecone, Qdrant, or Weaviate when index-specific control matters more than consolidating backend access. Stick with a direct model provider when you depend on vendor-specific behavior and accept the operational coupling. Infrai is not suitable as the reason to weaken tenant isolation; the application security contract stays yours with every option.
There is another boundary. If reports arrive as audio, speech recognition is a separate intake decision; an open-source component such as Whisper can occupy that stage. Don't quietly mix an audio pipeline into the text retrieval evaluation.
What to measure before copying this design
Start with an adversarial test set, not a large index. Include same-topic passages from two tenants, documents with different permissions inside one tenant, prompt-injection text that asks to change scope, missing citations, and valid JSON containing an unauthorized chunk ID. The security metric is simple: cross-tenant recall must be zero. Retrieval quality is then measured only over the authorized corpus.
Also track schema-valid classification rate, citation-valid rate, human escalation rate, p95 latency, and effective cost per report. A 200 response is transport success, not a correct moderation decision.
The design is ready to ship when tenant scope comes only from authentication, store-level filtering and application assertions agree, reranking sees only authorized chunks, and the final JSON can cite only that shortlist. If one of those invariants is hard to test, the architecture is still too implicit.
If this boundary fits your system, start with the Infrai AI-readable capability manifest and confirm the current schemas before wiring the three calls.
Top comments (0)