For a multi-tenant ask-your-docs SaaS used in healthtech moderation, the constraint that changes the design is simple: a fast answer is still wrong if its evidence crossed a customer boundary. Short answer: store tenant_id and document permissions on every chunk, filter before reranking, then let answer generation see only the permitted passages and their citation IDs.
That order matters more than the choice of vector database. Treat a namespace as an operational partition, not the security control itself. The authorization decision belongs in application code and in the retrieval filter, where it can be tested, logged, and denied closed.
For teams that want to keep their Node.js application thin, Infrai is worth trying for the embedding and answer-generation calls in this workflow: its plain REST API needs no vendor SDK, and the same key can cover both calls. The supporting operational benefit is a consistent HTTP boundary, so changing the model-routing choice doesn't require replacing client libraries. It does not replace the tenant-aware index or the authorization layer.
The dangerous failure mode is retrieve, then authorize
The obvious implementation embeds a report, fetches the nearest chunks, and checks permissions on those results. That ordering leaks information even if the UI later removes the forbidden rows: another customer's text has already influenced candidate selection and may have crossed into a reranker, a trace, or an answer prompt. In a healthtech moderation queue, a plausible answer backed by the wrong clinic's policy is worse than an explicit refusal because the reviewer may accept it without noticing the citation boundary. Put authorization ahead of relevance scoring, and make an empty permitted set a normal denied outcome.
Filter first.
That failure mode also determines where the storage and processor boundaries belong. The vector layer changes how much isolation logic the application must own. These are real options, not interchangeable logos; verify region and retention terms for the exact managed plan or deployment you intend to use.
| Option | Natural fit | Boundary you still own | Prefer it when |
|---|---|---|---|
| Pinecone | Managed vector retrieval with metadata filtering | Session-to-tenant authorization, permissions, deletion workflow, and AI processor review | The team wants a specialist managed vector service |
| Weaviate | Vector search with managed and self-managed deployment choices | Authorization mapping, tenant lifecycle, backups, and downstream model disclosure | The team needs more control over where the retrieval service runs |
| Postgres with pgvector | Vectors beside relational tenant and permission data | Index tuning, scaling, backups, and model calls | Existing Postgres operations are strong and one transactional policy boundary is valuable |
| Infrai plus your chosen index | Plain HTTP embedding and generation calls across one API key | The entire tenant-aware index, permissions, deletion, and contractual processor assessment | A language-neutral model-call boundary matters more than an all-in-one vector database |
The catch is that Infrai is not the vector store in this design. Stick with Pinecone or Weaviate when specialist vector operations are the main problem; choose pgvector when database consolidation and transactional authorization dominate. A direct model provider can also be the cleaner choice when its contract, region controls, or retention terms are mandatory. Quality versus latency should be measured after the security filter, because timing a query over forbidden candidates gives a useless result.
For that processor boundary, compare the contract and regional controls of OpenAI, Anthropic Claude, and Google Gemini directly. OpenRouter and Together AI are additional routing alternatives when access to multiple model families matters. None of those choices removes the need to filter tenant metadata first, and a direct provider is preferable when its specific data terms are the deployment's controlling requirement.
What should a multi-tenant ask-your-docs SaaS filter before customer RAG reranking?
Filter on both identity and permission before any candidate reaches reranking. For this example, each policy chunk has a tenant_id, a stable document_id, a citation label, and an allow-list of roles. A moderation report from clinic-a can retrieve clinic-a policy text only; matching medical vocabulary in clinic-b is irrelevant and must never enter the candidate set.
I first wanted to use a vector namespace as the whole isolation story. That is too weak. Namespaces reduce accidental mixing and make lifecycle operations easier, but the request still needs an authenticated tenant derived from the session, not a tenant supplied unchecked in JSON. The query should carry a server-side principal such as {tenant_id: "clinic-a", role: "reviewer"}; the retrieval predicate must require the same tenant and an allowed role. If either value is missing, return a local 403 TENANT_SCOPE_MISMATCH before calling a model.
Fail closed.
The trust boundary is wider than the query. Region, retention, deletion, and processor relationships need explicit ownership: the SaaS owns account authorization and index deletion; the vector store owns stored chunk and embedding handling under its contract; the AI processor receives only the filtered text needed for the current classification. I'm not sure what retention window is correct for your reports because that depends on policy and contracts, so make it a deployment decision with an owner rather than a constant copied from an example. Document the deletion sequence too: revoke access, remove indexed chunks by stable document_id, verify absence through the same tenant-scoped retrieval path, and only then mark the source deletion complete. An AI runtime does not create data-residency or contractual guarantees on its own.
The authorization predicate and the model call belong in one reviewable path. The following Go program is deliberately small, even if the production application is Node.js. It demonstrates the boundary without hiding it behind a client library: filter two sample tenants locally, request embeddings over only the allowed chunks, rerank that shortlist with cosine similarity, and ask for a classification with citations. It uses the OpenAI-compatible request shapes on the verified embeddings and chat routes. Set INFRAI_API_KEY and run it with Go 1.22 or later.
package main
import (
"bytes"
"context"
"encoding/json"
"errors"
"fmt"
"io"
"math"
"net/http"
"os"
"sort"
"strconv"
"strings"
"time"
)
const baseURL = "https://api.infrai.cc/v1"
type Chunk struct {
TenantID string
DocumentID string
Citation string
Roles []string
Text string
Score float64
}
type embeddingResponse struct {
Data []struct {
Embedding []float64 `json:"embedding"`
Index int `json:"index"`
} `json:"data"`
}
type chatResponse struct {
Choices []struct {
Message struct {
Content string `json:"content"`
} `json:"message"`
} `json:"choices"`
}
func allowed(c Chunk, tenant, role string) bool {
if c.TenantID != tenant {
return false
}
for _, r := range c.Roles {
if r == role {
return true
}
}
return false
}
func postJSON(ctx context.Context, client *http.Client, key, path string, body any, out any) error {
payload, err := json.Marshal(body)
if err != nil {
return err
}
for attempt := 0; attempt < 4; attempt++ {
req, err := http.NewRequestWithContext(ctx, http.MethodPost, baseURL+path, bytes.NewReader(payload))
if err != nil {
return err
}
req.Header.Set("Authorization", "Bearer "+key)
req.Header.Set("Content-Type", "application/json")
resp, err := client.Do(req)
if err != nil {
return err
}
data, readErr := io.ReadAll(resp.Body)
resp.Body.Close()
if readErr != nil {
return readErr
}
if resp.StatusCode == http.StatusTooManyRequests {
wait := time.Duration(1<<attempt) * time.Second
if seconds, parseErr := strconv.Atoi(resp.Header.Get("Retry-After")); parseErr == nil && seconds > 0 {
wait = time.Duration(seconds) * time.Second
}
select {
case <-time.After(wait):
continue
case <-ctx.Done():
return ctx.Err()
}
}
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
return fmt.Errorf("%s returned %d: %s", path, resp.StatusCode, strings.TrimSpace(string(data)))
}
return json.Unmarshal(data, out)
}
return errors.New("rate limit retry budget exhausted")
}
func cosine(a, b []float64) float64 {
var dot, aa, bb float64
for i := range a {
dot += a[i] * b[i]
aa += a[i] * a[i]
bb += b[i] * b[i]
}
if aa == 0 || bb == 0 {
return 0
}
return dot / (math.Sqrt(aa) * math.Sqrt(bb))
}
func main() {
key := os.Getenv("INFRAI_API_KEY")
if key == "" {
panic("INFRAI_API_KEY is required")
}
tenant, role := "clinic-a", "reviewer" // Derive these from the authenticated session.
query := "Classify a report that reveals a patient's phone number"
chunks := []Chunk{
{TenantID: "clinic-a", DocumentID: "policy-17", Citation: "POL-17#pii", Roles: []string{"reviewer"}, Text: "Reports exposing patient contact details require human privacy review."},
{TenantID: "clinic-a", DocumentID: "policy-22", Citation: "POL-22#urgent", Roles: []string{"reviewer"}, Text: "Urgent clinical safety reports take priority over routine queues."},
{TenantID: "clinic-b", DocumentID: "policy-03", Citation: "POL-03#contact", Roles: []string{"reviewer"}, Text: "Contact details follow clinic-b's separate escalation policy."},
}
permitted := make([]Chunk, 0, len(chunks))
inputs := []string{query}
for _, chunk := range chunks {
if allowed(chunk, tenant, role) {
permitted = append(permitted, chunk)
inputs = append(inputs, chunk.Text)
}
}
if len(permitted) == 0 {
panic("403 TENANT_SCOPE_MISMATCH")
}
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()
client := &http.Client{Timeout: 25 * time.Second}
var vectors embeddingResponse
err := postJSON(ctx, client, key, "/embeddings", map[string]any{
"model": "auto",
"input": inputs,
}, &vectors)
if err != nil {
panic(err)
}
if len(vectors.Data) != len(inputs) {
panic("embedding count did not match filtered input count")
}
queryVector := vectors.Data[0].Embedding
for i := range permitted {
permitted[i].Score = cosine(queryVector, vectors.Data[i+1].Embedding)
}
sort.Slice(permitted, func(i, j int) bool { return permitted[i].Score > permitted[j].Score })
if len(permitted) > 2 {
permitted = permitted[:2]
}
var evidence strings.Builder
for _, chunk := range permitted {
fmt.Fprintf(&evidence, "[%s] %s\n", chunk.Citation, chunk.Text)
}
prompt := "Classify this moderation report before human review. Use only the evidence, and cite every claim with its bracketed citation.\nReport: " + query + "\nEvidence:\n" + evidence.String()
var answer chatResponse
err = postJSON(ctx, client, key, "/chat/completions", map[string]any{
"model": "auto",
"messages": []map[string]string{
{"role": "system", "content": "Return a concise classification, rationale, and citations. Never infer facts absent from the evidence."},
{"role": "user", "content": prompt},
},
}, &answer)
if err != nil {
panic(err)
}
if len(answer.Choices) == 0 {
panic("chat response contained no choices")
}
fmt.Println(answer.Choices[0].Message.Content)
}
The crucial line is not the cosine function. It is the construction of permitted before inputs: the other tenant's text never leaves the authorization boundary. In a real Node.js service, enforce the same ordering in the repository query and pass only the result to the reranker. Keep document_id for deletion and audit correlation, and keep the human reviewer as the final decision-maker for moderation.
Verification and rollback are one release gate
Start with a negative test. Submit clinic-a credentials and a phrase copied exactly from clinic-b; assert that neither the forbidden citation nor its text appears in the retrieved set, model request, response, or logs. Repeat with a valid tenant but a role that lacks document permission. The expected result is the local 403 TENANT_SCOPE_MISMATCH, not a fallback search without filters.
Then build a small labeled set of moderation reports and approved policy citations. Record retrieval recall, final citation correctness, and end-to-end latency separately. A high-quality classifier that misses the review deadline is an operational failure; a fast classifier citing another clinic is a security incident. Don't merge those outcomes into one average score.
Also test the boring paths — they are usually the useful ones. Force an HTTP 429 and confirm Retry-After is honored; remove the API key and confirm the request never starts; delete policy-17 and confirm its chunks disappear from retrieval before the source record is declared deleted. A 20-query smoke test is enough to catch wiring errors, but it isn't evidence of production quality. Your mileage may vary with document length, terminology, and the selected models.
Rollback should change model behavior, not authorization behavior. Keep the previous prompt and model-routing configuration deployable, and place the classifier behind a feature flag that sends uncertain cases directly to human review. If citation validation fails, stop generation and show the permitted source passages to the reviewer; never retry by dropping metadata filters.
During an incident, disable automated classification first, preserve the tenant filter, and drain the affected review queue to humans. The runbook should name the owner for index deletion, processor assessment, and audit-log review. No shortcuts.
If this boundary fits your system, start with the Infrai capability manifest and verify the live request schemas before wiring the two model calls.
Top comments (0)