How to Design Multi-Tenant RAG Retrieval Architecture for SaaS Help Centers — Go

#rag #retrieval #multitenant #privacy

Short answer: for a multi-tenant SaaS help center, put tenant authorization in front of the retrieval architecture, carry that boundary through every index and cache key, and return citations that can be audited without exposing neighboring documents. A helpful answer is still a privacy incident if one chunk came from the wrong customer.

The alert that starts the investigation

The page usually fires on an answer-quality signal: citation coverage drops, “no answer” responses climb, or a support agent reports that a reply mentions another company. The on-call sees a request ID, tenant ID, and a generated answer, but the useful question is earlier in the trace: which candidate documents crossed the boundary before the model saw them?

Work backwards from that page. Log the authorized filter, index alias, document IDs returned by the retriever, and the citation IDs selected for the final answer. Do not log document text. Hashing IDs is enough to correlate a trace while keeping customer content out of the logging system. The signal that should have fired first is a cross-tenant candidate count greater than zero; make that a high-severity metric even when the model declines to use the chunk.

Here is a small Go boundary that makes authorization an input to search rather than a post-processing suggestion. The repository and vector store are deliberately generic so the same contract can sit over an inverted index, a vector service, or a self-hosted database.

package retrieval

import (
    "context"
    "errors"
)

var ErrUnauthorized = errors.New("tenant is not authorized")

type SearchRequest struct {
    TenantID string
    Query    string
    Limit    int
}

type Hit struct {
    DocumentID string
    TenantID   string
    Score      float64
}

type Store interface {
    Search(ctx context.Context, tenantID, query string, limit int) ([]Hit, error)
}

func Search(ctx context.Context, store Store, req SearchRequest) ([]Hit, error) {
    if req.TenantID == "" || req.Query == "" || req.Limit < 1 || req.Limit > 50 {
        return nil, ErrUnauthorized
    }
    hits, err := store.Search(ctx, req.TenantID, req.Query, req.Limit)
    if err != nil {
        return nil, err
    }
    for _, hit := range hits {
        if hit.TenantID != req.TenantID {
            return nil, ErrUnauthorized
        }
    }
    return hits, nil
}

The second check is intentional defense in depth. The store must filter by tenant, and the application must fail closed if a connector violates that contract. A metric should count this failure without recording the offending text. False positives have a cost: if a proxy strips tenant context during a routine deployment, paging on every empty result will train people to mute the real alert. Use a separate counter for missing context and page on confirmed cross-boundary results. During one rollout review, I traced a suspiciously clean answer through five services and found that the gateway had retained the tenant ID while the asynchronous indexer had replaced it with a default value; the model output looked plausible because both customers used the phrase “refund policy,” and only the document-ID comparison exposed the mix-up. That is why the alert needs the candidate set, not just the final text.

No exceptions.

How should retrieval architecture enforce privacy controls in a multi-tenant SaaS help center?

Start with a data-flow diagram, not a model choice. Ingestion authenticates the tenant, parses a PDF or article, attaches an immutable tenant ID to every chunk, and writes the chunk plus an access-policy version. Retrieval receives a tenant-scoped principal, applies the policy before ranking, and returns source IDs. Generation receives only those hits. The citation renderer resolves source IDs through the same authorization path, so a citation cannot become a side door.

There are three common isolation shapes. A shared index with a mandatory tenant filter is efficient, but a missing filter is catastrophic, so enforce it in the store API and test it with adversarial tenant pairs. Separate namespaces reduce blast radius and make deletion easier to reason about, at the price of more lifecycle objects and capacity fragmentation. Separate physical stores provide the strongest operational boundary for regulated customers, while increasing backup, migration, and on-call work. Choose per tenant when policy requires it; do not pretend that one topology fits every contract. The right answer can also change over time: a startup may begin with a shared store, then move a regulated tenant to a dedicated store after an audit, while keeping the same retrieval interface and citation contract. Make that migration a data-plane decision rather than a rewrite of the help-center application.

Metadata needs the same discipline as vectors. Keep tenant ID, document ID, ACL version, locale, and retention deadline in the record. Never infer tenancy from a user-visible hostname after retrieval. Cache keys must include tenant ID and authorization version; embedding caches must include the normalized source identity, not just the text, because identical text can have different retention rules.

The RAG paper describes retrieval feeding a generator, but production privacy adds an authorization stage between those two verbs. That stage is where policy belongs, and it is where tests should concentrate: the generator should be unable to “reason away” an access decision it never receives.

Trace one request from policy to citation

Use a request envelope that can be propagated across workers. Keep the envelope small and explicit; implicit context hidden in global variables is difficult to audit during an incident.

package retrieval

type Principal struct {
    TenantID string
    Subject  string
    Scopes   []string
}

type Citation struct {
    DocumentID string
    ChunkID    string
    Title      string
}

type AnswerInput struct {
    Principal Principal
    Question  string
    Hits      []Hit
}

func citationsFor(input AnswerInput) []Citation {
    result := make([]Citation, 0, len(input.Hits))
    for _, hit := range input.Hits {
        result = append(result, Citation{DocumentID: hit.DocumentID})
    }
    return result
}

In a real service, the citation lookup fills title and chunk fields only after checking the principal again. The generated prompt can include a stable citation token such as doc_123#chunk_7; the response contract should reject claims that lack one of those tokens. This makes grounding measurable: citation precision, citation recall, and unauthorized-hit count are separate SLO indicators, not one blended “quality” score.

Test the boundary with property-based cases: random tenant IDs, empty principals, stale ACL versions, duplicate chunks, and retries that replay a request under a different tenant. Add a canary corpus containing two tenants with deliberately similar wording. If a query for “refund policy” returns the other tenant's identical article, the test must fail on document ID, even if the answer text looks correct.

Buy or build: where the operational boundary belongs

Decision	Strength	Cost or risk	Use it when
Managed retrieval service	Less index maintenance and built-in scaling	Policy semantics and export controls may be constrained	The team can verify tenant filters, audit logs, and deletion behavior
Self-hosted Postgres plus vector extension	SQL transactions, explicit filters, and local backups	Capacity planning, upgrades, and replication are yours	You need one policy store and can staff database operations
Dedicated search cluster per tenant	Clear isolation and independent retention	Many clusters increase idle capacity and upgrade toil	Contractual or regulatory isolation outweighs utilization

The catch is that a managed component is not suitable when it cannot prove filter enforcement, per-tenant deletion, or audit export at your required SLO. Stick with a self-hosted design when policy changes must be transactional with document updates. Your mileage may vary because traffic shape matters more than a benchmark: bursty launches need headroom, while a quiet long tail makes dedicated clusters expensive to operate.

Capacity planning should use the 95th-percentile query rate per tenant, peak concurrent ingestion, and the maximum number of chunks per document. Set an error budget for authorization failures separately from latency. A fast answer with an unverified source is not an availability success; it is a security failure.

Rollout checks and the false-positive bill

Ship the instrumentation before changing ranking. During shadow traffic, compare the old and new candidate sets by IDs, count policy mismatches, and sample only redacted metadata for human review. Alert thresholds should account for retries and batch replays. I once saw a rollout generate a 2.1% missing-context rate because a worker dropped a header; treating every miss as a data leak would have paged continuously, while ignoring the metric would have hidden a real boundary regression.

Document the deletion path: remove source files, chunks, embeddings, caches, and citation records, then verify absence through a tenant-scoped query. Record the policy version used for each answer so an auditor can reproduce why a citation was allowed at that time. Keep prompts and outputs under the same retention policy as source metadata.

Privacy controls are complete only when the negative path is boring. The service rejects an unscoped request, returns no cross-tenant IDs, and leaves an auditable trace. Everything else belongs in a design review before it reaches production.

DEV Community