DEV Community

Elay osetrov
Elay osetrov

Posted on Edited on

Can user A's question pull user B's documents? A tenant-isolation checklist for RAG

Most RAG data leaks don't need a clever attacker. They need one missing filter.

A shared vector index, a retrieval call that forgot the tenant_id, a semantic cache keyed only by the question text, and suddenly customer B gets an answer stitched together from customer A's contract. OWASP lists this as part of LLM08:2025 Vector and Embedding Weaknesses: in multi-tenant setups that share a vector database, embeddings from one group can be retrieved for another group's queries.

Here's the short version of how I audit it.

1. The model never chooses the filter

The tenant and permission filter comes from the authenticated session and is applied in the retrieval layer. Not from the prompt, not from a request parameter the client can edit, and not from anything the model writes. If a prompt says "search all customers' documents", nothing about the query should change.

2. A missing filter fails closed

If you use one shared index plus metadata filters, a query that arrives without tenant context must return an error, not "everything". Test this explicitly; it's the default you least want to discover in production.

3. Every cache is keyed by scope

Semantic caches, response caches, re-rankers, "related documents" widgets and summaries all need the same tenant and user scope as the first retrieval. The cache is where a perfectly filtered retriever leaks anyway.

4. Permissions are checked at query time

Mirroring ACLs at ingest isn't enough. When someone loses access to a document, or it's deleted, the index should follow within a stated time. Citations should only point to documents the user could open directly.

5. Retrieved text is data, not instructions

Retrieved chunks go into the prompt marked as untrusted. The agent shouldn't have an ungated tool that can send retrieved content outward, and output rendering shouldn't auto-load image URLs that could carry data in a query string. OWASP's own example is a résumé with hidden white-on-white instructions that a RAG screener then follows.

6. Alert on the one thing that should never happen

Log the tenant of every returned chunk next to the tenant of the session. If they ever differ, page someone.

Fifteen tests to run before launch

Set up two test tenants with two users each, then try things like:

  • Tenant A asks a question only tenant B's documents can answer.
  • The API call's tenant_id is swapped to tenant B.
  • A request arrives with no tenant claim at all.
  • Tenant A and tenant B ask the same question inside the cache TTL.
  • A document is revoked in the source system, then queried after your SLA.
  • An uploaded file contains hidden instruction text.

Each should come back with nothing from the wrong scope.

The full audit

I packaged the full version, 39 checks in 7 sections, all 15 cross-tenant test cases with pass conditions, and a data-source inventory sheet, as the RAG Data-Leak & Tenant Isolation Audit. It's general guidance, not a guarantee, but it's the list I'd want walked before a multi-tenant RAG app touches real customer data.


Free download: the AI Agent Pre-Ship Mini Checklist (pay what you want, $0 is fine).

Top comments (1)

Collapse
 
pomereneio profile image
Kyle Ma •

One test I would add to your same-question-inside-the-TTL case: make the two tenants' questions differ only in a write step, and make one of them follow a declared external write. In trajectory-caching work I have measured, deterministic replay came out at 47ms against 3.9s for a fresh run, but that speed is exactly where a similarity-based cache is most dangerous: a fuzzy match can look harmless for a read and still be wrong for a write. The rules that held up in our checks were strict: writes never replay on a fuzzy match, only on an exact match; a hit never serves past a declared external write; and a no-taint result plus a context change is a miss, not a hit. Abstract matches also stopped being treated as exact repeats after an October fix. What is still open, not solved, is a general permission and state precondition check at replay time, and a boundary-marking check on read matches — so I would not let a cache key that omits tenant, permission scope, and write boundary pass this audit, even if the similarity score is high.