Multi-Tenant RAG in TypeScript: Keeping One Customer's AI Out of Another's Data

• Book: AI That Reads
• The series: AI in TypeScript — 5 books, from your first LLM call to agents in production — all five here
• My project: Hermes IDE | GitHub — an IDE for developers who ship with Claude Code and other AI coding tools
• Me: xgabriel.com | GitHub

---

In a normal CRUD app, a missing tenant filter shows you someone else's row and
you notice, because the screen has the wrong name on it.

In RAG, a missing tenant filter feeds another customer's document into a
language model, which paraphrases it into a fluent answer with no name
attached. Nobody notices. There is no wrong-looking screen — just an assistant
that suddenly knows something it should not.

That is the whole reason this deserves more care than a regular query.

The query that causes it

const rows = await db.query(
  `SELECT id, text FROM chunks ORDER BY embedding <=> $1 LIMIT 10`,
  [queryVec],
);

Nothing here is tenant-aware. It is correct in local development with one
tenant's fixtures, and it stays correct until the second customer is
onboarded.

The failure is not that someone wrote this deliberately. It is that a second
search path gets added later — a "related documents" widget, an admin tool, a
background summariser, and that one forgets.

Make the unscoped query unrepresentable

The strongest fix is a type, not a code review rule.

declare const brand: unique symbol;
export type TenantId = string & { readonly [brand]: "TenantId" };

export type SearchScope = {
  readonly tenantId: TenantId;
  readonly userId: string;
  readonly allowedDocSets: readonly string[];
};

export async function search(
  q: string,
  scope: SearchScope,     // not optional, not defaulted
  k = 10,
): Promise<Chunk[]> {
  return db.query(
    `SELECT id, text, doc_id FROM chunks
     WHERE tenant_id = $2 AND doc_set = ANY($3)
     ORDER BY embedding <=> $1
     LIMIT $4`,
    [await embed(q), scope.tenantId, scope.allowedDocSets, k],
  );
}

Two properties worth the extra lines. scope is a required parameter, so a
new call site cannot omit it — the compiler asks for it. And TenantId is
branded, so passing a userId or an org slug by mistake does not type-check.

Do not export a lower-level search that takes no scope. If it exists, it will
be used.

Four isolation patterns

Shared table, filtered column. One chunks table with tenant_id.
Simple, cheap, and everything depends on that WHERE. Fine for many tenants
of modest size — with the caveat below about ANN indexes.

Row-level security. Postgres enforces the filter, so a forgotten WHERE
is not a breach:

ALTER TABLE chunks ENABLE ROW LEVEL SECURITY;

CREATE POLICY tenant_isolation ON chunks
  USING (tenant_id = current_setting('app.tenant_id')::uuid);

await client.query("SET LOCAL app.tenant_id = $1", [scope.tenantId]);

SET LOCAL inside a transaction, never SET — on a pooled connection a
session-level setting leaks to whoever draws that connection next, which
recreates the exact bug you are preventing.

RLS is the strongest option per line of code. Its cost is that every query
path must run inside a transaction with the setting applied, including
migrations and background jobs.

Schema per tenant. Real separation, easy to back up and delete per
customer. Cost: migrations multiply, and connection pooling gets awkward past
a few dozen tenants.

Database or index per tenant. Strongest, and the right answer when tenants
are few and large or contractually require it. Operationally heaviest.

Most teams should start at shared-table-plus-RLS and move up only for tenants
that ask.

The ANN index problem nobody warns about

This one is specific to vector search and it surprises people.

Approximate indexes (HNSW, IVFFlat) traverse a graph built over all rows.
The tenant filter is applied to what the traversal finds. So with a
restrictive filter, the index visits mostly other tenants' vectors and
discards them, and you can ask for ten results and receive three.

Not wrong, not an error, just quietly incomplete. Your assistant says "I could
not find much about that" for a document that is definitely there.

Two mitigations:

-- partial index per large tenant
CREATE INDEX chunks_hnsw_acme ON chunks
  USING hnsw (embedding vector_cosine_ops)
  WHERE tenant_id = 'acme-uuid';

Good when a handful of tenants dominate. Impractical at a thousand tenants.

// over-fetch, then filter, and detect starvation
const raw = await searchUnfiltered(qVec, k * 20);
const mine = raw.filter((r) => r.tenantId === scope.tenantId).slice(0, k);
if (mine.length < k) metrics.increment("rag.filter_starvation");

Over-fetching leaks other tenants' rows into your process even though you
discard them, which some compliance regimes will not accept. Know which
constraint you are under before choosing.

The metric matters either way: filter starvation is invisible without it.

Test it like a security control

it("never returns another tenant's chunk", async () => {
  await seed(TENANT_A, ["acme internal pricing is 40% margin"]);
  await seed(TENANT_B, ["globex internal pricing is 25% margin"]);

  const res = await search("internal pricing", scopeFor(TENANT_A));

  expect(res).not.toHaveLength(0);
  for (const c of res) expect(c.tenantId).toBe(TENANT_A);
  expect(JSON.stringify(res)).not.toMatch(/globex|25%/i);
});

it("returns nothing when the tenant has no matching docs", async () => {
  await seed(TENANT_B, ["globex pricing"]);
  const res = await search("pricing", scopeFor(TENANT_A));
  expect(res).toHaveLength(0);          // NOT globex's chunk
});

The second test is the one that catches the real bug. A broken filter still
passes the first test — tenant A has matching docs, so plausible results come
back and you never look closely. Only the empty case exposes it.

Seed deliberately distinctive strings. Generic fixture text makes a leak
unnoticeable in an assertion.

Also scope everything downstream

The vector query is not the only path to another tenant's data.

The embedding cache, if keyed only on content hash, is shared, which is
usually fine (a vector is derived from text you already have) but worth a
deliberate decision rather than an accident.

Citations resolve a chunk id to a document. That resolver needs the same
scope, or a leaked id becomes a readable document.

Logs. Chunk text in a log line crosses a tenant boundary the moment two
engineers share a dashboard. Log ids and scores, not text.

The rule

Scope is a required argument with a branded type, enforced in the database
rather than by convention, tested with the empty case, and applied to every
path that can turn an id back into text.

A missing WHERE in a normal query is a bug. In a RAG pipeline it is an
incident report, and the paraphrasing makes it one you may never find out
about.

---

If this was useful

AI That Reads covers retrieval in real
systems — tenancy, filtering, ANN index behaviour under restrictive filters,
and keeping provenance intact from query to citation.

The full series is at
xgabriel.com/ai-in-typescript.