Someone in finance asks your text-to-SQL agent what the compensation band is for a role they're hiring for. The agent writes correct SQL. The query runs. Row-level security strips every row. The user is told:
No records found.
That answer is wrong, and it is wrong in the worst available way. There are records. The user simply isn't allowed to see them. Nothing in the chain can tell the difference between "this data does not exist" and "you may not have this data" — so the system picks the first and says it with confidence.
Your logs show a successful query. Your RLS policy shows it working exactly as configured. Everybody's dashboard is green, and the person who asked now believes something false about your company.
Where this comes from
It comes from the order of operations, and almost every text-to-SQL stack has the same order.
question → pick tables → write SQL → execute → apply row-level security → answer
Access control is the second-to-last step. By the time it runs, the model has already seen the table, already reasoned about its columns, already written a query that names it. RLS is doing its job — it protects the data. It cannot protect the answer, because the answer was shaped four steps earlier by a model that had no idea the table was off-limits.
This is not a Vanna problem. It's the default shape.
Which matters now, because Vanna is archived
vanna-ai/vanna was archived on 29 March 2026 and is read-only. 23.8k stars, 2.5k forks — a lot of people integrated it, and a fair number are now deciding what to move to.
Vanna 2.0 was a rewrite around user-aware agents: it resolves a User with group memberships, gates tools on those groups, and applies row-level security when the SQL runs. That's more identity handling than most of its alternatives have. It still has the ordering above.
So if you're picking a replacement, this is worth checking before you pick: does the model ever see a table this caller cannot read? For nearly everything on the shortlist the answer is yes, and the protection is downstream.
The other order
question + who's asking → pick tables THEY can read → write SQL → execute → answer
A restricted table isn't ranked lower. It isn't filtered afterwards. It is absent from the prompt. The model cannot write SQL against a table it was never shown, so there's no query to strip rows from and no confident wrong answer to deliver.
The knock-on effect is the one I didn't expect when I started: when the model can't reach the data, it says so. "I don't have a table that covers compensation" is a true statement about what it was given, and a user can act on it — ask someone, request access, escalate. "No records found" is a dead end built out of a false premise.
If you're migrating
This is the mapping I use. It's deliberately boring:
| Vanna | equivalent |
|---|---|
vn.train(ddl=...) (1.x) / tools reading a configured DB (2.x) |
cat = Catalog().bootstrap("postgresql://…") — reflects the schema once |
vn.train(documentation=...) |
cat.hint("orders", "…") — a human note that outranks everything |
User(id=…, group_memberships=[…]) |
Principal("okta:jdoe", roles={…}) |
a tool's access_groups
|
cat.restrict("hr_compensation", ["payroll"]) — on the table, not the tool |
vn.ask(question) |
sel = cat.select(question, principal=p), then sel.prompt_fragment() in your prompt |
vn.generate_sql(...) |
keep whatever model call you already have |
Your User object drops in more or less unchanged — id becomes the principal's subject, group_memberships become its roles:
from schemagate import Catalog, Principal
cat = Catalog().bootstrap("postgresql://localhost/app")
cat.restrict("hr_compensation", ["payroll"]) # once, at startup
def build_prompt(question, user): # per request
p = Principal(f"okta:{user.id}", roles=set(user.group_memberships))
sel = cat.select(question, top_k=6, principal=p)
return f"Schema:\n{sel.prompt_fragment()}\n\nQuestion: {question}"
Whatever generated SQL before still does. It just never sees hr_compensation unless the caller holds payroll.
Column-level works the same way — cat.restrict_column("employees", "salary", ["payroll"]) — so a table can be visible with a column withheld, which is the common real case.
What you give up, honestly
Vanna trained on question/SQL pairs and learned from feedback. This doesn't learn. It reflects the schema and ranks it.
So: if your accuracy came from a large question–SQL memory, keep that memory and use the identity gate only for the selection step. If your accuracy came from schema documentation, cat.hint() covers the same ground with less machinery. If it came from the feedback loop, you will miss it, and I'd rather say that here than have you find out in week three.
Certification status, plainly: live-tested on PostgreSQL 16, Oracle 26ai and SQLite. SQL Server and MySQL reflect but I haven't had a live instance to run them against — not because I expect trouble, but because I haven't done it. There's a certify_dialect.py script if you want to run it and tell me what happens.
Try it without installing anything against your database
pip install schemagate
schemagate demo "which customers owe us money" --answer
That runs against a bundled 42-object schema. No database to set up. No API key required — without one it prints a prompt you can paste into any chat. There's a browser demo too, no signup.
Repo: github.com/ashishsinha1602/schemagate, Apache-2.0. It reflects any SQLAlchemy database, and it ships an MCP server, so if your agent lives in Claude Desktop, Cursor or Zed it gets the same per-caller boundary.
If you're migrating off Vanna and you take one thing from this, let it be the question rather than the library: when your agent says "no records found", can you tell whether that's true? If you can't, that's worth fixing regardless of what you migrate to.
Top comments (0)