Most text-to-SQL stacks send the model the whole schema and put the access check after the query is written. That ordering is the bug, and it is easier to show than to argue about.
The whole idea in one picture
Left -- the caller holds no roles. hr_compensation never reaches the model at all: 9 of 42 objects selected, 570 prompt tokens instead of 2,511, one object hidden from this caller.
Right -- same question, same schema, same database. The caller now holds the payroll role, so hr_compensation is the first table in the prompt, and nothing is hidden.
Nothing about the question changed. The identity did.
Running against a real schema
That is the Studio on a claims schema: 14 of 51 objects selected, 1,212 prompt tokens against 3,237 for the full schema, and three objects withheld because this caller has neither actuarial nor phi. The restricted ones are listed on the left, so you can see what was withheld rather than guess.
Install
pip install schemagate
SQLite, PostgreSQL 16, Oracle 26ai, SQL Server 2022 and MySQL 8.4 are each certified against a live instance. There is also an MCP server -- now in the official MCP registry -- a LangChain retriever, and a CLI.
The honest part
A selection step is only as good as its retrieval, and mine is not state of the art. On Spider pooled into one catalog (876 tables, no hint about which database the question belongs to), every gold table is present 82.6% of the time at top 10. On Spider 2.0-lite, which is real BigQuery and Snowflake schemas rather than benchmark ones, it is 64.0% at top 10 across all 247 usable questions. Both numbers, the harness, and the two measurement errors I had to correct along the way are in BENCHMARKS.md.
Apache-2.0: github.com/ashishsinha1602/schemagate
Top comments (0)