Most text-to-SQL prompts carry the whole schema. schemagate sends only the tables the question needs: 91% fewer schema tokens on 50+ table databases. It is #1 on BIRD's Mini-Dev leaderboard in SQLite, PostgreSQL and MySQL (81.0 / 72.6 / 72.4), and works as a LangChain and LangGraph retriever.
Oracle is where text-to-SQL breaks. On UniQL, BIRD's 1,534 questions translated into 16 dialects, schemagate with Gemini 3.8 Flash scores 63.69% on the first try (64.93% with repair). The best published Oracle score is 49.61% (Claude 4.5 Sonnet).
What got it there
- Two Oracle rules in the prompt (no AS before a table alias; quote reserved words) took the first try from 35% to 64%.
- Names stored in lower case are shown to the model in the double quotes Oracle needs.
What running it found
- Four gold queries that error.
- An evaluator that upper-cased string literals.
- A loader whose users Oracle treats as system schemas.
The UniQL team fixed all of it and re-released on Nov 12; this score is on the fixed release.
Limits: Oracle only. The published baselines predate the fix and haven't been recomputed. Our leaderboard row is pending.
Results and reproduction: https://github.com/ashishsinha1602/schemagate/tree/main/benchmarks/results/uniql_oracle ยท pip install schemagate
Top comments (0)