Vanna proved you can get surprisingly far by retrieving similar queries and letting the model pattern-match.
Then someone asks a question that resembles nothing in the training set.
How RAG-based text-to-SQL works, and where that ends
Find queries like yours, adapt them. When the question is close to something seen before, it's fast and often right. That's a legitimate architecture and it explains the traction.
| Limit | Why it's structural |
|---|---|
| Corpus coverage | Accuracy tracks how well history covers the question space |
| Novel joins | No precedent to retrieve, so the model improvises |
| No entitlement concept | Similarity doesn't know who's asking |
| Silent corpus decay | Schema evolves; old queries stay in the index |
The entitlement row is the one that blocks enterprise deployment. Retrieval has no notion of authorisation — you can filter the result, but the query already ran with whatever credentials the service holds.
The alternative isn't a bigger corpus
More examples raise the floor and don't move the ceiling. The ceiling is that pattern-matching has no model of meaning, so it cannot distinguish "no precedent exists" from "improvise something plausible."
Resolve-then-prove instead of retrieve-then-guess:
- Parse language into a typed intent
- Resolve entities and metrics against a versioned semantic graph
- Prove a join path exists — fail compilation if it doesn't
- Inject policy predicates for the caller
- Emit dialect-perfect SQL
Step 3 is what a retrieval corpus fundamentally cannot provide.
When Vanna is still the right call
Open source, fast to pilot, full visibility, no procurement. For a proof of concept on a bounded schema it's a reasonable starting point — just budget for the fact that production governance is yours to build.
The full breakdown — the scored alternatives comparison, and the architectural difference between retrieval and resolution — is here:
👉 Vanna AI Alternatives: When RAG-Trained Text-to-SQL Needs Governance
Originally published at colrows.com/blogs/vanna-ai-alternatives
Top comments (0)