DEV Community

Ashish sinha
Ashish sinha

Posted on

Vanna is archived. What that means if you have it in production.

The vanna-ai/vanna repository was archived by its owner on 29 March 2026. 23.8k stars, read-only, no pinned explanation that I could find. I'm not going to speculate about why - that's the maintainers' business, and a lot of people got real value out of it for free.

What I do want to write down is what archiving actually costs you if it's in your stack, because "it still works" is true right up until it isn't.

What read-only actually means

Your pinned version keeps working. Nothing breaks today. What stops is everything around it: no dependency bumps, no fixes for the driver change that lands next quarter, no security patches, and issues and pull requests are closed - so the workaround you'd normally find in a comment thread never gets written.

The practical horizon is the next breaking change in something underneath it. sqlalchemy, your DB driver, an LLM SDK. That's usually months, not years.

Vanna was five things, not one

This is the part that catches people out when they start looking for a replacement. Unbundled, it did roughly:

  1. A training store - your DDL, docs and example SQL, embedded
  2. Retrieval - pick the relevant schema for this question
  3. Prompt assembly
  4. The LLM call
  5. UI and chart helpers

There is no single library that is "Vanna but maintained", because those five pieces live in different places now. LangChain's SQL agent and LlamaIndex's NLSQLTableQueryEngine cover 2-4 with different opinions. MCP-based toolboxes cover 3-4 and hand you 1-2. Nobody hands you all five in one import.

The piece that carries the risk is #2

When you rebuild retrieval, you are deciding what the model is allowed to see. That step runs before the query executes - which means it runs before row-level security, VPD, or any policy layer can act.

If your replacement isn't identity-aware, here's the failure: the model is handed a table this caller can't read. It writes perfectly correct SQL. The policy layer filters every row. The agent reports "there are no orders for that customer."

That is not an access-denied message. It's a wrong answer in a confident tone, and the person reading it cannot tell the difference between "you may not see this" and "this does not exist." Worse, nothing logs an anomaly, because nothing went wrong - every component did its job.

With GRANT you'd get a clean permission error. With RLS you get a plausible sentence. If you're carrying Vanna's retrieval forward or replacing it, that asymmetry is the thing to design against.

A migration checklist that isn't just "pick a framework"

  • Pin your Vanna version and your transitive deps now, before something moves under you
  • Write down which of the five pieces you actually use. Most teams use 2, 3 and 4 and never touched 5
  • Export your training data. It's yours, it's the expensive part, and it's portable
  • Decide where identity enters. Per-request, before retrieval - not after execution
  • Test the empty-result case explicitly. Ask something a restricted user shouldn't see, and check what your agent tells them

That last one takes ten minutes and is the one nobody runs.

Disclosure

I wrote a library called schemagate that covers piece #2 only - retrieval plus identity scoping. It is not a Vanna replacement and I'd rather say that plainly than have someone find out after migrating. It filters the catalog by caller identity before ranking, so a restricted table's name never enters the prompt; ranking is BM25 plus a hashed embedder, offline, no API key and no model call.

On a deliberately messy 127-object schema, recall@6 was 47% with bare identifiers, 60% with Gemini 2.5 Pro writing table descriptions, 80% with Sonnet - so if you're rebuilding retrieval, budget for the description pass, not just the embeddings. That's the part I'd have wanted to know before starting.

Demo runs in your browser, no signup and no database: https://ashishsinha1602.github.io/schemagate/
Apache-2.0: https://github.com/ashishsinha1602/schemagate

If you're mid-migration off Vanna I'm genuinely interested in what you hit - the retrieval half is the part I've spent the most time in.

Top comments (1)

Collapse
 
raknaos profile image
Baptiste Le Bouquin

The identity-aware retrieval point is the sharpest thing in this piece. "Perfectly correct SQL, policy-filtered to zero rows, reported as 'no orders'" is exactly the failure mode that doesn't show up in any dashboard, because every component did its job — the bug lives in the gap between the retrieval layer and the policy layer, and nobody owns that gap.

It also inverts the usual archiving risk. The common advice is "pin the version, you have months," and that's true for the code. But the training store is the part that silently rots: your DDL drifts from the real schema, your example SQL teaches the model patterns your DBA has since forbidden, and nothing errors — answers just get gradually worse. With a maintained project someone files an issue when retrieval quality drops; with an archive, you're the instrumentation.

Did you end up rebuilding the training store separately from retrieval, or did you fold your DDL/docs into whatever MCP toolbox or agent you replaced it with? I'm curious whether anyone keeps that curation as a first-class artifact after the unbundling.