Text to SQL Agent Cost: What an "Ask Your Data" Feature Really Takes

Text to SQL agent cost for a production SaaS feature lands somewhere between a few weeks of focused engineering and a quarter of a small team's time. The spread in that text to SQL agent cost comes almost entirely from three things: how messy your schema is, how strict tenant isolation has to be, and how many questions the agent must get right before customers see it. The model is the cheap part. The semantic layer and the safety rails are where the money goes.

What founders are actually buying

The feature on the roadmap is a text box in the dashboard that says "ask your data". A customer types "which accounts churned last quarter after a failed payment", the agent writes SQL, runs it, and returns a table or a chart. Every vendor demo of this works beautifully, because every vendor demo runs on five tables with names like customers and orders.

Your database has status, status_v2 and a deleted_at column that half the queries forget. It has a tenant_id on most tables and an organisation_id on the older ones. It has three ways to compute revenue, and finance only trusts one of them. An LLM reading that schema cold will produce plausible SQL that is wrong in ways a customer cannot detect from the output. That gap between the demo and your schema is the project.

Text to SQL agent cost: where the money actually goes

The semantic layer is the biggest line

A semantic layer is a written description of what your data means: which tables the agent may touch, how they join, what "active customer" and "monthly revenue" are defined as, and which columns are legacy noise. Recent tooling in this space exists precisely to stop agents guessing SQL from raw column names, and the idea is sound. The model stops inventing joins and starts filling in a template you control.

The expensive part is that writing it is business work, not just engineering. Someone who knows the product has to sit down and decide, in writing, what the metrics mean. If your team has never agreed on a churn definition, this is the week you find out. Budget for it honestly. On a mature product with a hundred tables, the semantic layer alone can be a third of the total cost, and it is also the part that keeps paying off because your human analysts end up using it too.

Tenant isolation and permissions

A text to SQL agent is a program that runs arbitrary queries written by a language model against your customers' data. Treat it that way. The database connection the agent uses should be a dedicated read-only role, never the application user and never anything with write grants. In PostgreSQL, enforce the tenant boundary with row-level security policies keyed on the tenant, so a wrong WHERE clause from the model returns nothing instead of returning someone else's rows. Set a statement_timeout on that role so a bad join cannot sit on your primary for ten minutes.

This is the same design problem as permission-aware RAG: the access control has to live below the model, in a layer the model cannot talk its way around. If a vendor's answer to "how do you prevent cross-tenant leaks" is "the prompt tells it to filter by tenant", walk away.

The eval set

Before the feature ships, you need a bank of real questions with known correct answers, pulled from support tickets and sales calls, run against a copy of production data. Fifty questions is a thin start. Two hundred is comfortable. Each one costs someone's time to write the gold SQL and verify the result, and this is the deliverable that tells you whether the thing works. We have written before about why evals belong in the vendor contract, and this feature is the clearest case for it: without the question bank, "it works" means "it worked on the three queries we tried in the demo".

The model call itself

The inference cost is almost a rounding error next to the above, and the architecture around it should be simple. The pattern that tends to win is one call with the semantic layer, the relevant table definitions and a handful of worked examples in context, returning SQL plus a one-line explanation. Multi-step chains (a planner that picks tables, a writer that drafts SQL, a fixer that repairs errors) sound more rigorous and in practice add latency, cost and new places to fail.

We saw the same thing in a very different system. Our own outreach engine scrapes a prospect's site with a self-hosted crawler and a local model, then writes one tailored email per company. We started with a chain that extracted facts in one step and drafted in another. A single call that did both, with good context, beat the chain on cost and on output quality. Text to SQL behaves the same way: put the knowledge in the context, not in the orchestration. A retry on a SQL error is fine. A five-stage pipeline is usually a sign the semantic layer is missing.

Build vs buy

There are three realistic options.

• The AI feature inside your existing BI tool. Cheapest and fastest. Good for internal dashboards or a single-tenant product. Weak when every customer may only see their own rows, and the metric definitions are whatever the tool infers.
• An embedded analytics vendor with natural language on top. Reasonable middle ground if your schema is clean and your tenancy model matches theirs. Check where the data is processed. If your customers are in the Gulf or in healthcare, data residency can rule this out on its own.
• Build it on your own stack. Most expensive up front, and the right call when tenant isolation is non-negotiable, when your metrics have house definitions, or when the feature is part of what you sell rather than a nice-to-have.

The honest test is whether the "ask your data" box is a product differentiator or a convenience. If customers would pay more for it or churn without it, build. If it is a checkbox for the sales deck, buy and move on.

What a sane first version looks like

Scope the first release tightly. A version that answers a narrow set of questions correctly beats one that attempts everything and is right most of the time.

1. Pick one domain (billing, usage, or support) and ten to fifteen tables.
2. Write the semantic layer for that domain only, with metric definitions signed off by whoever owns the numbers.
3. Create the read-only role, row-level security policies and timeouts before the first prompt is written.
4. Build the eval set from real customer questions and run it on every change.
5. Show the generated SQL and the row count next to every answer so power users can catch mistakes.
6. Ship to a handful of friendly accounts and read every query they run for two weeks.

Expand to the next domain only once the eval pass rate on the first one is boring.

Questions to ask whoever quotes you

A quote for this feature that comes back as a single number with no questions attached is a quote for the demo, not the product. Ask:

• How will the agent connect to the database, and with which grants?
• Where does the tenant filter live, and what happens if the model omits it?
• Who writes the metric definitions, and how long is that estimated to take?
• What does the eval set look like at handover, and what pass rate is the acceptance criterion?
• Which model is used, and what does the per-query cost look like at a thousand questions a day?
• What happens when the schema changes next quarter?

The answers tell you whether the team has shipped this against a real schema or against a tutorial. If you want a second opinion on a quote, or a scoped estimate for your own database, let's talk.

---

Originally published on the Pykero blog.