DEV Community: Vivek Kumar

How to Choose a BI Tool for a Small Team (Without Regretting It in Six Months)

Vivek Kumar — Fri, 25 Sep 2026 13:44:50 +0000

Picture a founder on a Sunday night. The company has fifteen people, revenue is climbing, and every important number — signups, churn, cash, support volume — lives in a different spreadsheet that someone updates by hand. She decides it's time for a "real" analytics tool. She opens a browser, searches "best BI tool," and finds thirty options, each claiming to be the fastest, smartest, and most affordable. Two hours later she has fourteen tabs open and no decision.

If that's you, this guide is for you. BI stands for business intelligence — software that connects to your data and turns it into dashboards, reports, and answers to questions like "how did we do last month?" Picking one shouldn't require a data science degree. But it's easy to get wrong, and getting it wrong is expensive: you pay for a tool nobody uses, and you're back to spreadsheets within a quarter.

Here's how to choose well, what to look for, and the traps that catch most small teams.

Start with your requirements, not the feature list

The single most common reason companies pick the wrong tool: they start by comparing features instead of writing down what they actually need. Feature lists are designed to impress. Almost every BI tool can make a bar chart. The question is whether this tool fits your team, your data, and your budget — today and a year from now.

Before you look at a single product, answer four plain questions:

Who will actually build the reports? A non-technical founder, a marketing manager, a part-time analyst?
Where does your data live? A single Postgres database? A mix of a database, Stripe, and a CRM?
What decisions do you need to make weekly? Be specific: "Is churn getting worse?" beats "we want insights."
What can you spend — now and when you're twice the size?

Those answers are your scorecard. Everything else is a demo.

The five things that actually matter

Once you know your requirements, judge every tool against the same five criteria. In roughly this order of importance for a small team:

Criterion	The question to ask	Why it matters most for small teams
Ease of use	Can the people on my team actually build a report themselves?	A tool nobody can use is shelfware, no matter how powerful.
Time to value	Can I connect my data and get one useful answer within an hour?	Small teams don't have weeks to spend on setup.
Data connections	Does it plug into where my data already is?	A BI tool is only as good as the data it can reach.
Pricing that scales	What does this cost at 5 users? At 50?	The cheap plan today can become the expensive one next year.
Governance & security	Can I control who sees what?	Not everyone should see payroll or customer PII.

Let's unpack the two that trip people up most.

Ease of use is the whole game. Research consistently finds that most BI projects underdeliver, and the number one reason isn't the technology — it's that people never adopt it. Forrester's analytics research has found that only about a third of employees actively use the BI tools their company pays for. If your marketing manager can't build a dashboard without filing a ticket, she'll go back to her spreadsheet, and your investment quietly dies. Pick the tool your team can use this week, not the one that matches the skills you wish you had.

Data connections decide whether it works at all. Your numbers are probably scattered — some in a database, some in Stripe, some in a support tool. A good BI tool connects to those sources directly, without you exporting CSVs by hand every Monday. If your core data sits in a SQL database (Postgres, MySQL, and the like), make sure any tool you consider connects to it cleanly, because that's usually where the real answers live.

A simple way to compare, without a spreadsheet marathon

You don't need a 40-row comparison matrix. Score two or three shortlisted tools from 1 to 5 on the five criteria, weighted for your situation. Something like this is plenty:

Criterion	Tool A	Tool B
Ease of use	5	3
Time to value	4	3
Data connections	4	5
Pricing that scales	3	4
Governance	4	4
Total	20	19

The point isn't the exact math — it's forcing yourself to rate the things that matter instead of being dazzled by a slick demo. If a tool scores a 2 on ease of use, no amount of fancy charting saves it.

Run a real trial before you commit

Demos are theater. The salesperson drives, the data is perfect, and everything works. You learn almost nothing about how the tool behaves with your messy data and your actual team.

So run a short proof of concept. Connect your real data, pick one question that matters to the business, and see how far a non-expert on your team can get in an afternoon. A few concrete tests:

Connect your production database (read-only) and build a "revenue this month vs. last month" chart.
Hand the tool to someone non-technical and ask them to change a date range without help.
Try to give a teammate access to some dashboards but not others.
Add up what it would cost when your headcount doubles.

If the tool passes those four tests, you've learned more than any demo could tell you.

Common mistakes that quietly kill BI projects

Most bad BI decisions come from the same handful of errors. Watch for these:

Buying for the team you wish you had. Choosing a powerful, complex tool because someday you'll hire a data team leads to low adoption right now. Buy for the people in the room today.

Judging cost by the sticker price. The monthly fee is only part of the total cost of ownership. Factor in setup time, ongoing maintenance, and — the big one for small teams — how pricing grows. A tool that's $20/user feels cheap at five people and painful at fifty. Model both.

Skipping the trial. As long as the spreadsheet still works, people keep using it. If you don't prove the new tool is easier during evaluation, it won't magically become easier after you buy.

Over-engineering for scale you may never reach. The flip side of buying too small is buying an enterprise platform "to be safe." The best tool for a 30-person company is rarely the best tool at 200 people, and vice versa. Aim for where you'll be in roughly 18 months — not 5 years, and not today.

Ignoring who sees what. It's tempting to skip permissions early on. But the day you show a customer's dashboard to the wrong customer, or a junior hire stumbles onto salary data, you'll wish you'd checked for row-level security and role-based access up front.

Where different tools fit

You don't have to pick a category blind. Broadly, small teams land in one of a few buckets: free, lightweight tools (great for marketing dashboards and getting started), self-service BI platforms (a balance of power and approachability for most growing teams), and heavier enterprise suites (usually overkill until you have a dedicated data team). If your data lives in a SQL database and you want dashboards and reports without a big setup, a self-service tool pointed straight at that database — Draxlr is one example — often gets a small team from raw tables to a live dashboard quickly. The category matters more than any single brand: match it to your team, not to a leaderboard.

Key takeaways

Choosing a BI tool comes down to a few disciplined moves. Write down your requirements before you look at features. Judge every option on ease of use, time to value, data connections, pricing that scales, and governance — in that order for a small team. Score your shortlist instead of admiring demos. And above all, run a real trial with your own data and your own people, because adoption is where most BI projects live or die. Get those right and you'll pick a tool your team actually opens on Monday morning — which is the only measure of a BI tool that counts.

Your turn

How did you choose your current analytics tool — and would you make the same call again? What's the one criterion you underrated the first time around? Drop your experience (and the tools you've loved or abandoned) in the comments — small teams learn best from each other's mistakes.

Read-Only Isn't Enough: Query Guardrails That Keep AI From Taking Down Your Database

Vivek Kumar — Mon, 21 Sep 2026 02:24:51 +0000

You did the responsible thing. Before pointing Claude, Cursor, or an in-app assistant at your database, you created a read-only role. INSERT, UPDATE, DELETE, DROP — all rejected. The AI can look but not touch. Safe, right?

Not quite. Read-only protects your data from being modified. It does nothing to protect your database server from being overwhelmed. And the way large language models write SQL — confidently, sometimes with a missing join condition or a forgotten WHERE — makes them very good at producing queries that are perfectly valid, perfectly read-only, and perfectly capable of pinning your CPU to 100% at 2pm on a Tuesday.

This is the guardrail layer most teams skip. It's not about permissions; it's about resource limits. Below are the guardrails that matter, why each one exists, and how to actually set them up. They're vendor-neutral and apply whether you're wiring up a raw connection, an MCP server, or an internal reporting tool.

The failure mode read-only doesn't cover

Imagine an AI assistant is asked, "How many orders did each customer place last year?" It knows there's an orders table and a customers table, so it writes:

-- The AI forgot the join condition
SELECT c.name, COUNT(*)
FROM customers c, orders o
GROUP BY c.name;

That comma join with no ON clause is a cartesian product. If you have 50,000 customers and 2 million orders, the database tries to build a 100-billion-row intermediate result before it ever gets to counting. It's a legal SELECT. Your read-only role happily allows it. And it will chew through memory and CPU until something falls over.

The same thing happens with an innocent-looking SELECT * FROM events against a table with 500 million rows, or an unindexed ORDER BY on a huge dataset. None of these are attacks. They're just the ordinary output of a model that doesn't know how big your tables are. So the job of the guardrail layer is to make sure that even a badly-shaped query fails cheaply instead of expensively.

Guardrail 1: Cap every result set

The first and simplest defense: never let a query return an unbounded number of rows. Even if the AI writes SELECT * FROM events, wrap what it sends in an outer limit before executing:

-- What the AI generated
SELECT * FROM events WHERE user_id = 42;

-- What you actually run
SELECT * FROM (
  SELECT * FROM events WHERE user_id = 42
) AS ai_query
LIMIT 1000;

A sensible default is somewhere between 500 and 1,000 rows. The AI almost never needs more than that to answer a question or summarize a trend, and the cap protects both your database (less data to materialize and sort) and the process consuming the results from an out-of-memory surprise. Make the limit configurable, but always have one on by default. "No limit" should never be the default state for a query you didn't write yourself.

Guardrail 2: Kill runaway queries with a statement timeout

A row limit caps what comes back, but a cartesian join can burn resources long before it produces a single row. That's what a statement timeout is for: it tells the database to abort any query that runs longer than a set duration.

In PostgreSQL you can set this per role, so it applies automatically to every session the AI opens:

-- Any query from ai_readonly that runs longer
-- than 30 seconds gets terminated automatically
ALTER ROLE ai_readonly SET statement_timeout = '30s';

MySQL has an equivalent via max_execution_time (as a query hint or system variable). The exact number depends on your workload — 30 seconds is a common starting point for interactive/analytical use — but the principle is universal: a query the AI wrote should never be able to run indefinitely. When it hits the ceiling, the database cancels it and returns a clean error the AI can react to, instead of silently degrading everything else on the box.

Crunchy Data puts it well: a statement timeout ensures no connecting client "will have queries running longer than that." It's one line of config for an enormous reduction in blast radius.

Guardrail 3: Point the AI at a read replica, not production

The strongest structural guardrail isn't a setting — it's which server the AI talks to. If the assistant only needs to read and analyze, it never has any business touching your primary database. Point it at a read replica instead.

This buys you two things at once. First, exploratory and reporting queries don't compete with real user traffic for CPU, memory, or locks on your hot tables — a heavy analytical scan on the replica leaves checkout and login untouched on the primary. Second, a replica is read-only by the laws of physics: a PostgreSQL standby will reject writes no matter what, so even if you fumbled the read-only role config, there's a hard floor under you. It's defense in depth.

Setup	Protects data from writes	Protects production from load	Operational cost
Full-access role on primary	No	No	Low
Read-only role on primary	Yes	No	Low
Read-only role + timeout + row cap on primary	Yes	Partially	Low
Read replica with the same guardrails	Yes	Yes	Higher

A replica does add operational overhead — a second server, replication lag to monitor, failover to think about — so it's not always worth it. For a small database or a tiny team, a read-only role with a timeout and a row cap on the primary gets you most of the safety. But once your database matters and your AI usage is more than occasional, the replica is the cleanest boundary you can draw.

Guardrail 4: Constrain the connection itself

Timeouts and row limits handle a single bad query. A connection limit handles many bad queries. If an AI agent (or a customer-facing feature powered by one) can open unlimited concurrent sessions, a burst of activity can exhaust your connection pool and lock out your real application.

-- Cap how many simultaneous connections
-- the AI's role can hold open
ALTER ROLE ai_readonly CONNECTION LIMIT 5;

Combine that with the statement timeout from earlier and, as one Postgres engineer put it, a "statement timeout and connection limit on a read-only role gives you 90% of the safety with 10% of the complexity." It's a great return on two lines of SQL.

Guardrail 5: Inspect the query shape (advanced)

For higher-stakes setups, you can validate the structure of a generated query before running it. A reporting query has a predictable shape: SELECT ... FROM ... WHERE .... If one suddenly references system catalogs, stacks multiple UNIONs, or joins a dozen tables, you can reject it at the application layer before it reaches the database. This is easy to over-engineer, so treat it as a layer you add when the earlier four aren't enough — the row cap, timeout, and replica cover the common cases with far less effort.

Where MCP fits

A lot of teams reach for the Model Context Protocol to connect AI clients to their databases, and a well-built MCP server is a natural home for these guardrails: the AI client sends a request, and the server — not the model — decides what actually runs. That's exactly the right place to enforce SELECT-only access, an outer row limit, and a statement timeout, so the same rules apply no matter which AI tool is on the other end.

You can build this yourself, or use a managed MCP server that ships with the guardrails in place — Draxlr's, for example, is read-only (SELECT only) and sits between the AI and your database — so you inherit the safe defaults instead of hand-rolling them. Either way the point is the same: enforcement lives in the broker, where the model can't talk it out of them.

Common mistakes and gotchas

Assuming read-only means safe. It means safe from writes. Resource exhaustion is a separate problem with separate fixes.

Setting the timeout too high "just in case." A 10-minute timeout on an interactive assistant defeats the purpose. If a query genuinely needs longer, run it deliberately, not through the AI's default path.

Forgetting the timeout on the replica. Replicas need their own statement_timeout — the setting doesn't automatically follow from the primary. A runaway query on an unbounded replica is still a runaway query.

Relying on the AI to add its own LIMIT. It often will. It sometimes won't. Guardrails you enforce are the ones you can count on; guardrails you hope the model remembers are not guardrails.

No connection ceiling. Everyone tunes the single-query case and forgets that concurrency can take you down just as easily.

Key takeaways

Read-only access is the floor, not the ceiling, of safe AI database access. The queries that hurt you won't be malicious writes — they'll be well-meaning, valid SELECTs that scan too much. Cap every result set with an outer LIMIT, set a statement_timeout so runaway queries die cheaply, put a CONNECTION LIMIT on the role, and — when it's worth the cost — point the AI at a read replica so its worst query can't touch production. Enforce these in a broker or connection layer, not in the prompt, so they hold regardless of what the model writes.

Your turn

How are you protecting your database from AI-generated queries — read replica, tight timeouts, query inspection, or something else? Have you had an assistant produce a query that surprised you? Drop it in the comments; the war stories are usually the most useful part.

Sources: Crunchy Data — Control Runaway Postgres Queries With Statement Timeout, Rietta — Protect Production SQL Databases from AI/LLM Agentic SQL Query Risks, Model Context Protocol.

Giving AI Access to Your Database? Hide the PII First

Vivek Kumar — Sun, 20 Sep 2026 09:26:09 +0000

You wire up an AI assistant to your production database. You ask it a harmless question — "how many active users signed up last week?" — and it happily writes the SQL. Nice.

But here's the part nobody thinks about until it bites them: that assistant can read every column your connection can reach. users.email. customers.phone. payments.card_last4. patients.diagnosis. The moment you gave it a way to run SELECT, you also gave it a way to pull personal data into a chat window, a log file, or an LLM provider's context — often without anyone intending it.

This isn't a reason to keep AI away from your data. It's a reason to be deliberate about which data it can see. The good news: the tools to do this already live in your database, and the patterns are the same whether the query comes from an AI agent, a BI tool, or a junior analyst. Let's walk through them, from the crudest to the cleanest.

The core problem: AI inherits your connection's blast radius

An AI assistant querying your database is only as constrained as the credentials behind it. If it connects as a superuser or your app's main role, it sees everything that role sees. Security researchers reviewing the Model Context Protocol (MCP) — the standard many tools now use to connect AI to databases — repeatedly flag over-permissioning as the fastest way these integrations go wrong: the connector exposes more than the task needs, and the agent returns data well beyond what you'd ever want in a prompt.

So the first principle is boring but non-negotiable: least privilege. The AI should connect through its own dedicated, read-only role that can touch only what it genuinely needs. Everything below builds on that.

-- A dedicated, read-only role for AI/analytics access
CREATE ROLE ai_reader NOLOGIN;

-- No blanket access to the whole schema
REVOKE ALL ON ALL TABLES IN SCHEMA public FROM ai_reader;

-- Grant only what's needed, table by table
GRANT SELECT ON analytics_events TO ai_reader;

Layer 1: Column-level GRANT

Most people know GRANT SELECT ON table. Fewer know PostgreSQL (and MySQL 8+, with slightly different syntax) lets you grant SELECT on specific columns. If a table mixes public and sensitive fields — and most do — this is the sharpest tool you have.

Say your users table looks like this: id, name, email, phone, country, plan, created_at. The AI needs country, plan, and created_at to answer product questions. It has no business reading email or phone.

-- Remove table-wide read access
REVOKE SELECT ON users FROM ai_reader;

-- Grant only the safe columns
GRANT SELECT (id, country, plan, created_at) ON users TO ai_reader;

Now if the AI writes SELECT email FROM users, the database itself rejects it with a permission error — before a single row of PII is touched. You didn't have to trust the model, the prompt, or the tool. The constraint lives where it belongs: in the database.

The trade-off is bookkeeping. As tables grow, tracking who can see which column gets fiddly. A simple column access matrix keeps you honest:

Column	ai_reader	support_role	app_role
country, plan, created_at	read	read	read/write
email, phone	—	read	read/write
card_last4	—	—	read/write

Layer 2: Masking views (keep the shape, hide the value)

Sometimes you don't want to hide a column — you want the AI to see a masked version. It's useful for the model to know an email exists and group by its domain, without ever reading the real address. This is where views shine.

Create a view that transforms sensitive columns, revoke access to the base table, and point the AI role only at the view:

CREATE OR REPLACE VIEW users_ai AS
SELECT
  id,
  country,
  plan,
  created_at,
  -- keep the domain, drop the local part
  '***@' || split_part(email, '@', 2) AS email_domain,
  -- last 2 digits only, for support triage
  'xxx-xxx-' || right(phone, 2)        AS phone_masked
FROM users;

REVOKE ALL   ON users     FROM ai_reader;
GRANT  SELECT ON users_ai  TO   ai_reader;

One important detail: add WITH (security_barrier) to the view. Without it, the planner can sometimes push a WHERE clause below your masking expression and leak the underlying value. The barrier forces your masking to run first.

CREATE VIEW users_ai WITH (security_barrier) AS
  SELECT ... FROM users;

Here are the masking styles worth knowing:

Technique	What it does	Good for
Nulling / redaction	Replaces the value with NULL or a constant	Fields AI should never read
Partial masking	Shows a fragment (domain, last 2 digits)	Triage, grouping, dedup
Substitution	Swaps in realistic fake values	Demos, shared environments
Hashing	Deterministic token instead of the value	Joins/counts without exposure

Layer 3: Add row-level security for two-dimensional control

Column controls decide which fields; row-level security (RLS) decides which rows. Combine them and you get precise, two-dimensional access — the AI sees safe columns, and only the rows it's allowed to.

This matters most for multi-tenant apps, where an AI feature answers questions for one customer and must never surface another customer's records.

ALTER TABLE orders ENABLE ROW LEVEL SECURITY;

CREATE POLICY tenant_isolation ON orders
  FOR SELECT
  TO ai_reader
  USING (tenant_id = current_setting('app.tenant_id')::int);

With the tenant ID set per session, the same AI query returns only that tenant's rows — no matter how the question is phrased. Column masking hides the what; RLS hides the whose.

Layer 4: The connection layer matters too

Database grants are the backstop, but a well-designed access layer between the AI and your database adds guardrails that SQL alone can't. The MCP security literature converges on a few:

Read-only enforcement. Reject UPDATE, DELETE, and DDL outright, so an errant (or manipulated) prompt can't modify data.
Redaction in the data flow. Scan tool responses and store only redacted representations when PII or secrets slip through.
Scoped exposure. Publish only approved tables and queries to the AI, not the whole schema.
Auditability. Log every query the AI runs, so access is reviewable and revocable — not scattered across long-lived credentials in prompts and config files.

This is a big part of why managed database brokers exist. A gateway like Draxlr's MCP server, for example, connects over OAuth and is read-only by design, so the AI can explore schema and run SELECTs without ever holding raw credentials or issuing a write. Whatever tool you use, the pattern is the point: put a controllable layer between the model and the database, and enforce PII rules there and in the database itself. Defense in depth beats trusting any single boundary.

Common mistakes and gotchas

Connecting AI as the app's main user. The single biggest error. Give it a dedicated, minimal, read-only role.
Masking in the SELECT but leaving the base table readable. If the role can still hit the base table, your view is theater. Revoke base access.
Forgetting security_barrier on masking views. The planner may leak the raw value through a pushed-down predicate. Set the barrier.
Masking the value but not the WHERE. WHERE email = 'jane@acme.com' in an AI-written query can confirm a specific person exists even if the output is masked. Restrict filtering on sensitive columns too.
Assuming "internal only" means safe. Data pulled into an LLM context can be logged or cached downstream. Treat every AI query as if it might leave your walls.
No audit trail. If you can't answer "what did the AI read last Tuesday?", you can't prove compliance or catch abuse.

Key takeaways

Giving AI access to your database is not the risky part — giving it unscoped access is. Lead with least privilege and a dedicated read-only role. Use column-level GRANT to hide sensitive fields entirely, and security_barrier masking views when you want the shape without the value. Layer in row-level security for multi-tenant isolation, and put an auditable, read-only broker between the model and the database so the rules are enforced twice. Do that, and your AI assistant becomes genuinely useful and something your security team can sign off on.

Your turn

How are you handling PII when AI tools touch your database — column grants, masking views, a broker, or all three? Have you hit a gotcha I didn't list? Drop it in the comments; I'm collecting patterns and would love to hear what's working (or what blew up) in your setup.

The Dialect Trap: Why AI Writes SQL Your Database Refuses to Run

Vivek Kumar — Fri, 18 Sep 2026 01:02:03 +0000

You ask an AI assistant for "the 10 most recent orders." It confidently hands you this:

SELECT TOP 10 *
FROM orders
ORDER BY created_at DESC;

Looks fine — until you run it on PostgreSQL and get syntax error at or near "10". The logic was perfect. The dialect was wrong. TOP is SQL Server syntax; Postgres wants LIMIT.

This is one of the most common and most frustrating failure modes of text-to-SQL. The query reads like valid SQL, passes a human eyeball test, and still bounces off your database because SQL isn't really one language — it's a family of closely-related dialects that disagree on the details. If you're building any kind of natural-language-to-SQL feature (or just pasting AI output into a query console), understanding the dialect trap will save you a lot of confusing error messages.

SQL is a standard that nobody fully follows

There's an ANSI SQL standard, but every database vendor extends and diverges from it. The core SELECT ... FROM ... WHERE is portable. Almost everything interesting around it is not. Here's a small slice of where the major engines disagree:

Task	PostgreSQL	MySQL	SQL Server	BigQuery
Limit rows	`LIMIT 10`	`LIMIT 10`	`TOP 10`	`LIMIT 10`
Concatenate strings	`a \|\| b`	`CONCAT(a, b)`	`a + b`	`CONCAT(a, b)`
Current timestamp	`NOW()`	`NOW()`	`GETDATE()`	`CURRENT_TIMESTAMP()`
Extract month	`EXTRACT(MONTH FROM d)`	`MONTH(d)`	`DATEPART(month, d)`	`EXTRACT(MONTH FROM d)`
Quote an identifier	`"my col"`	`my col`	`[my col]`	`my col`

None of these are cosmetic. Each one is the difference between a query that runs and a query that throws. And some differences are worse than a clean error: they run and quietly return the wrong result.

Why AI gets this wrong so often

Large language models learn SQL from the enormous pile of SQL on the public internet. Two things about that pile work against you.

First, the training data is dominated by certain dialects. A lot of public text-to-SQL research and tutorial content is written against SQLite and MySQL, so models lean toward that syntax by default. Ask for a query without saying which database you use, and you'll often get SQLite-flavored or MySQL-flavored SQL — which may or may not match what you're actually running. One analysis found that roughly 32% of queries from a popular text-to-SQL benchmark (Spider) threw syntax errors when executed against PostgreSQL, even though they were "correct" for the dialect they were generated in.

Second, models blend dialects. Because the training data mixes every flavor together, a model can start a query in one dialect and finish it in another — using a Postgres || concatenation in the SELECT and a SQL Server TOP in the same statement. The result is SQL that belongs to no real database.

The model isn't "wrong" about SQL. It just doesn't know which of the several SQLs you meant, so it guesses.

The fix starts with telling the model what it's talking to

The single highest-leverage thing you can do is stop making the model guess. State the exact engine — and version — every time.

Compare a vague prompt:

Give me the 10 newest users.

with a dialect-anchored one:

You are writing SQL for PostgreSQL 16.
Return only a query, no explanation.
Give me the 10 newest users.

Schema:
users(id BIGINT, email TEXT, created_at TIMESTAMPTZ, plan TEXT)

The second prompt reliably produces:

SELECT id, email, created_at
FROM users
ORDER BY created_at DESC
LIMIT 10;

Notice two things are doing the work here. "PostgreSQL 16" pins the dialect. And the schema snippet — with real column types — quietly reinforces it: TIMESTAMPTZ and TEXT are Postgres-isms, so the model stays in the right neighborhood. Sharing your schema isn't just about avoiding hallucinated column names; the types are a dialect signal too.

Add a couple of examples in your dialect

If you're building this into a product, a system prompt plus one or two worked examples (few-shot prompting) pins the dialect far more firmly than an instruction alone. You're showing, not just telling:

-- Example (PostgreSQL):
-- Q: revenue per plan last month
SELECT plan, SUM(amount) AS revenue
FROM subscriptions
WHERE created_at >= date_trunc('month', now()) - interval '1 month'
  AND created_at <  date_trunc('month', now())
GROUP BY plan;

That one example teaches the model your date-math idioms (date_trunc, interval), your casing conventions, and your dialect all at once. For a handful of canonical question shapes, a few pinned examples go a long way. For a large, varied set, retrieving the most relevant example pairs per question (a RAG approach) scales better than stuffing them all in.

Validate before you trust — then let the error teach the model

Even with a perfect prompt, treat generated SQL as a draft. The cheapest safety net is to check the query compiles before you run it for real. Most engines give you a dry-run path:

-- PostgreSQL / MySQL: ask the planner to parse & plan without returning rows
EXPLAIN
SELECT plan, COUNT(*) FROM subscriptions GROUP BY plan;

-- BigQuery supports an actual dry run that validates and estimates cost,
-- without scanning data or incurring charges.

If the parse fails, don't just surface a raw error to your user — feed it back to the model:

Your query failed on PostgreSQL 16 with:
ERROR: function datepart(unknown, timestamp with time zone) does not exist

Rewrite it using PostgreSQL-native date functions.

This execution-feedback loop catches most dialect slips automatically. The model used DATEPART (SQL Server); the error tells it exactly what the target engine rejected, and the retry almost always lands on EXTRACT or date_trunc.

A transpiler as a backstop

Prompting reduces dialect errors; it doesn't eliminate them. If you need a hard guarantee, put a SQL transpiler between the model and the database. Libraries like sqlglot can parse SQL written in one dialect and re-emit it in another:

import sqlglot

# Model produced SQL Server-flavored SQL; we run Postgres.
generated = "SELECT TOP 10 name + '!' AS shout FROM users"

fixed = sqlglot.transpile(generated, read="tsql", write="postgres")[0]
# -> SELECT name || '!' AS shout FROM users LIMIT 10

Now even if the model drifts into the wrong dialect, the SQL that reaches your database is normalized to the one you actually run. It also gives you a parse step for free: if the transpiler can't parse it, you reject it before execution.

Common gotchas to watch for

A few dialect traps burn people repeatedly, even when the basic syntax is right:

Silent wrong answers, not errors. Integer division is the classic one: 5 / 2 is 2 in PostgreSQL and SQL Server but 2.5 in MySQL and BigQuery. No error — just a quietly wrong number. These are the dangerous cases validation won't catch.
Case sensitivity of identifiers. Postgres folds unquoted names to lowercase; MySQL's behavior depends on the OS and config. An AI that quotes "UserID" can make a column that worked suddenly "not exist."
Fully-qualified names in BigQuery. BigQuery references tables as `project.dataset.table` and uses STRING instead of VARCHAR, with no timezone-naive TIMESTAMP. Models trained mostly on other engines routinely get this wrong.
NULL handling and string funcs. CONCAT with a NULL argument behaves differently across engines; so does sorting NULLs. Don't assume portability.
Version drift. FETCH FIRST 10 ROWS ONLY works on modern SQL Server and Oracle but not older versions. Saying "SQL Server 2022," not just "SQL Server," matters.

Key takeaways

SQL's dialects are the hidden reason so much AI-generated SQL "looks right but doesn't run." To stay out of the trap:

Do this	Why it helps
Name the engine and version in every prompt	Stops the model from guessing or blending dialects
Pass the schema with real column types	Types are a dialect signal and kill hallucinated columns
Give one or two examples in your dialect	Shows idioms the model can't infer from an instruction
Dry-run / EXPLAIN before executing	Catches syntax errors cheaply, before users see them
Feed execution errors back for a retry	Self-corrects most dialect slips automatically
Transpile as a backstop	Guarantees the SQL that reaches the DB is the right flavor

If you're wiring an AI assistant to a database through a schema-aware layer — a managed connector or MCP server, for example — much of this can be handled for you: the layer already knows which engine it's connected to and can pass that context (and the schema) along so the model writes for the right dialect from the start. That's a nicer place to solve the problem than re-explaining your database in every prompt.

Have you been bitten by the dialect trap? I'm curious which engine trips up your AI tools the most — drop your worst "looked right, wouldn't run" query in the comments, and share the tricks that fixed it for you.

One AI Assistant, Many Databases: Wiring Up Multiple Data Sources with MCP

Vivek Kumar — Thu, 17 Sep 2026 12:58:32 +0000

You wired up an AI assistant to your database and it's great. You ask "what was MRR by plan last month?" and get back working SQL and an answer in seconds. Then reality sets in: that was your analytics database. Your user records live in a Postgres replica. Your billing events sit in a separate MySQL box that a previous team stood up and nobody wants to touch. Real teams almost never have one database — they have a small constellation of them.

So the interesting question isn't "can an AI query my database?" It's "how do I give one assistant safe, sane access to all of my databases at once — and make sure it queries the right one, in the right dialect, without me babysitting every request?"

The Model Context Protocol (MCP) is built for exactly this. Let's walk through the two ways to connect multiple databases, how the AI decides which one to use, the dialect trap that quietly breaks multi-database setups, and the mistakes that bite people along the way.

Quick refresher: what MCP actually connects

MCP is an open standard that lets AI clients — Claude, Cursor, ChatGPT, VS Code, and others — call tools through a consistent interface. The AI application is the host. For every MCP server it talks to, it spins up a dedicated client connection. A database MCP server typically exposes a few tools like list_databases, get_schema, and run_query.

The key idea: the AI never holds your credentials or speaks the Postgres wire protocol. It calls a tool by name, and the server does the real database work behind the credentials it holds. That indirection is what makes connecting several databases tractable — you're composing tool calls, not juggling connection strings in a chat window.

Two ways to connect many databases

There are two patterns, and the right one depends on how uniform your databases are.

Pattern A: one MCP server per database

You run a separate server for each database and register them all with your AI client. Configuration looks like a list of servers:

{
  "mcpServers": {
    "users-postgres": {
      "command": "npx",
      "args": ["-y", "postgres-mcp-server"],
      "env": { "DATABASE_URL": "postgres://readonly:***@replica.internal:5432/users" }
    },
    "analytics-warehouse": {
      "command": "npx",
      "args": ["-y", "postgres-mcp-server"],
      "env": { "DATABASE_URL": "postgres://readonly:***@warehouse.internal:5432/analytics" }
    },
    "billing-mysql": {
      "command": "npx",
      "args": ["-y", "mysql-mcp-server"],
      "env": { "DATABASE_URL": "mysql://readonly:***@billing.internal:3306/billing" }
    }
  }
}

This is flexible — each database can use a purpose-built server — but it has a well-known sharp edge: tool name collisions. If all three servers expose a tool called run_query, the host ends up with three tools of the same name in one aggregated list, and it can call the wrong one. The MCP spec acknowledges this and leaves disambiguation to the host. In practice, hosts and proxies solve it by namespacing tools with the server name — billing-mysql.run_query vs. users-postgres.run_query — so each call routes to the intended server.

Pattern B: one gateway server fronting many databases

The alternative is a single MCP server that knows about all your databases and takes the database as a parameter. Instead of three run_query tools, you get one, plus a list_databases tool the AI calls first:

Tools exposed:
  list_databases()            -> [ {id, name, engine}, ... ]
  get_schema(database_id)     -> tables + columns for that DB
  run_query(database_id, sql) -> rows

A typical session flows: the AI calls list_databases, sees your three sources, picks the relevant one, pulls its schema, then runs a query scoped to that database_id. No collisions, one connection to manage, and one place to enforce read-only access and auditing. Managed MCP servers tend to use this shape — for example, Draxlr exposes a single read-only (SELECT-only) endpoint over OAuth that lists your connected databases and runs schema/query calls per database id. Whichever you pick, the gateway pattern usually scales better once you're past two or three databases.

Here's the trade-off at a glance:

Concern	Server per database	One gateway server
Tool name collisions	Possible; needs namespacing	None — database is a parameter
Adding a database	Edit client config, restart	Register it once in the gateway
Central audit / read-only	Per server	One enforcement point
Mixing DB engines	Easy — different server each	Depends on gateway support
Best for	2–3 heterogeneous databases	Many databases, shared access

How does the AI pick the right database?

This is where good setups separate from frustrating ones. The AI routes based on names and descriptions, not magic. A database registered as db_47 with no description is a coin flip; one called analytics_warehouse described as "aggregated events and revenue, read replica, updated hourly" gives the model everything it needs to choose correctly.

So when you have a question that could plausibly hit two sources, name and describe your databases like you're onboarding a new analyst. Then a prompt like this resolves cleanly:

"How many trial users from the users database converted to paid, using the subscription events in the billing database?"

The assistant recognizes it needs two sources. Since most databases can't join across a network boundary, it does what a human would: query each separately and combine. First, against the users source:

-- users-postgres
SELECT id
FROM users
WHERE plan = 'trial'
  AND created_at >= NOW() - INTERVAL '30 days';

Then, against billing, filtering to those ids:

-- billing-mysql
SELECT DISTINCT user_id
FROM subscription_events
WHERE event_type = 'subscription_started'
  AND user_id IN (/* ids from the previous step */);

The assistant stitches the two result sets together and reports the count. The important behavior: it did not try to write one query joining users.users to billing.subscription_events — those live in different engines on different hosts, and no single query spans them.

The dialect trap

Here's the gotcha that surprises people: the same English question produces different SQL depending on which database answers it. Ask "revenue by month" against Postgres and MySQL and the correct queries are not identical.

Task	PostgreSQL	MySQL
Extract the month	`EXTRACT(MONTH FROM created_at)`	`MONTH(created_at)`
Null fallback	`COALESCE(x, 0)`	`IFNULL(x, 0)`
Concatenate group	`STRING_AGG(name, ', ')`	`GROUP_CONCAT(name)`
Quote an identifier	`"order"` (double quotes)	`order` (backticks)

This is exactly why schema-awareness matters so much in a multi-database setup. When the MCP server tells the AI not just the table shapes but which engine each database runs, the model targets the right dialect. When it doesn't, you get Postgres syntax fired at MySQL and a confusing error. If your assistant keeps generating queries in the wrong dialect, that's usually the tell that the engine isn't being surfaced in the schema.

Common mistakes and gotchas

Assuming the AI can join across databases. It can't run a single query across two separate engines. Expect a query-each-then-combine approach, and know that huge intermediate result sets (a million ids in an IN (...) list) will be slow or get truncated.

Vague database names. db1, db2, prod_copy_final — the model can't route what it can't understand. Descriptive names and one-line descriptions are the cheapest accuracy win available.

Pointing at primary databases. Exploratory AI queries can be heavy and unpredictable. Connect read replicas, and keep access read-only so an experimental query can never issue an UPDATE or DROP.

Ignoring tool-name collisions. In the server-per-database pattern, verify your host namespaces tools. Two run_query tools with no prefix is a silent routing bug waiting to happen.

Dumping every schema into context at once. Ten databases with hundreds of tables each can blow past the model's context window. Prefer a list_databases → get_schema(one_db) flow that fetches schema on demand rather than loading everything up front.

Key takeaways

Connecting multiple databases to one AI assistant comes down to a few decisions. Use a server per database for a handful of heterogeneous sources, or a single gateway that takes the database as a parameter once you have many. Name and describe every database clearly so the AI routes correctly. Respect dialect differences by making sure each database's engine is part of what the AI sees. Don't expect cross-engine joins — expect query-and-combine. And keep every connection read-only against a replica so exploration stays safe.

Get those right and you go from "the AI can talk to a database" to "the AI understands our whole data estate" — which is where it actually starts saving your team time.

How many databases would your assistant need to reach to be genuinely useful — and are they named well enough for it to tell them apart? I'd love to hear how you've wired up multi-database access, and what tripped you up. Drop a comment.

Local vs. Remote MCP Servers for Your Database: Which One Should You Actually Use?

Vivek Kumar — Fri, 11 Sep 2026 06:37:02 +0000

You wired up an AI assistant to your database last week. It reads your schema, writes surprisingly good SQL, and answers questions like "what was MRR by plan last month?" in seconds. It works great — on your laptop.

Then a teammate asks for the same thing. Or you want it running in CI. Or your security lead asks where, exactly, the production database password is stored. Suddenly the setup that felt magical raises a real architecture question: should the MCP server that connects the AI to your database run locally on each machine, or remotely as a shared service?

This is the fork in the road most teams hit once "AI can talk to our database" stops being a demo and starts being infrastructure. The two options behave very differently in terms of speed, security, and who can use them. Let's break down how each works, when each makes sense, and the hybrid pattern that tends to win in practice.

Quick refresher: what an MCP server does for a database

The Model Context Protocol (MCP) is an open standard that lets AI clients — Claude, Cursor, ChatGPT, VS Code, and others — call tools through a consistent interface. An MCP server for a database typically exposes a handful of tools like list_tables, get_schema, and run_query. The AI doesn't hold your credentials or speak Postgres wire protocol; it calls a tool, and the server does the actual database work behind the credentials it holds.

That indirection is the whole point. But where that server runs — and how the client talks to it — is exactly what "local vs. remote" decides.

The two transports: stdio and Streamable HTTP

MCP defines two standard transports, and they map almost one-to-one onto local vs. remote.

stdio is for local servers. The client launches the server as a child process and talks to it over standard input/output — JSON-RPC messages in, JSON-RPC messages out. No network, no ports. It's the natural fit when the server and client live on the same machine.

Streamable HTTP is for remote servers. The server runs as an independent, long-lived process that clients reach over HTTP (optionally streaming responses via Server-Sent Events). It's built for network access, multiple concurrent clients, and central hosting. It replaced the older HTTP+SSE transport.

Here's the practical contrast:

Property	stdio (local)	Streamable HTTP (remote)
Where it runs	Same machine as the AI client	A shared host or cloud service
Concurrent users	One client, one process	Many clients at once
Auth model	Reads credentials from the local environment	OAuth 2.1 at the transport layer
Audit / central control	None built in	Single point to log and revoke
Latency	Near-zero (no network hop)	Network round-trip
Best for	Solo dev work, local/dev databases	Teams, customers, production data

The spec's own guidance: clients should support stdio whenever possible, but you switch to Streamable HTTP the moment the server needs to be reachable by more than one machine.

Local MCP servers: fast, private, single-player

A local server runs on your box and connects to a database it can reach directly. Configuration usually looks like a small block in your AI client's settings that tells it how to launch the process:

{
  "mcpServers": {
    "analytics-db": {
      "command": "npx",
      "args": ["-y", "some-postgres-mcp-server"],
      "env": {
        "DATABASE_URL": "postgres://readonly:***@localhost:5432/appdb"
      }
    }
  }
}

Now you can ask questions in plain English and the server turns them into SQL against your schema:

You: How many trial users converted to paid last month?

SELECT COUNT(DISTINCT u.id) AS converted_users
FROM users u
JOIN subscriptions s ON s.user_id = u.id
WHERE u.trial_started_at >= '2026-08-01'
  AND s.status = 'active'
  AND s.started_at >= '2026-08-01'
  AND s.started_at <  '2026-09-01';

 converted_users
-----------------
             418

This is a great developer experience: zero network latency, nothing leaves your machine, and it's dead simple to spin up against a local or dev database. For solo exploration and building queries, local is hard to beat.

The catch shows up when you try to scale it to people. stdio has no transport-layer authentication, so it inherits whatever credentials sit in your local environment. Every teammate who wants the same capability copies a config file with a connection string in it. There's no central audit log, no shared query cache, and no single place to rotate a leaked password. It's single-player by design.

Remote MCP servers: shared, governed, multi-player

A remote server flips the model. It runs once, as a service, and every AI client connects to the same endpoint over HTTP — authenticating with OAuth instead of a pasted connection string.

{
  "mcpServers": {
    "analytics-db": {
      "type": "http",
      "url": "https://mcp.internal.example.com/mcp"
    }
  }
}

The first time a client connects, it runs an OAuth flow and receives a short-lived token. The database credentials never touch the client, the prompt, or anyone's config file — they live only on the server. Because there's one gateway, you also get the things stdio can't offer: a shared connection pool, centralized audit logs of every query the AI ran, per-user or per-tenant scoping, and one-click revocation when someone leaves the team.

Managed options exist here too. Draxlr, for example, runs a hosted MCP server you connect as a read-only OAuth custom connector — one implementation of the remote pattern where the server holds the credentials and only SELECT gets through. Whether you self-host or buy, the defining trait of the remote model is the same: the database stops being directly exposed to every developer's laptop, which shrinks your attack surface considerably.

The trade-off is operational weight. Someone has to run that service, keep OAuth configured correctly, and monitor it. There's a real network hop. And a misconfigured public endpoint is a much bigger deal than a stdio process that only you can launch.

A decision framework

If you…	Lean toward
Are one developer exploring a local or dev database	Local (stdio)
Need the lowest possible latency and full privacy	Local (stdio)
Want several teammates to share the same access	Remote (HTTP)
Must keep production credentials off end-user machines	Remote (HTTP)
Need audit logs, revocation, or per-tenant scoping	Remote (HTTP)
Are exposing data to customers or non-technical staff	Remote (HTTP)

The pattern is simple: local optimizes for one developer's speed and privacy; remote optimizes for a team's governance and reach. The more people and the more sensitive the data, the further you slide toward remote.

The hybrid setup most teams land on

You don't have to pick one forever. A very common progression:

Develop locally. Run a stdio server against a dev or read-replica database while you build and test queries. Fast feedback, no infrastructure.
Promote to remote. Once a workflow is worth sharing, put it behind a remote HTTP server with OAuth so the whole team — and their various AI clients — hit the same governed endpoint.
Keep production remote-only. Never let a stdio config with production credentials float around laptops. Production data lives behind the shared, audited gateway.

This gives you the local dev loop and the centralized control, without forcing everyone through a network hop while they're just iterating on a query.

Common mistakes and gotchas

Putting production credentials in a stdio config. The single most common footgun. A DATABASE_URL in a local config is fine for a dev database and dangerous for production — it's a plaintext secret that gets copied, committed, and forgotten. Keep production behind a remote server.

Assuming stdio is "secure" because it's local. Local means private to that machine, not governed. There's no audit trail and no way to revoke access short of changing the database password for everyone.

Standing up a remote server without OAuth. A remote MCP endpoint without proper authorization is just an unauthenticated database proxy on the internet. If you go remote, OAuth 2.1 (short-lived tokens, central revocation) is table stakes, not a nice-to-have.

Forgetting read-only. Whichever model you choose, scope the database role the server uses to SELECT only. An AI exploring your data should never be one hallucinated DELETE away from a bad day. Enforce it at the database and the server layer.

Ignoring multi-tenancy. If customers or different teams share one remote server, a single missing WHERE tenant_id = ... leaks everyone's data. Push tenant isolation into row-level security or scoped credentials — don't rely on the AI to remember it.

Key takeaways

MCP's two transports map cleanly onto the choice: stdio for local, Streamable HTTP for remote.
Local wins for a single developer: fast, private, trivial to set up against dev databases.
Remote wins for teams and production: OAuth instead of pasted credentials, central audit and revocation, one endpoint for many AI clients, and a database that isn't exposed to every laptop.
Most teams use both — build locally, share and ship remotely, keep production strictly behind the gateway.
Regardless of model: read-only roles, OAuth for anything networked, and real tenant isolation.

Your turn

How are you running MCP against your database today — a local stdio server per developer, a shared remote one, or some hybrid? What tipped you from one to the other? Drop your setup (and your favorite gotcha) in the comments — I'd love to hear what's working for other teams.

Share the Schema, Not the Password: How Schema-Aware AI Stops Inventing Columns

Vivek Kumar — Sun, 06 Sep 2026 04:04:34 +0000

You ask an AI assistant, "How many active subscriptions did we add last month?" It confidently returns a query:

SELECT COUNT(*)
FROM subscriptions
WHERE status = 'active'
  AND signup_date >= '2026-08-01';

Clean, readable, and completely wrong for your database. Your table calls the column created_at, not signup_date. There is no status column — you track state in canceled_at IS NULL. The query either fails loudly with column "signup_date" does not exist, or worse, it runs against a lookalike column and hands you a confident, incorrect number that nobody catches until it's in a board deck.

This is a hallucinated column, and it happens for one simple reason: the model was guessing. It never saw your schema, so it invented plausible names based on the thousands of SaaS databases in its training data. The fix is not a cleverer prompt. It's giving the model the one thing it was missing — your actual schema — without handing over the keys to the whole database.

Why the model guesses in the first place

An LLM is a pattern machine. Ask it for SQL and give it nothing but the question, and it will produce SQL shaped like the most common answer to questions like yours. If most subscriptions tables in its training data have a status column, that's what you get — regardless of what your table actually looks like.

There are two obvious but bad ways to fix this. The first is to paste your schema into the chat by hand every time. That works for one query and falls apart by the third, and it goes stale the moment someone runs a migration. The second is to hand the AI a live database connection string so it can look things up itself. That "fixes" accuracy by creating a much bigger problem: now an AI tool holds credentials that can read every row of PII and write to production, and your chat log is one screenshot away from leaking them.

The better framing separates two things that usually travel together:

What the AI actually needs	What it does NOT need
Table names, column names, and types	Your database password
Primary and foreign keys	Network access to the DB host
Constraints and relationships	Write or DDL permissions
Enough structure to write correct SQL	The ability to read raw rows unsupervised

To write correct SQL, the model needs your schema. It does not need your credentials. Once you see that split, the design almost writes itself: share the map, keep the keys.

Schema-awareness: grounding the model in reality

The term for feeding the model your real structure before it writes anything is schema grounding (or schema-aware reasoning). Instead of asking "what would a subscriptions query usually look like," the model first reads the actual definition:

-- What the model reads before writing a single line of SQL
CREATE TABLE subscriptions (
  id           BIGINT PRIMARY KEY,
  account_id   BIGINT REFERENCES accounts(id),
  plan         TEXT NOT NULL,
  created_at   TIMESTAMPTZ NOT NULL,
  canceled_at  TIMESTAMPTZ            -- NULL means still active
);

Now the same question produces SQL grounded in what exists:

SELECT COUNT(*)
FROM subscriptions
WHERE canceled_at IS NULL
  AND created_at >= '2026-08-01'
  AND created_at <  '2026-09-01';

No status. No signup_date. The model can't invent region or revenue out of thin air, because the boundaries of what it's allowed to reference are set by the schema in front of it. Grounding doesn't make the model smarter — it makes the space of possible answers smaller and truer.

Where the connection layer comes in

Manually pasting that CREATE TABLE block works for a demo. In real life you want the schema delivered automatically, kept current, and delivered without also delivering the credentials. That's exactly the job of a database connector layer — and it's the core idea behind the Model Context Protocol (MCP), the open standard for connecting AI clients to tools and data sources.

The pattern looks like this: a small server sits between the AI client and your database. It holds the connection details. It exposes a couple of capabilities to the AI — typically a way to read the schema and a way to run a read-only query — and nothing else. When the AI needs to write SQL, it asks the server for the schema, gets back real table and column definitions as structured data, and generates its query against that. When it runs the query, the server executes it on a read-only connection and returns rows.

// A generic, vendor-neutral connector config — note there is no
// password anywhere near the AI client
{
  "mcpServers": {
    "analytics-db": {
      "url": "https://your-broker.example.com/mcp",
      "auth": "oauth"          // token brokered here, never pasted into a prompt
    }
  }
}

A typical exchange, in plain terms:

Step	Who does it	What crosses the wire
1. "List the tables"	AI → broker	Table names only
2. "Describe `subscriptions`"	AI → broker	Columns, types, keys
3. Generate SQL	AI (locally)	Nothing — it's grounded now
4. Run the SELECT	broker → DB	Rows back, read-only

The AI got everything it needed to be accurate and nothing it could use to be dangerous. Managed brokers like Draxlr's MCP server implement exactly this shape — OAuth-based, read-only (SELECT only), able to hand the model your schema and run queries without the assistant ever seeing a credential — but the pattern is the point, and you can build a minimal version of it yourself in an afternoon.

The security dividend

Schema-sharing started as an accuracy fix, but it quietly solves a security problem too. Because the AI never receives credentials:

There are no long-lived secrets sitting in prompts, chat history, or config files that get committed to a repo by accident.
Access is centrally revocable — kill the token at the broker and every connected client loses access at once, no credential rotation across a dozen laptops.
The database is not network-exposed to every machine running an AI client; only the broker talks to it.
A read-only, SELECT-only connection means even a perfectly-worded "delete all canceled accounts" request has nowhere to land.

You get more accurate SQL and a smaller attack surface from the same architectural decision. That's rare.

Common mistakes and gotchas

Dumping a 400-table schema into context. Schema grounding consumes tokens, and a giant schema can blow past the context window or bury the relevant tables in noise. Expose a focused subset, or let the model list tables first and pull only the definitions it needs.

Sharing schema but skipping read-only enforcement. Grounding fixes accuracy, not authorization. If the underlying connection can write, a cleverly phrased request (or a prompt-injection in your data) can still do damage. Enforce SELECT-only at the database role, not just by asking nicely in a system prompt.

Assuming grounded means correct. A model that can see your schema will stop inventing columns, but it can still get joins or business logic wrong — using created_at when you meant activated_at. Always eyeball the generated SQL before it hits production, especially for anything aggregated.

Letting the schema go stale. If your connector caches the schema, a migration can put you right back to hallucinated columns. Make sure schema reads reflect the live structure, or refresh the cache on deploy.

Leaking data through the schema itself. Column names like patient_ssn or comments in your DDL can be sensitive. Share the structure you want the AI to see, not necessarily every internal table.

Key takeaways

AI invents columns because it's guessing without your schema. The fix is to ground it in your real structure — but you can do that without ever handing over your credentials. Split the two: the model needs the map (tables, columns, types, keys) to write correct SQL; it does not need the keys (passwords, write access, raw network reach) to do its job. A connector or MCP-style broker delivers the schema automatically, keeps it current, runs queries read-only, and holds the credentials so the AI never sees them. You end up with SQL that actually runs and a database that's harder to hurt.

Have you connected an AI assistant to your database yet? What broke first — the hallucinated columns, or the security review? Drop your setup (and your favorite "there is no such column" story) in the comments.

Sources and further reading: Introducing the Model Context Protocol (Anthropic), MCP Specification, Reducing Hallucinations in Text-to-SQL (Wren AI), Improving Text-to-SQL Accuracy with Schema-Aware Reasoning, Building MCP servers for your database (Microsoft). If you want a managed, read-only MCP server that implements this pattern, see the Draxlr MCP docs.

Let Non-Technical Teammates Query the Database — Without Handing Out DB Access

Vivek Kumar — Sat, 05 Sep 2026 09:53:23 +0000

You know the messages. "Quick one — how many trial users converted last month?" "Can you pull the top 20 accounts by revenue?" "What's our churn for the enterprise plan?" Each one is a two-minute query for you, and a half-day wait for the person asking. Multiply that across a sales team, a support team, and a couple of founders, and you've quietly become a human query API.

There are two obvious ways out of this, and both are traps. You can give people direct database access — hand out a read-only connection string and a SQL editor and tell them to help themselves. Now credentials for your production data are sitting in six laptops, three password managers, and at least one Slack DM, and you have no idea who ran what. Or you can keep the ticket queue and stay the bottleneck. Neither is good. This post is about a third option: putting an AI assistant in front of the database so people can ask questions in plain English, while the actual credentials, the read/write boundary, and the row-level scoping all stay firmly under your control.

The problem with the two easy answers

Self-serve analytics has been promised for a decade and mostly hasn't landed. When teams lock everything down, business users go right back to filing tickets and the data team is the bottleneck again. When teams open everything up, adoption soars but trust collapses — the same metric gets defined three different ways and nobody knows which "active users" number is real. And underneath both failure modes is a security question people gloss over: who actually holds the keys to the database?

Handing raw credentials to non-technical teammates fails on every axis. Credentials leak. They're long-lived and painful to rotate. A well-meaning person can run a monster query that locks a table on your primary during peak traffic. And you get no audit trail worth the name. What you actually want is for people to get answers without ever touching a credential.

The third option: an AI assistant behind a broker

The pattern that makes this safe is a broker (increasingly, a Model Context Protocol server) that sits between the AI tool and your database. The person talks to an AI assistant — Claude, Cursor, ChatGPT, whatever they already use. The assistant doesn't connect to your database. It connects to the broker. The broker holds the real connection and enforces the rules.

Here's the important part: the AI never learns your database credentials. It sends a request like "run this SELECT" to the broker, and the broker — which authenticates the user, not the model — decides whether to run it and with what privileges. Connecting the AI tool looks roughly like this on the client side, with no connection string in sight:

{
  "mcpServers": {
    "company-db": {
      "url": "https://your-broker.example.com/mcp",
      "transport": "http",
      "auth": "oauth"
    }
  }
}

The teammate signs in through OAuth. No password gets pasted into a prompt, no secret ends up in a chat log, and access can be revoked centrally the moment someone leaves. That single property — credentials live in one place, not scattered across every laptop — is most of the security win right there.

What "safe" actually means here

A broker in the middle only helps if it enforces real guardrails. There are four worth insisting on.

Read-only by design. The connection the broker uses should be a database role that can SELECT and nothing else. In Postgres:

CREATE ROLE ai_readonly LOGIN PASSWORD '...';
GRANT CONNECT ON DATABASE app TO ai_readonly;
GRANT USAGE ON SCHEMA public TO ai_readonly;
GRANT SELECT ON ALL TABLES IN SCHEMA public TO ai_readonly;

-- new tables should inherit the same restriction
ALTER DEFAULT PRIVILEGES IN SCHEMA public
    GRANT SELECT ON TABLES TO ai_readonly;

Now if the assistant ever generates a write — because a prompt was sloppy or someone got clever — the database itself rejects it:

-- assistant attempts this on the read-only connection:
DELETE FROM users WHERE last_login < now() - interval '1 year';
-- ERROR: permission denied for table users

The AI can explore all day and never modify a byte. That's not a promise you're trusting the model to keep; it's enforced one layer below it.

Row-level security scopes what each person sees. Read-only isn't the same as "everyone sees everything." If your support lead should only see EU orders, row-level security enforces that regardless of what SQL the AI writes:

ALTER TABLE orders ENABLE ROW LEVEL SECURITY;

CREATE POLICY team_sees_own_region ON orders
    FOR SELECT
    USING (region = current_setting('app.current_region'));

The broker sets app.current_region from the authenticated user's identity, so the same natural-language question returns different rows for different people, and nobody can widen their own scope by rephrasing the prompt.

Schema awareness keeps the SQL honest. The broker can share your schema — table names, columns, types — with the assistant. This is the difference between an AI that guesses at a customer_name column that doesn't exist and one that writes SELECT full_name FROM users because it can see the real shape of your data. Sharing schema (not credentials) is what turns text-to-SQL from a party trick into something you'd let a colleague rely on.

Everything is logged. Because every query flows through one gateway, you get a single audit trail: who asked, what SQL ran, when. Compare that to auditing six SQL editors on six laptops — which is to say, not auditing anything.

What it looks like in practice

Once this is wired up, a teammate who has never written SQL just asks. Here are realistic questions and the SQL a schema-aware assistant produces behind the scenes:

What they type	What runs (they never see this)
"How many trials converted last week?"	`SELECT count(*) FROM subscriptions WHERE plan <> 'trial' AND status = 'active' AND converted_at >= now() - interval '7 days';`
"Top 5 accounts by revenue this quarter"	`SELECT a.name, sum(o.amount) AS revenue FROM orders o JOIN accounts a ON a.id = o.account_id WHERE o.created_at >= date_trunc('quarter', now()) GROUP BY a.name ORDER BY revenue DESC LIMIT 5;`
"Which features did churned users touch least?"	`SELECT e.feature, count() FROM events e JOIN users u ON u.id = e.user_id WHERE u.churned = true GROUP BY e.feature ORDER BY count() ASC;`

A slightly meatier one — weekly signups versus conversions — shows the assistant handling a join and a couple of filtered aggregates the person would never want to write by hand:

SELECT
    date_trunc('week', u.created_at)               AS signup_week,
    count(*)                                        AS signups,
    count(*) FILTER (WHERE s.status = 'active'
                     AND s.plan <> 'trial')         AS converted
FROM users u
LEFT JOIN subscriptions s ON s.user_id = u.id
WHERE u.created_at >= now() - interval '8 weeks'
GROUP BY 1
ORDER BY 1;

The teammate gets a table back. You never got pinged. And nobody touched a credential.

Common mistakes to avoid

Pointing the broker at your primary. Even read-only queries consume CPU and can cause lock contention. Point the read-only role at a replica so a curious sales rep's 10-million-row scan can't slow down checkout.

Exposing the entire schema on day one. Start restrictive. Grant SELECT on the ten tables people actually ask about, not all forty including password_resets and internal_audit. Expand based on real questions.

Skipping the semantic layer. If "active user" means three different things in three tables, the AI will pick one and sound confident. Define your core metrics once — in views or a semantic layer — so everyone's "active users" resolves to the same SQL.

Trusting output without spot-checks. AI-generated SQL is usually right and occasionally, fluently wrong. For any number that drives a decision, have someone technical eyeball the generated query the first few times a new question type shows up. The read-only boundary protects your data; it doesn't guarantee the logic.

Assuming read-only means private. Read-only still returns whatever the role can see. If people shouldn't see salaries or other customers' data, that's row-level and column-level security's job — not something "read-only" covers on its own.

If you'd rather not stand up and maintain the broker yourself, managed MCP servers implement these patterns for you — Draxlr, for instance, runs one that connects over OAuth and is read-only (SELECT only), so the AI can list schema, run queries, and build dashboards without ever holding your credentials. The point stands whichever route you take: the credentials, the read/write boundary, and the scoping belong to you, not to the model.

Key takeaways

The goal isn't to teach your whole company SQL, and it definitely isn't to hand out database passwords. It's to let people ask questions in the language they already speak while the guardrails stay yours. Put a broker between the AI and the database so the model never sees credentials. Make the connection read-only at the database level, not by asking nicely. Scope rows per person with RLS. Share schema so the SQL is accurate. Log everything through one gateway. Do that, and the "quick one — can you pull this?" messages mostly stop, without you trading away a single ounce of control.

Your turn

How does your team handle data requests today — ticket queue, shared read-only login, a BI tool, or something AI-driven? If you've let non-technical teammates query your database, what guardrail turned out to matter most, and what surprised you? Drop it in the comments.

Let Your SaaS Customers Ask Their Data Questions — Without Handing Over the Keys

Vivek Kumar — Mon, 31 Aug 2026 04:10:28 +0000

Every SaaS product eventually collects a data request it didn't plan for. A customer emails: "Can you tell me how many active seats we used each month last quarter?" Your dashboard shows totals, not that exact cut. So an engineer writes a one-off query, pastes the result into a reply, and moves on — until the next customer asks something slightly different.

Dashboards can only anticipate so many questions. The dream is to let customers ask in plain English — "what was our churn last month?", "which of my projects has the most open tickets?" — and get an answer straight from the data you already store on their behalf. The scary part is obvious: you'd be pointing an AI at a shared, multi-tenant production database where one wrong WHERE clause leaks Customer A's numbers to Customer B.

The good news is that this is a solved problem, and it doesn't require trusting the AI to get filtering right. It requires two layers that have nothing to do with the AI at all: row-level security in the database, and a broker (typically an MCP server) that scopes every connection to one tenant. Let the database enforce isolation and let the broker hold the credentials. The AI just writes SQL against a door that only opens onto one customer's data.

The setup: one database, many customers

Most B2B SaaS apps are multi-tenant — every customer's rows live in the same tables, tagged with a tenant_id (or account_id, org_id, whatever you call it).

CREATE TABLE subscriptions (
  id           bigint PRIMARY KEY,
  tenant_id    uuid NOT NULL,
  plan         text NOT NULL,
  seats        int  NOT NULL,
  status       text NOT NULL,       -- 'active', 'canceled', 'trialing'
  created_at   timestamptz NOT NULL
);

The whole game is making sure that no query — whether written by your app, an engineer, or an AI — can ever return rows where tenant_id doesn't match the customer who's asking. Doing that in application code means remembering to add AND tenant_id = $1 to every single query, forever. Miss it once and you have a data breach. So push the rule down into the database instead.

Layer 1: row-level security does the isolation

PostgreSQL's row-level security (RLS) lets you attach a policy to a table so the database itself filters rows based on a runtime value — no matter what query arrives.

-- Turn on RLS for the table
ALTER TABLE subscriptions ENABLE ROW LEVEL SECURITY;

-- Only return rows for the tenant in the current session context
CREATE POLICY tenant_isolation ON subscriptions
  USING (tenant_id = current_setting('app.current_tenant')::uuid);

Now the tenant identity travels with the connection, not the query. You set it once per request:

-- Scope the value to THIS transaction only (critical for pooled connections)
SET LOCAL app.current_tenant = '8f3a...c2';

SELECT count(*) FROM subscriptions WHERE status = 'active';
-- Returns only tenant 8f3a...c2's active subscriptions, always.

Notice the query has no tenant_id filter in it at all. The database added it. Even a careless SELECT * FROM subscriptions returns only the current tenant's rows. As one Crunchy Data write-up puts it, there's "zero chance of forgetting a tenant filter" because Postgres won't allow a cross-tenant query in the first place.

Two details that turn a demo into something production-safe:

SET LOCAL, not SET. Connection poolers reuse physical connections across requests. SET LOCAL binds the value to the current transaction so it can't leak into the next customer's request on the same connection. SET persists for the whole session and will eventually hand one tenant another tenant's context.

Index the tenant column first. RLS effectively prepends tenant_id = ? to every scan. Without tenant_id as the leading column of your indexes, those scans get dramatically slower — often two orders of magnitude on large tables.

CREATE INDEX idx_subs_tenant_status
  ON subscriptions (tenant_id, status);

Layer 2: a broker gives customers a door — not your credentials

RLS handles isolation. But how does a customer's AI assistant actually reach the database? You are not going to email them a connection string. This is where a broker sits in the middle — increasingly, one that speaks the Model Context Protocol (MCP), an open standard for connecting AI assistants to tools and data.

The pattern looks like this. Each customer authenticates to the broker over OAuth and gets a token that encodes their tenant_id. When their AI client wants to run a query, it sends the request to the broker, which:

Validates the customer's OAuth token and extracts the tenant_id from it.
Opens a database connection using your credentials (the AI never sees them).
Runs SET LOCAL app.current_tenant = <tenant_id from the token>.
Executes the AI's SQL — now automatically filtered by RLS — and returns just the rows.

The properties that fall out of this are exactly what you want for a customer-facing feature:

Concern	How the two layers handle it
Credential exposure	The broker holds the DB connection. The AI model and client never see raw credentials, tokens, or secrets.
Cross-tenant leaks	RLS filters every query by `tenant_id` at the database level, even if the AI writes a bad query.
Accidental writes	Give the broker a read-only role — SELECT only, so an AI can explore but never modify data.
Revoking access	OAuth tokens are centrally revocable; no long-lived secret lives in a prompt, chat log, or config file.
Auditability	The broker logs the natural-language question, the generated SQL, and the result for every request.

You can build this broker yourself, or use a managed MCP server. Draxlr, for example, offers one that connects over OAuth and is read-only by design — one implementation of the pattern above — but the architecture matters more than any product: RLS in the database, a credential-holding broker in front of it.

Putting it together: a request's journey

Say a customer opens their AI assistant and types:

"How many active seats did we have at the end of last month?"

The AI, given only your schema (not your data), writes:

SELECT sum(seats) AS active_seats
FROM subscriptions
WHERE status = 'active';

The broker sets app.current_tenant from the customer's token and runs it. RLS scopes the sum to that tenant. The result comes back:

active_seats
142

The customer sees 142 — their number, and only their number. They never learned SQL, never got a database login, and couldn't have reached another tenant's row if they'd tried. The same door works whether they ask from Claude, Cursor, ChatGPT, or an in-app chat box, because the isolation lives in the database, not the client.

Common mistakes and gotchas

Table owners bypass RLS by default. In Postgres, the role that owns a table isn't subject to its own RLS policies unless you force it. If your broker connects as the table owner, isolation silently does nothing. Add:

ALTER TABLE subscriptions FORCE ROW LEVEL SECURITY;

Superuser and BYPASSRLS roles ignore policies entirely. Never let the broker connect as a superuser or a role with BYPASSRLS. Give it a dedicated, low-privilege, read-only role.

Forgetting policies on new tables. RLS is per-table and off by default. Add a new table, forget the policy, and it's wide open to every tenant. Make "enable RLS + add tenant policy" part of your migration checklist, or write a test that fails when a tenant-scoped table lacks a policy.

SET instead of SET LOCAL on pooled connections. Worth repeating because it's the subtlest leak: a persistent SET outlives the request and the next tenant inherits it. Always scope to the transaction.

Trusting the AI to add the filter. The AI should never be your isolation boundary. If your only protection is "the prompt tells it to filter by tenant," you have no protection. RLS is what makes the AI's mistakes harmless.

No composite index on tenant_id. Correct but slow is still a bug when a customer is waiting on an answer. Lead your indexes with the tenant column.

Key takeaways

Letting customers ask their data questions in plain English is no longer exotic — but the safety comes from architecture, not from trusting the model. Enforce isolation in the database with row-level security so no query, human or AI, can cross tenants. Put a broker in front that holds your credentials, scopes each connection to one tenant via OAuth, and stays read-only. Then the AI is just a convenient way to write SELECT statements against a door that only ever opens onto one customer's data. Index your tenant column, force RLS on owned tables, keep the broker's role low-privilege, and audit every generated query.

Do that, and "can you pull this number for us?" stops being a support ticket and starts being something your customers answer themselves — in seconds, safely.

Your turn

Are you exposing data back to your customers yet — through dashboards, an API, or natural language? If you've built multi-tenant RLS in production, I'd love to hear what bit you: the pooling gotcha, the owner-bypass surprise, or something else entirely. Drop it in the comments.

How to Actually Measure Whether Your Text-to-SQL Is Any Good

Vivek Kumar — Fri, 28 Aug 2026 16:31:44 +0000

You wired an LLM up to your database. You asked it "how many active users signed up last month?", it wrote a tidy SELECT, the number looked plausible, and everyone in the demo nodded. Ship it.

Then a founder asks the same question a slightly different way, gets a number that's off by 20%, and now nobody trusts the feature. The uncomfortable truth about text-to-SQL is that a query that runs tells you almost nothing about whether it's right. "No error" and "correct answer" are two completely different things, and the gap between them is where trust quietly dies.

If you're building a natural-language query feature — an internal analytics box, a customer-facing "ask your data" panel, an AI assistant hooked to production — you need a way to measure accuracy that goes beyond "it looked fine when I tried it." This post walks through how text-to-SQL is actually evaluated, the sharp edges in those metrics, what the famous benchmarks do and don't prove, and how to build a lightweight eval suite for your own schema.

"It ran" is the weakest possible signal

Consider a question against a typical SaaS schema:

"How much revenue did we make from paid plans in July?"

Here are two queries an AI might produce:

-- Query A
SELECT SUM(amount)
FROM invoices
WHERE status = 'paid'
  AND created_at >= '2026-07-01'
  AND created_at <  '2026-08-01';

-- Query B
SELECT SUM(amount)
FROM invoices
WHERE created_at >= '2026-07-01'
  AND created_at <  '2026-08-01';

Both run. Both return a single number. On a test database where every July invoice happens to be paid, both return the same number. Query B is wrong — it silently includes refunded and failed invoices — but nothing about the execution surfaces that. This is the core problem: to evaluate text-to-SQL you have to compare against a notion of correct, not just runnable.

The three ways people measure accuracy

There are three broad approaches, and they trade off strictness against fairness.

Metric	How it works	Weakness
Exact-set match (ESM)	Compares the SQL text/clauses against a reference query, component by component	Punishes correct queries that are written differently
Execution accuracy (EX)	Runs both queries and compares the result sets	Two different queries can return the same rows by accident
Semantic equivalence	Judges whether two queries mean the same thing (often via an LLM or query analysis)	Harder to automate, can itself be wrong

Exact-set match is the oldest and the most brittle. Reference SQL says WHERE status = 'paid'; the model writes WHERE status IN ('paid'). Identical meaning, "wrong" by string comparison. ESM's rigid matching overlooks semantically correct but stylistically different queries, so it systematically under-counts good answers.

Execution accuracy fixed the obvious flaw: instead of comparing text, run both queries and compare the results. It has become the dominant metric in modern benchmarks precisely because it treats syntactically distinct but semantically equivalent queries as equal. If two queries produce the same rows, who cares how they're written?

The catch is that execution equality does not imply semantic equivalence — which brings us to the trap.

The trap: execution accuracy lies on small test data

Go back to Query A and Query B above. On a sparse test database, they agree. That's a false positive: a wrong query scored as correct because the test data wasn't diverse enough to expose the difference. An incomplete WHERE clause slips through whenever every row in the test set happens to satisfy the missing condition.

This isn't a rare edge case. Studies of execution-based evaluation have measured false-positive rates around 11% — roughly one in nine "correct" queries is actually wrong and just got lucky on the test data. If your eval database is a handful of tidy demo rows, that number is worse, not better.

The fix that the research community landed on is elegant: test-suite evaluation. Instead of one small database, you evaluate the query against several databases specifically constructed so that a wrong query is very likely to diverge from the right one on at least one of them. The idea (from Semantic Evaluation for Text-to-SQL with Distilled Test Suites) is to distill a compact set of databases that achieves high code coverage of the reference query, giving a tight approximation of true semantic accuracy without needing to prove equivalence formally.

You can apply the spirit of this cheaply. Seed your test data with rows that would break a lazy query:

-- Adversarial seed rows for the revenue example
INSERT INTO invoices (amount, status, created_at) VALUES
  (100, 'paid',     '2026-07-15'),  -- should count
  (100, 'refunded', '2026-07-16'),  -- must NOT count
  (100, 'failed',   '2026-07-17');  -- must NOT count

Now Query A returns 100 and Query B returns 300. The bug is visible. A good eval dataset is one where being sloppy costs you.

What Spider and BIRD actually tell you

Two public benchmarks dominate the conversation, and it's worth knowing what each is really measuring before you quote a leaderboard number to your team.

Spider contains 10,181 questions over 5,693 unique queries across 200 databases spanning 138 domains. Its whole point is cross-domain generalization: the test databases are unseen at training time, so a high score means the model can handle a schema it has never met. That maps well to "will this work on my customers' databases," which are all different.

BIRD is the more real-world sibling — over 12,000 queries across 95 databases in 37 professional domains — and it adds two things Spider mostly ignores. First, it rewards using external knowledge (the messy business context real questions require). Second, it measures efficiency, not just correctness, via a Valid Efficiency Score: a query that returns the right rows but does a full table scan where an indexed lookup would do scores lower than a fast, correct one.

That efficiency dimension matters more than people expect. Two queries can both be "correct" and differ wildly in cost:

-- Correct but expensive: function on the column kills the index
SELECT * FROM events
WHERE DATE(created_at) = '2026-07-15';

-- Correct and cheap: sargable range keeps the index usable
SELECT * FROM events
WHERE created_at >= '2026-07-15'
  AND created_at <  '2026-07-16';

Both return the same rows. In a customer-facing feature, only one of them is acceptable at scale.

The thing to remember: a leaderboard score is measured on someone else's schemas and questions. It's a useful signal for picking a model, but it is not a substitute for testing against your database and your users' phrasing.

Building your own eval suite

The good news is you don't need a research pipeline. A useful eval is just a set of question → reference-query pairs (call them golden queries) plus a harness that runs the model's output and compares result sets.

Start by collecting real questions — from support tickets, from analysts' saved queries, from whatever people actually ask. For each, write the SQL you know is correct:

-- golden_queries.yaml (conceptually)
-- id: revenue_paid_july
--   question: "How much revenue from paid plans in July?"
--   sql: |
--     SELECT SUM(amount) FROM invoices
--     WHERE status = 'paid'
--       AND created_at >= '2026-07-01'
--       AND created_at < '2026-08-01';

Then the harness, in pseudocode:

passed = 0
for case in golden_queries:
    predicted_sql = model.generate(case.question, schema)
    try:
        got = db.run(predicted_sql)          # your candidate
    except SQLError:
        record(case, "invalid_sql"); continue
    want = db.run(case.sql)                   # your golden answer
    if result_sets_equal(got, want):         # order-insensitive compare
        passed += 1
    else:
        record(case, "wrong_result", got, want)

print(f"Execution accuracy: {passed / len(golden_queries):.1%}")

A few details make this far more honest than a naive version. Compare result sets, not row order, unless the question asked for a specific ordering — otherwise a correct GROUP BY fails just because rows came back shuffled. Run every case against the adversarial seed data from earlier so false positives can't hide. And bucket your failures — invalid SQL, wrong table, missing filter, wrong aggregation — because "62% accurate" is far less useful than "most failures are a missing tenant filter."

If you'd rather not run the model against production directly during all this, a read-only gateway helps: managed MCP servers like Draxlr expose a database to an AI client over a read-only (SELECT-only) connection, which is a sane place to point an eval harness so a buggy generated query can't do anything but return wrong rows.

Common mistakes and gotchas

The most common one is treating a green demo as evidence. A feature that answers five questions correctly on stage can be 60% accurate across the long tail of real phrasings, and you'll never know until you measure the tail.

The second is a tiny, tidy eval database. Sparse data is exactly what manufactures false positives; if every row satisfies the filters your model forgets, your eval will happily bless broken SQL. Diverse, adversarial rows are a feature.

The third is scoring only correctness and ignoring cost. A query that's right but scans a 200-million-row table is a production incident waiting to happen — track query time or plan cost alongside accuracy.

Fourth: overfitting to the questions you happened to write. If your golden set is 30 questions and you tune prompts until all 30 pass, you've built a model that's great at those 30 questions — keep a held-out set you don't tune against. And version the eval itself: when you change the prompt, schema description, or model, the number that matters is the delta from a saved baseline, not the absolute score.

Key takeaways

Evaluating text-to-SQL is about measuring correctness, not runnability. Exact-match under-counts good queries; execution accuracy is the practical standard but produces false positives on thin test data; test-suite-style adversarial data is how you close that gap. Public benchmarks like Spider (cross-domain generalization) and BIRD (real-world knowledge plus efficiency) are great for choosing a model but never a substitute for testing on your own schema. Build a golden-query set from real questions, run it against deliberately diverse data, compare result sets rather than SQL text, track cost as well as correctness, and re-run it on every change. That's the difference between "it worked in the demo" and "we know it's 94% accurate and we watch that number."

How are you measuring your text-to-SQL feature today — golden queries, LLM-as-judge, eyeballing, or nothing yet? Drop your setup (and your favorite false-positive horror story) in the comments.

From Plain English to a Live Dashboard: Automating Reporting with MCP

Vivek Kumar — Thu, 27 Aug 2026 04:00:23 +0000

Someone in your Slack asks, "How many people signed up last week?" You open a SQL client, write a query you've written a dozen times before, run it, copy the number, and paste it back. A week later, the same question lands in a different channel. Same query, same copy-paste, same five minutes gone.

Multiply that by every "quick number" your team asks for — signups, active users, MRR, refunds, top accounts — and reporting quietly becomes a part-time job that nobody signed up for. The knowledge lives in your head and in a folder of .sql files nobody else can find.

The Model Context Protocol (MCP) offers a cleaner path. It's an open standard for connecting AI assistants to external systems — including your database — through a consistent, permissioned interface. Instead of the AI guessing at a connection or you pasting credentials into a chat box, an MCP server sits in between and exposes a small set of safe operations: read the schema, run a read-only query, save it, add it to a dashboard. This article walks through that whole loop — plain-English question to living report — and where it can go wrong.

What actually sits between a question and a chart

MCP servers expose three kinds of capabilities, and it helps to know which is doing what (Speakeasy has a clear breakdown):

Capability	What it is	In a database context
Resources	Read-only data the model can pull in for context	Your table and column definitions — the schema
Tools	Actions the model can invoke	Run a query, save a query, add it to a dashboard
Prompts	Reusable templates for common workflows	"Build a weekly signups report" as a repeatable recipe

The important part: the AI never touches your database directly. It asks the server for the schema, drafts SQL, and asks the server to run it. Because the query tool is read-only by design, a DELETE or DROP the model dreams up simply gets rejected. The AI explores freely; your data stays intact.

This is also why schema-awareness matters so much. When the model can read your actual tables and columns as a resource, it stops inventing plausible-but-wrong names like user.signup_date when your column is really users.created_at. Grounding the model in the real schema is the single biggest thing that keeps generated SQL honest.

Step 1 — Ask in plain English

Here's the shape of an interaction. Assume a typical SaaS database with users, subscriptions, and events tables. You type:

"How many users signed up in the last 7 days, grouped by day?"

The assistant fetches the schema, sees users(id, email, created_at, plan), and produces:

SELECT
  DATE(created_at) AS signup_day,
  COUNT(*)         AS signups
FROM users
WHERE created_at >= NOW() - INTERVAL '7 days'
GROUP BY DATE(created_at)
ORDER BY signup_day;

You didn't specify the column name, the date function, or the grouping. The model inferred them from the schema. That's the difference between an AI that's guessing and one that's reading.

Step 2 — Verify before you trust

The generated SQL is a draft, not gospel. The advantage of the read-only setup is that running it to check is completely safe. The result comes back:

signup_day	signups
2026-08-20	41
2026-08-21	38
2026-08-22	22
2026-08-23	19
2026-08-24	54
2026-08-25	47
2026-08-26	29

Glance at it. Do the weekends dip the way they usually do? Is the total in the right ballpark? A thirty-second sanity check here saves you from confidently reporting a number that's off because "signup" quietly meant "row created, including invited-but-not-activated users." More on that trap below.

Step 3 — Save the query so it's never rewritten

This is the step that breaks the treadmill. Once the query is correct, save it with a name and description through the server's save tool:

"Save that as Weekly Signups by Day."

Now it's a named, reusable report. Next week nobody rewrites it — they ask to run Weekly Signups by Day and get fresh numbers against live data. You've turned a throwaway query into an asset the whole team can call by name. This is also where a prompt template earns its keep: "produce a signups report for the last N days" becomes a recipe you invoke, not SQL you retype.

Step 4 — Pin it to a dashboard

The last move is to make the report ambient so people stop asking at all. Add the saved query as a dashboard tile:

"Add Weekly Signups by Day to the Growth dashboard as a bar chart."

Chain a few of these together and you've assembled a real reporting surface — signups, activation rate, MRR, churn — entirely from plain-English requests, each one a verified, saved, named query underneath. Managed MCP servers implement exactly this loop; Draxlr's is one example that connects over OAuth and is read-only (SELECT only), with tools to list databases, fetch schema, run and save queries, and build dashboards. The pattern is the same regardless of which server you use.

A fuller example: three reports, one conversation

Say you want a small revenue snapshot. You ask for three things in a row.

New MRR from subscriptions started this month:

SELECT SUM(monthly_amount) AS new_mrr
FROM subscriptions
WHERE status = 'active'
  AND started_at >= DATE_TRUNC('month', NOW());

Top 5 plans by active subscribers:

SELECT plan, COUNT(*) AS subscribers
FROM subscriptions
WHERE status = 'active'
GROUP BY plan
ORDER BY subscribers DESC
LIMIT 5;

Refunds issued in the last 30 days:

SELECT COUNT(*) AS refunds, SUM(amount) AS refunded_total
FROM events
WHERE type = 'refund'
  AND created_at >= NOW() - INTERVAL '30 days';

Save all three, drop them on a "Revenue Health" dashboard, and you've built in five minutes what used to be a recurring manual chore. The AI wrote the SQL; you supplied the judgment about what's worth measuring.

Common mistakes and gotchas

Treating the first answer as the final answer. LLMs can produce SQL that runs cleanly but answers a subtly different question than you asked. Always read the query and eyeball the result before you save it or share the number.

Fuzzy metric definitions. The word "active" can mean logged-in-this-week, has-a-paid-plan, or has-any-event-ever. If your team hasn't agreed on definitions, the AI will pick one for you — and it may not be the one your board is using. Industry write-ups on AI reporting consistently flag ungoverned metric definitions as the top source of "hallucinated" analytics: the number is real, but the definition behind it is wrong. Where you can, point the model at a governed view or a semantic layer rather than raw tables, so "revenue" means one thing everywhere.

Granting more access than you need. The whole security benefit collapses if you connect the AI with a read-write account. Use a read-only role, and prefer a setup where the server enforces SELECT-only regardless of the credential. The AI should be able to read everything it's allowed to and change nothing.

No audit trail. If you can't later see which queries the AI ran, you can't debug a wrong number or satisfy a compliance question. Favor an approach where access is centralized and queries are logged, not scattered across personal database clients.

Skipping the schema step. If the model isn't given the real schema, it falls back to guessing table and column names. That's where hallucinated columns come from. Schema-first, always.

Key takeaways

The reporting treadmill isn't a SQL problem — it's a reuse problem. You already know how to write the query; the pain is writing it again and again and keeping it somewhere findable. MCP addresses that by putting a safe, schema-aware, read-only interface between the AI and your database, then letting you promote good queries into saved reports and dashboard tiles.

The loop is small and repeatable: ask in plain English, verify the generated SQL against live data, save the query with a clear name, and pin it to a dashboard. Keep humans in the verification seat, nail down your metric definitions, and never hand the AI more than read access. Do that and "quick number" requests stop interrupting your day — they answer themselves.

Your turn

How does your team handle recurring reporting today — a folder of saved queries, a BI tool, or a lot of copy-paste? If you've wired an AI assistant to your database through MCP, I'd love to hear what worked and what surprised you. Drop a comment with the setup you're using.

Sources: MCP core concepts — Speakeasy, What is MCP? A Data Person's Guide to Agentic Analytics — MotherDuck, AI Report Generation guide — Improvado.

Who Just Queried Prod? Auditing and Controlling AI Database Access Across a Team

Vivek Kumar — Wed, 26 Aug 2026 08:43:01 +0000

Six months ago, exactly one person on your team could query the production database from an AI tool. Today it's everyone. Someone pasted a connection string into an AI assistant, it worked beautifully, and the pattern spread. Now three engineers, a product manager, and a support lead all ask a chatbot questions that quietly turn into SELECT statements against live data.

That's genuinely useful. It's also a governance blind spot. If someone asks "why did this customer's numbers look weird last Tuesday," can you answer who — or what — ran the query that touched their records? With scattered connection strings and shared credentials, the honest answer is usually "no idea."

This article is about closing that gap: how to give a team AI-powered database access while keeping it identifiable, scoped, and auditable. The goal isn't to slow anyone down. It's to make sure that when access grows from one person to twenty, you still know what's happening.

The failure mode: shared secrets and no paper trail

The default way people connect an AI tool to a database is to hand it a connection string:

postgresql://app_user:s3cr3t@db.internal:5432/production

Do that across a team and you inherit four problems at once. Everyone shares one database identity, so every query looks identical in the logs. The secret lives in prompts, chat histories, config files, and screenshots. Rotating it means chasing down every place it was pasted. And the database has no idea whether a human or a model issued a given statement.

The database's own audit log doesn't save you here, because from its point of view there's a single user, app_user, doing everything. You've lost the two facts that matter most for governance: which person the access belongs to, and whether their tool was allowed to do what it did.

What good looks like: a broker between the AI and the database

The pattern that fixes this is putting a broker — often an MCP (Model Context Protocol) server — between AI clients and the database. Instead of each tool holding raw credentials, they connect to one governed gateway that holds the connection and enforces the rules.

Model Context Protocol is an open standard for connecting AI assistants to external systems through a consistent interface. For databases, an MCP server exposes a small set of operations (list tables, fetch schema, run a read-only query) and becomes the single place where identity, permissions, and logging live.

Routing everyone through one gateway gives you five properties that scattered connection strings can't:

Property	What it means in practice
Per-person identity	Each teammate authenticates as themselves (usually via OAuth), so every query is attributable to a human.
Least privilege	The broker can be read-only by design — `SELECT` passes, `UPDATE`/`DROP` get rejected before they reach the DB.
Central revocation	Off-boarding is one toggle. No long-lived secret to hunt down across chat logs.
Audit trail	One chokepoint logs who ran what, against which database, and when.
No shadow access	The database isn't network-exposed to every laptop; one gateway is.

Managed MCP servers like Draxlr's implement this shape — connected over OAuth, read-only, with the credentials held server-side — but the pattern matters more than any product. You can build the same thing in-house. What follows works either way.

Building the audit trail

Whatever broker you use, the non-negotiable is a log of every query with enough context to answer "who did this and why." At minimum, capture the acting identity, the SQL, the target database, a timestamp, and a correlation ID that ties a query back to the conversation that triggered it.

A simple audit table looks like this:

CREATE TABLE ai_query_audit (
    id            BIGSERIAL PRIMARY KEY,
    actor_email   TEXT        NOT NULL,   -- the human, via OAuth
    ai_client     TEXT        NOT NULL,   -- claude, cursor, chatgpt...
    database_name TEXT        NOT NULL,
    sql_text      TEXT        NOT NULL,
    row_count     INTEGER,
    status        TEXT        NOT NULL,   -- 'allowed' | 'rejected'
    reason        TEXT,                   -- why rejected, if it was
    correlation_id UUID       NOT NULL,
    created_at    TIMESTAMPTZ NOT NULL DEFAULT now()
);

CREATE INDEX idx_audit_actor_time ON ai_query_audit (actor_email, created_at DESC);

The broker writes one row per attempt — including the rejected ones, which are often the interesting ones. Now the earlier question has an answer:

-- Who touched customer 4821's data in the last week?
SELECT actor_email, ai_client, created_at, status, sql_text
FROM   ai_query_audit
WHERE  sql_text ILIKE '%customer_id = 4821%'
  AND  created_at >= now() - INTERVAL '7 days'
ORDER  BY created_at DESC;

    actor_email      | ai_client | created_at          | status  | sql_text
---------------------+-----------+---------------------+---------+------------------------------------------
 dana@acme.com       | claude    | 2026-08-19 14:02:11 | allowed | SELECT * FROM orders WHERE customer_id...
 support@acme.com    | chatgpt   | 2026-08-18 09:41:55 | allowed | SELECT status, total FROM orders WHERE...

You can build governance dashboards straight off this table — top queriers, rejected-write attempts over time, which tables get hit most, unusual after-hours activity. That's the observability layer security teams actually ask for.

Scoping access per team

Not everyone needs the same reach. Least privilege for AI access should be at least as sharp as the RBAC you give employees — arguably sharper, because a model will cheerfully try anything you let it. Two levers do most of the work: which databases an identity can see, and whether it's read-only.

A policy config for a broker might look like this:

roles:
  support:
    databases: [production_readonly]
    mode: read-only
    row_filter: "region = :user_region"   # row-level scoping
  analytics:
    databases: [production_readonly, events_warehouse]
    mode: read-only
  engineering:
    databases: [production_readonly, staging]
    mode: read-only

Notice there's no write mode anywhere. For an AI exploration workflow, that's usually correct: the model can read the world and help you understand it, but it cannot modify a single row. If a prompt ever produces DELETE FROM subscriptions, the broker rejects it and logs the attempt rather than executing it.

Row-level filters are what make this safe for customer-facing use, too. Bind a filter like tenant_id = :current_tenant at the broker and a support agent asking "show me recent orders" only ever sees their own region's data — no matter how the model phrases the SQL.

What the workflow actually feels like

Governance shouldn't be visible to the person doing the work. From their side it's still plain English:

User: How many trial accounts converted to paid last month, by plan?

The AI client fetches the schema through the broker (so it uses real column names instead of hallucinating them), writes SQL, and the broker runs it read-only under that user's identity:

SELECT s.plan,
       COUNT(*) AS conversions
FROM   subscriptions s
JOIN   users u ON u.id = s.user_id
WHERE  u.trial_started_at >= date_trunc('month', now()) - INTERVAL '1 month'
  AND  u.trial_started_at <  date_trunc('month', now())
  AND  s.status = 'active'
GROUP  BY s.plan
ORDER  BY conversions DESC;

The user gets their answer. Meanwhile, one row lands in ai_query_audit tagged with their email, the client they used, and a correlation ID. Nobody typed a credential; nobody can later ask "wait, who ran that?" and come up empty.

Common mistakes and gotchas

Logging the query but not the identity. A log full of anonymous SQL is barely better than none. The acting human's identity is the column that turns a log into an audit trail — capture it first.

Treating a shared service account as "the AI user." If every teammate's AI traffic authenticates as one account, you've rebuilt the connection-string problem with extra steps. One identity per person (or per agent), tied to your IAM.

Long-lived tokens in config files. Permanent keys can't be revoked cleanly and tend to leak. Prefer OAuth grants that expire and can be killed centrally the moment someone leaves.

Read-only in name only. "We told people not to run writes" is not a control. Enforce it at the broker so a rejected UPDATE is a logged event, not a trust exercise.

Ignoring shadow MCP. The moment a governed path exists, make it the only path. If people can still point tools straight at the database, your audit log has holes exactly where the risky queries are.

No retention or tamper-evidence on the log. An audit trail someone can quietly edit isn't much of an audit trail. Ship logs somewhere append-only, with a retention window that matches your compliance needs.

Key takeaways

Scattered connection strings give a team speed and take away accountability. You can keep the speed. Put a broker between AI clients and the database, make each person authenticate as themselves, keep access read-only and scoped by role, and log every attempt — allowed and rejected — with the identity attached. Whether you build that broker yourself or adopt a managed MCP server, the properties are the same: attributable, revocable, least-privilege, auditable.

The test is simple. If someone asks "who queried prod at 2 a.m. and why," you should be able to answer in one SELECT.

How does your team handle this today — shared credentials, a homegrown proxy, or a managed gateway? And what do you actually log per query? I'd love to hear what's working (and what's bitten you) in the comments.