*AI can write SQL in seconds. So I gave it a harder test.
*
We asked AI 10 questions a real business user might ask a database — not simple SELECT statements, but questions involving joins, dates, revenue, customer behaviour, and business logic. Some queries were surprisingly good. Some looked completely correct but gave the wrong answer. And one of them taught me something I didn’t expect:
*The biggest problem with AI-generated SQL isn’t always the SQL. It’s whether the AI understands what the data actually means.
*
The database
To make the experiment realistic, let’s use a simple e-commerce database.
A small schema with plenty of room to go wrong.
At first, this looks simple. But even this small database creates plenty of opportunities for mistakes. Customers and orders are stored separately; products and orders are connected through order items; orders can have different statuses; product prices can change over time; and business terms such as “revenue,” “last month,” and “active customer” need definitions. That means the AI needs more than SQL syntax. It needs context.
**Question 1: Which customers generated the most revenue?
This sounds easy. The AI generated:
✓ Mostly correct
The query connects customers to orders, calculates revenue, groups the results, and returns the top 10. But there’s a small problem: what if two customers have the same name? A safer version would group by the customer ID as well. This looks correct at first glance but still deserves a review.
Lesson: AI can generate the structure quickly, but you still need to check the details.
Question 2: Which products generated the most revenue?
⚠ Looks right. But is it?
The query multiplies quantity by the current product price. But imagine a product was ₹500 last year and ₹700 today. If an old order contained five units, using today’s price would produce the wrong historical revenue. The SQL itself is valid; the business answer may not be. This is one of the biggest differences between syntactically correct SQL and correct data analysis.
Question 3: Which region had the highest sales last month?
⚠ Depends on what “last month” means
“Last month” could mean the previous 30 days, the previous calendar month, or the current month compared with the previous one — and those aren’t the same thing. If the business means “July 2026” and today is August 19, a rolling 30-day filter isn’t necessarily what they want. The AI generated a valid query, but the question itself was ambiguous.
Lesson: Sometimes the biggest problem isn’t the query. It’s the question.
Question 4: Which customers haven’t ordered in the last 90 days?
The AI initially generated:
✗ Wrong
This is a classic logical problem. A customer could have an order from six months ago and another order yesterday — and the query still returns them because it found an old order. But the actual question is: which customers have no orders in the last 90 days? A better approach:
Now we’re checking whether a recent order exists at all — the classic anti-join pattern.
Lesson: A query can look reasonable and still completely misunderstand the logic.
Question 5: What was our average order value by month?
✓ Technically correct
This one looks good — the monthly DATE_TRUNC grouping does what you’d expect. But again: which orders should count? If the database contains cancelled orders, should they be included? What about refunded orders? Test orders? If the company’s definition of revenue only includes completed orders, the query needs something like WHERE status = 'completed'. The AI may not know that business rule unless the context is available.
Question 6: Which products are selling less than last year?
Now things get more complicated. The AI needs to define the current period, define the comparison period, calculate product sales, match products between periods, and compare the results. A simplified approach:
⚠ Needs clarification
What does “last year” mean — rolling 12 months, previous calendar year, or the same period last year? A human needs to define that before the AI can reliably write the query. Ambiguous business language creates ambiguous SQL.
Question 7: Which customers have spent more than ₹1 lakh?
✓ Correct
This is the kind of question AI handles particularly well: a clear metric, a clear threshold, a straightforward relationship, a simple aggregation. The less ambiguity there is, the easier it is for AI to produce reliable SQL.
Question 8: Which products have never been purchased?
✓ Correct
A common anti-join pattern — looking for products with no matching record in order_items. AI handles these standard relational patterns quite well.
Question 9: Why did revenue drop last month?
This is where things get interesting. The AI can’t simply write one perfect query — because “why” isn’t a database column. To investigate, we need to break it down: did the number of orders decline? Did average order value decline? Did a particular region decline? Did a particular product decline? Did customer activity change? Now we have several smaller queries.
✗ No single SQL query can answer the whole question
The AI can help write the individual queries. But someone still needs to decide what to investigate. This is where business understanding becomes extremely important.
Question 10: What should we do to increase sales?
And this is the hardest question of all. A database can tell you which products sell the most, which customers spend the most, which regions are growing, which products are declining, and how order frequency is changing. But it cannot automatically tell you “here’s the perfect strategy for your business.” That’s a decision. Data can provide evidence; AI can help analyze it; but someone still needs to understand the business and decide what action makes sense.
✗ Not a pure SQL question
And that’s an important distinction: not every business question is a database query.
So, how did AI actually perform?
After looking at all ten questions, a pattern appeared.
AI shines on clear questions and stumbles on ambiguous ones.
So the answer isn’t “AI is bad at SQL,” and it’s not “AI can completely replace SQL skills.” The reality is more interesting.
The biggest problem wasn’t SQL
The biggest surprise was that most difficult problems weren’t about syntax. They were about context. Consider these two requests: “find revenue” versus “find completed-order revenue, excluding refunds and cancellations, using the amount actually paid by customers.” The second contains business logic. The database might have everything needed to answer it — but the AI needs to know those rules. That’s why AI-generated SQL isn’t just a language-model problem. It’s also a database-context problem.
What happens when AI has better database context?
This is where AI-powered database tools become interesting. Instead of asking a generic chatbot to guess what your database looks like, the AI can work closer to the actual database environment.
For example, DBx Studio is built around AI-assisted interaction with databases. The workflow becomes business question → database context → SQL generation → review → execution → result → follow-up — very different from business question → generic AI → guess the schema → generate SQL.
With context, the AI guesses less.
The more context the AI has, the less it needs to guess. You can learn more about how DBx approaches this through its documentation.
Would I trust AI-generated SQL without checking it?
After this experiment: no. But that doesn’t mean I wouldn’t use it. I’d use AI to generate SQL, explain unfamiliar queries, suggest joins, explore a new database, rewrite queries, find possible approaches, and help investigate a problem. But I’d still check the tables, the joins, the filters, the date ranges, the calculations, the business definitions, and the final result. The goal isn’t “AI writes everything for me.” It’s “AI helps me get from question to answer faster.”
The real lesson
After testing these ten questions, my takeaway isn’t that AI is amazing at SQL. And it isn’t that AI is terrible at SQL. It’s this:
AI is very good at translating clear questions into SQL. It becomes much less reliable when the question itself requires context, interpretation, or business judgment.
Because the future of working with data probably isn’t humans write SQL or AI writes SQL. It’s more likely: humans define the problem, AI helps navigate the database, and humans validate and interpret the result. That’s a much more interesting future.
Top comments (1)
Honestly, this is a really good way to test AI SQL. A query can look completely correct and still give the wrong answer because of the business logic. Would be interesting to see which queries failed and where the AI went wrong.