I recently worked on a project. To better understand which method works better for us, it is very important to understand the nature of the database. The database I was working on consisted of 16 columns and 20,001 rows, all related to company addresses and a few other details regarding their statuses. I was tasked with fetching data according to user queries, but I had to ensure there was zero hallucination. The system also needed to support aggregate functions, such as calculating maximums and averages. Since the chatbot was developed as a Proof of Concept (POC), the database size was intentionally kept small.
Now that I am clear on the requirements and the nature of the dataset, I'll explain my approach and would like feedback on how I can improve it, as I am fairly new to the trade. I opted for a Text-to-SQL architecture because it provides higher deterministic accuracy than RAG and can perform average, max, and other operations; this is what we wanted to use on the database as it was a core functionality of the chatbot. Furthermore, since there will be a maximum of 100 concurrent users and the database supports horizontal scaling without the computational overhead of re-indexing vectors, I believe it is a more efficient approach—especially since the type of dataset I am working on has columns (like 'capital') that might be changed frequently.
Top comments (2)
This is a solid line of thinking, honestly — and you’re asking the right question for the kind of guarantees you want.
For a dataset like the one you described (small-ish, structured, well-defined schema, aggregates required, zero tolerance for hallucination), Text-to-SQL is the correct default, not RAG. You didn’t “miss” anything obvious there.
A few thoughts from my side, building on what you already reasoned through:
First, the zero-hallucination requirement basically disqualifies pure RAG. Once you allow a model to synthesize answers from retrieved text, you’re already accepting probabilistic interpretation. That’s fine for exploratory knowledge tasks, but not when the source of truth is a table and the output must be exact. SQL gives you a contract. Either the query runs and returns rows, or it doesn’t. That determinism matters more than people admit.
Second, aggregates are a big tell. The moment you need
AVG,MAX,GROUP BY, or even consistent filtering across changing columns, RAG starts feeling like you’re fighting the tool. You can simulate aggregates in RAG, but you’re essentially rebuilding a query engine in natural language. Letting the database do what databases are good at is the sane move.Third, your point about column volatility is important and often overlooked. If fields like
capitalchange frequently, vector re-indexing becomes operational friction very quickly. Text-to-SQL sidesteps that entirely — the schema stays stable, the data changes, and the system keeps working. That’s a huge win in real systems, especially under concurrency.Where I’d gently push you further isn’t “RAG vs Text-to-SQL” — it’s how much guard-railing you put around the Text-to-SQL layer.
Some practical things that tend to harden this approach:
If anything, this POC is a good example of why people sometimes overreach with RAG. RAG shines when the knowledge is unstructured, ambiguous, or narrative-heavy. Your problem isn’t that — it’s structured truth retrieval under constraints.
So yeah, from where I’m standing: your instinct was right. If you ever add RAG later, it should be adjacent — for explaining results, clarifying schemas, or handling fuzzy business questions — not as the core query mechanism.
Good thinking overall. You’re not “new to the trade” in the way this question is framed — this is already systems-level reasoning.
Truly an eye opener. Thanks for the clarification!