An LLM that hallucinates isn't malfunctioning. It's doing exactly what it was built to do — and understanding why is the first step to stopping it.
"Hallucination" is the polite word for an LLM stating something false with total confidence. It's the single biggest blocker to using these models in serious systems. But it isn't a bug you can patch — it's a direct consequence of how the model works. Here's the honest mechanism, and the patterns that actually contain it.
Why it happens
A language model has one job: predict the next token given the previous ones. It's an extraordinarily good statistical model of what text looks like. What it is not is a database of facts or a model of truth. Nowhere in the architecture is there a notion of "is this true."
So when you ask a question, the model generates the most plausible continuation — the sequence of words that fits the pattern. When it actually knows the answer, plausible and true coincide. When it doesn't, it still produces something plausible, because producing plausible text is the only thing it can do. A fluent, confident, fabricated citation is not the model failing; it's the model succeeding at the wrong objective.
This is why "just tell it not to hallucinate" doesn't work. You can't prompt away the fundamental behavior. You have to change the system around the model.
The patterns that actually work
Every reliable approach comes down to the same principle: stop asking the model to recall, and start giving it something to work from — or something to check against.
1. Ground it with retrieval. Feed the model the relevant facts at query time (this is RAG) and instruct it to answer only from that context. It can't fabricate a number that isn't in front of it if it's forbidden from answering otherwise.
2. Constrain what it's allowed to produce. The strongest version: don't let the model state facts at all. Have it generate a query — SQL, an API call, a structured filter — and let a deterministic system produce the actual answer. The model decides what to ask; something trustworthy decides what's true. I build systems around this idea precisely because it moves the model out of the position where it can lie.
3. Demand citations. Require the model to point at the source for each claim. It both improves grounding and makes fabrication visible — an uncited claim becomes a red flag you can catch automatically.
4. Verify after generation. Treat the model's output as a draft. Check its claims against a source of truth before showing them to a user. In high-stakes domains, a cheap verification pass is worth more than a better prompt.
5. Lower the temperature. For factual tasks, high randomness is just more room to wander. Turning it down won't create truth, but it stops the model from creatively embellishing.
The mindset shift
The engineers who ship reliable LLM products stop thinking of the model as an answer machine and start thinking of it as a fluent reasoner that must be kept on a leash of real data. The intelligence is real; the trust has to be engineered around it.
A concrete example
The cleanest illustration is a finance tool, where a single invented number is a disqualifying failure. In "Making an LLM trustworthy over bank statements", I walk through a platform where the LLM never answers from its own knowledge at all — its only output is a SQL query against a strict schema, and the database returns the facts. The model literally cannot hallucinate a transaction, because it's never in a position to state one.
Hallucination isn't solved by a smarter model. It's solved by an architecture that never trusts the model with the one thing it was never built to guarantee. More on how I approach that at www.divyakush.com.
Divyakush Punjabi · Full-Stack & AI Engineer
Portfolio · GitHub · LinkedIn
Top comments (0)