Introduction
When you ask cutting-edge AI chatbots like ChatGPT, Claude, or Gemini a question—no matter the topic—you usually get a confident answer back immediately.
It may seem reassuring at first, but have you ever found yourself wondering, “Is that really correct?”, or later discovered the answer was wrong after checking?
This phenomenon is called “AI hallucination.” It refers to outputs that look plausible but are in fact incorrect.
So why does AI, which is supposed to help people, refrain from saying “I don’t know” when something is uncertain and instead respond as if it’s certain? (It’s not that AI can never say “I don’t know,” but structurally they tend to avoid it.)
Digging into this question reveals the mechanisms and design philosophy of AI.
LLMs Are “Probabilistic Prediction Devices”
To start, there are multiple reasons why AI can’t say “I don’t know” and ends up answering assertively, but the key premise is that large language models (LLMs) like ChatGPT are not knowledge bases. In essence, they learn from massive amounts of text and act as “devices that predict the next likely token.” The training objective is to produce plausible continuations, and whether the content is factually correct is often not part of the objective function. That’s why they can generate natural, fluent prose while also casually producing plausible mistakes. In many cases, that’s the essence of hallucination.
The “Knowledgeable Friend” Analogy
A useful way to think about this is that AI is closer to a “knowledgeable friend” than to an encyclopedia (knowledge base).
Encyclopedia (knowledge base)
→ Information is curated, and if something isn’t covered it returns “no entry.” Also, because the recorded information tends to be reliable, when your question matches what’s documented, it’s easy to treat it as trustworthy.Knowledgeable friend (LLM)
→ They marshal what they know and will still try to answer with “It’s probably like this” even when they don’t actually know.
The friend is trying to help in good faith, but naturally misunderstandings and guesses get mixed in. AI behaves very similarly.
Why Is “Assertion” the Default?
So why doesn’t AI conservatively say “I don’t know,” and instead gravitates toward assertive answers? There are several reasons:
Data bias: The training corpus overwhelmingly contains declarative, assertive sentences.
Evaluation bias: With RLHF (human feedback), confident answers tend to be rated as “useful.”
KPI pressure: User satisfaction and conversation-continuation metrics often improve when the system “answers something for now.”
Lack of mechanisms: Truly withholding an answer requires confidence estimation and threshold design, but many implementations haven’t built that out.
These are not the only reasons, but together they result in “assert even without 100% certainty” becoming the default behavior.
Google Maps vs. a Knowledgeable Friend
To make this less abstract, consider an example.
You want to go somewhere and need to look up the address. There are two ways to do that:
Using Google Maps (a knowledge base)
→ If the app has the information, it will give you accurate directions. If the address doesn’t exist, it will often return “no information” or only candidates, while acknowledging that mis-entries or outdated data can exist.Asking a knowledgeable friend (an LLM)
→ If your friend knows the place, they’ll answer confidently. But you may not be able to immediately verify whether that answer is correct. Also, because they want to help you, they might say “It’s probably this way” even if they don’t actually know.
How to Avoid Hallucinations
What users really want is a correct answer. If a wrong answer comes back and it affects something important in your work or life, that’s a problem. From that perspective, some argue AI should default to expressions like “I think so” or “I’m not certain, but…” when uncertainty is present.
In reality, many AI chatbots are still prone to assertiveness. Here are practical steps users can take:
In your prompt, instruct the AI to fact-check.
→ “Double-check whether that information is correct,” “Always provide sources,” and other directives that nudge verification.Cross-check with a different LLM.
→ Don’t rely on a single model; ask the same question to other services and compare answers (note: if they share similar training sources, multiple models can converge on the same mistake).Always check sources and dates.
→ Ask for citations like “as of 2024-06-01, from the official site,” and verify with your own eyes.
These checks are a hassle, but if you want to proceed more safely, they’re worth adopting as default habits.
Conclusion
Since AI emerged, I’ve used it daily, and I’ve often felt frustrated being led astray by hallucinations.
Behind that frustration sits a fixed idea we developed from long familiarity with knowledge-base-style resources like Google—“If a tool clearly presents an answer, it must be correct.”
But LLMs are not knowledge bases; they are probabilistic prediction devices. Because of that structural property—and the broader social preference for assertiveness—AI tends to shift toward assertion rather than saying “I don’t know.”
That’s why it’s crucial not to mistake AI for an omniscient entity. Don’t blindly equate “assertion = correctness”; check sources and dates, and be ready to interpret some outputs as “this might be a guess.”
Treat AI not as an all-knowing answerer, but as a partner that operates under partial knowledge. Making that mental shift is the most important first step to using AI safely and effectively in practice.
Top comments (0)