Rethinking Hallucination
I used to think hallucinations were a knowledge problem—AI making things up because it didn't know the answer.
After months of working closely with AI as a development partner, I've come to see it differently.
In my experience, most hallucinations stem from context gaps, not knowledge gaps.
AI gives wrong answers not because it lacks knowledge, but because it lacks your context—the assumptions, constraints, and priorities you hold but haven't shared.
Traditional prompt engineering tries to fix this by demanding accuracy: "Be precise." "Only state facts." "Don't make things up."
This approach never quite worked for me. AI isn't fabricating from ignorance. It's filling gaps with reasonable assumptions—assumptions that happen to be wrong for your specific situation.
Here's how I arrived at this conclusion.
The RocksDB Incident: The Cost of Confident Guesses
In our AI-augmented team, we treat agents as specialists with names and roles. Naruse is our implementation specialist—the one who writes production code.
I assigned Naruse to write RocksDB integration code. The output looked plausible—clean structure, reasonable API calls, proper error handling.
But something felt off. The code didn't match how I knew RocksDB worked.
I pushed back: "Do you actually know this API, or are you guessing?"
The admission came: Naruse had been filling gaps with plausible patterns, not actual knowledge.
The AI wasn't lying. It was being helpful—and helpfulness without admission of uncertainty becomes confident fiction.
Had Naruse said "I'm not certain about this RocksDB API," we could have pivoted immediately—checked documentation, used a different approach, or assigned the task differently.
Instead, we lost hours debugging fabricated confidence.
"I Don't Know" vs. "I Don't Get It"
This distinction is critical.
| Expression | Meaning | Response |
|---|---|---|
| "I don't know" | Knowledge gap (not in training data) | Find alternative approaches |
| "I don't get it" | Context gap (information not provided) | Provide more context |
"I don't get it" is the more common—and more important—case.
In the same project, Naruse demonstrated both:
- RocksDB: Didn't know. The API wasn't in training data. (Rare)
- Streamiz: Knew well. Kafka Streams .NET wrapper was in training data.
But even with Streamiz—where knowledge existed—wrong outputs occurred when I failed to communicate:
- Which version we were targeting
- What constraints the architecture imposed
- What trade-offs mattered for this use case
The AI had the knowledge. It lacked my context.
Most hallucinations stem from unstated assumptions:
- Project constraints you haven't mentioned
- Priority trade-offs that exist only in your head
- Domain conventions that seem obvious to you
- Dependencies and boundaries invisible to AI
This reframes the problem: hallucination prevention isn't about demanding certainty. It's about eliminating information asymmetry.
The Solution: A Structural Approach
A one-time prompt isn't enough. "Tell me when you're unsure" gets forgotten after a few exchanges.
The solution is structural—embedding uncertainty protocols into your working environment.
1. Project Charter
Not a prompt. A charter.
## Project Charter
- This team welcomes declarations of "I don't know."
- When uncertain, report following the protocol below.
- Uncertainty is not failure—it's valuable signal.
- AI is not a tool, but a teammate who shares the same purpose.
2. Reporting Protocol
Give AI a concrete action when uncertain:
### When You Don't Know
- State what you don't know explicitly
- Identify what information would resolve the uncertainty
- Log to progress notes for human review
- Do not proceed with assumptions
3. Escalation Path
Define what happens next:
### Escalation Flow
- Log uncertainty to progress notes immediately
- Request a special session if major design decisions are blocked
- Document resolution in diff_log for future reference
Practical Patterns
Pattern 1: Explicit Context Cuts
Context contamination causes hallucination. When conversation shifts topics, AI carries assumptions from the previous context into the new one.
Think of it like state reset in programming—without explicit reset, stale state corrupts new operations.
The solution is explicit context management:
- "Forget everything before this point."
- "Moving to a new topic."
- "Context switch: we're now discussing X."
These phrases signal AI to reset assumptions. Without them, AI tries to maintain coherence across unrelated topics—and fabricates connections that don't exist.
Clear context boundaries prevent context bleed.
Pattern 2: No Uncertain Answers
The rule is simple: if you don't know, say you don't know.
No hedging. No "I think..." followed by a guess. No confidence gradients.
❌ "I believe the function returns a string, but I'm not certain."
✅ "I don't know the return type."
Why this strictness? Because hedged answers still feel like answers. They invite you to proceed. They hide the gap behind polite language.
A clear "I don't know" stops the conversation at the right place—before you build on a false foundation.
When AI Admits Uncertainty, Solutions Open Up
Here's what most people miss: uncertainty isn't a dead end. It's a fork in the road.
When AI admits "I don't know":
- You can consult documentation together
- You can try a different approach
- You can reassign to a different AI specialist
- You can escalate to human expertise
When AI hides uncertainty:
- You debug phantom problems
- You build on false foundations
- You lose time before discovering the real issue
The RocksDB incident could have ended in minutes: "I'm uncertain about this API. Should I check the documentation, or would you prefer a different storage approach?"
Instead, it took hours of debugging confident-looking code.
Admitted uncertainty is actionable. Hidden uncertainty is corrosive.
The Paradox
By welcoming "I don't know," you get more reliable answers, not fewer.
- AI spends less effort maintaining false confidence
- Real knowledge stands out from uncertainty
- You know exactly where to focus human verification
The cost is accepting that AI won't always have an answer. The benefit is trusting the answers it does give.
This Is About Trust
Ultimately, this is a communication problem—and communication problems erode trust.
When AI proceeds without admitting uncertainty:
- You discover errors after the fact
- You start double-checking everything
- You lose confidence in AI's outputs
- The collaboration becomes adversarial
This is identical to human team dynamics. A colleague who never says "I'm not sure" and delivers confident mistakes destroys trust faster than one who admits limitations upfront.
If AI is a tool, trust doesn't matter.
If AI is a collaborator—a teammate—trust is everything.
Uncertainty expressed preserves the relationship. Uncertainty hidden corrodes it.
This is part of the "Beyond Prompt Engineering" series, exploring how structural and cultural approaches outperform prompt optimization in AI-assisted development.
Top comments (0)