DEV Community

Cover image for Less Context, Fewer AI Hallucinations
Richard Kersey
Richard Kersey

Posted on Originally published at chirpper.com AI-assisted

Less Context, Fewer AI Hallucinations

Originally published on Chirpper.

Everyone is trying to figure out how to give AI more context so it hallucinates less: bigger context windows, more retrieval, longer prompts, entire codebases pasted in. Sometimes less context is better. When you’re asking AI to check work for mistakes, it nearly always is. Here’s why.

Article cover 1920×1080@2x

More context isn’t free

The assumption is that more input means better output. The research says otherwise.

Stanford’s “Lost in the Middle” study found that model performance drops as input gets longer, even for models built for long context. Chroma’s 2025 “Context Rot” report tested 18 frontier models, including GPT-4.1, Claude 4, and Gemini 2.5. All of them got less reliable as input length grew, even on simple tasks.

A bigger context window is a bigger space for the model to get lost in.

The wrong context is worse than none

The most common advice for catching AI mistakes is to ask the model to “double check your work.” That usually means the same model, in the same conversation, with all of its original reasoning still sitting in context.

In 2024, Google DeepMind researchers tested whether large language models can correct their own reasoning without outside help. Mostly, they can’t. Without external feedback, models struggled to fix their mistakes, and sometimes they did worse after “correcting” themselves. (Huang et al., ICLR 2024)

There’s also a bias problem. A NeurIPS 2024 paper by Panickssery, Bowman, and Feng found that LLMs rate their own outputs higher than equally good text from other models or from people. The models can also recognize their own writing, and the better they are at recognizing it, the stronger the bias.

So a checker that can see everything has two problems. It’s anchored to the original reasoning, and it’s working in a degraded context window. It’s grading its own homework with the answer key it wrote.

The fix: separate the author from the auditor

Here’s the setup I use for code review:

  1. Agent 1: “Code review this PR.”
  2. Agent 2: “List every statement of fact in this review.”
  3. Agent 3 gets the PR and Agent 2’s list, but not Agent 1’s reasoning: “Verify each statement against the code.”
  4. Agent 3’s findings go back to Agent 1 to correct.

Agent 3 never sees how Agent 1 reached its conclusions. It only sees the claims and the source material, so it can’t be talked into agreeing. It has to check.

Rules that make it work

Strip the reasoning, keep the source. “Less context” doesn’t mean less information. The DeepMind finding is that self-correction fails without external feedback. The code, the document, or the dataset is that external feedback. Remove the author’s argument, not the ground truth.

Turn output into checkable claims. “Is this review good?” is a vibe check. “Is it true that this function never handles a null input?” can be verified. The extraction step in Agent 2 does most of the work.

Use a different model for verification when you can. The self-preference research suggests a fresh instance of the same model may still favor text that sounds like its own. A different model family reduces that.

Cap the loop. One or two correction rounds usually catch what matters. “Repeat until perfect” burns tokens and can create new problems.

Spend it where it counts. Anthropic reported that its multi-agent research system beat a single agent by 90.2% on its internal research eval, but at roughly 15 times the tokens of a normal chat. That’s research rather than code review, but the tradeoff is the same. Use this for output where a wrong fact is expensive, not for every prompt.

The takeaway

None of this is new. It’s why developers don’t QA their own code and writers don’t copyedit their own books. The person who made the thing is too close to it.

AI just makes the principle easy to apply. Spinning up a reviewer with clean context costs one more prompt. Give it the facts, hide the reasoning, and let it check.


Sources:
Huang et al., “Large Language Models Cannot Self-Correct Reasoning Yet”;
Panickssery, Bowman & Feng, “LLM Evaluators Recognize and Favor Their Own Generations” (NeurIPS 2024);
Liu et al., “Lost in the Middle” (TACL 2024);
Hong, Troynikov & Huber, “Context Rot” (Chroma, 2025); Anthropic, “How we built our multi-agent research system” (2025).

Top comments (0)