Stuffing the Context Window Is Making Your Agent Dumber: What the Research Says

Disclosure: I maintain Lynkr, an open-source gateway that (among other things) compresses agent tool outputs — so I have a horse in this race. This piece, though, is about the research, and every number in it is cited to its primary source.

There's an intuition almost every LLM user shares: more context = better answers. Million-token context windows are marketed as a capability. We paste in whole files "just in case." Our coding agents accumulate every grep result, every file read, every test log, on the theory that the model might need it.

The research says this intuition is not just wrong — it's backwards, and the size of the effect is embarrassing.

The result that should change how you build

The cleanest demonstration comes from the Hindsight memory system (arXiv:2512.12818, demo at ACL 2026). On LongMemEval — a benchmark of questions over long conversational histories — the same open-source 20B model scores:

• 39.0% when handed the full context — everything, the whole history
• 83.6% when handed a curated slice selected by a structured memory system

Same model. Same available information. The difference is that one setup made the model read everything, and the other selected what mattered. +44.6 points from subtraction.

It gets more uncomfortable: that 20B model with curated context also beats full-context GPT-4o, which scores 60.2% on the same benchmark. A model a fraction of the size, winning because someone cleaned its desk. As the authors put it, the memory architecture — not model scale — drives the performance.

One benchmark, one paper? No — this is a pile-on:

• "Lost in the middle" (Liu et al., 2023) established the shape of the problem early: models attend well to the start and end of long contexts and poorly to the middle — exactly where your agent's fifteenth tool result lives.
• Context-rot studies (notably Chroma's 2025 report) showed performance degrading as context grows even when the added tokens are relevant, and degrading faster when they're distractors.
• A whole 2026 research wave now treats context as a resource to be managed, not maximized: "Agentic Context Engineering" was accepted at ICLR 2026, active context compression systems prune their own working memory (arXiv:2601.07190), and two consolidating surveys (arXiv:2512.13564, arXiv:2603.07670) formalize memory as a write–manage–read loop — with "manage" doing the heavy lifting.

The marketing said "bigger window." The research says "better librarian."

Why more context makes things worse

Three mechanisms, all well-documented:

1. Attention is a budget, not a spotlight. Every token in context competes for attention mass. Pack in 50k tokens of tool output and the three lines that matter are now competing with 49,900 tokens of noise. Needle-in-a-haystack benchmarks — the ones vendors publish — test retrieval of a planted string, which models are good at. Real tasks require reasoning over the context, which degrades much faster.

2. Distractors don't just dilute — they actively mislead. The context-rot findings show semantically-similar-but-irrelevant content is worse than random filler. Your agent's context is full of this: old versions of the file it's editing, error messages from an already-fixed bug, grep hits from a deprecated module. Each is a plausible-looking wrong answer sitting one attention head away.

3. Position effects compound over turns. Agents append. Every turn pushes the important early material (the task! the constraints!) toward the middle of the context — the attention dead zone — while burying the recent signal under boilerplate tool output. A long agent session is a machine for constructing worst-case attention layouts.

Coding agents are the pathological case of all three at once: they generate enormous, distractor-dense, structurally-repetitive context (JSON tool results, file dumps, test logs) at machine speed, across dozens of turns.

What the research says works instead

The successful systems in the literature share a shape — they spend compute deciding what the model sees instead of showing it everything:

• Selection over inclusion. Hindsight's four memory networks (facts vs. experiences vs. summaries vs. beliefs) exist so retrieval pulls the right kind of memory for each question. The general lesson: retrieval into a small context beats residence in a big one.
• Compression as a first-class operation. Active-context-compression agents treat "shrink my working set" as an action the agent itself takes, on par with tool calls. Summarize the resolved, drop the superseded, keep the live.
• Structure beats soup. Tabular, labeled, deduplicated context consistently outperforms raw dumps of the same information — the model spends attention on content, not parsing.

What you can do about it today

You don't need a research memory system to benefit:

1. Treat context as a liability with interest, not an asset. Every tool result you leave in the window is re-read (and re-billed) every subsequent turn, while making each turn slightly dumber.
2. Compact aggressively and early. Whatever your agent's compaction/clear mechanism is (/compact, /clear, fresh sessions per task), use it before quality degrades — by the time you notice the agent going in circles, the context has been hurting you for many turns.
3. Scope sessions to tasks. One task, one session. The 40-turn omnibus session is the exact scenario the position-effect research warns about.
4. Isolate research from execution. Subagents (or separate sessions) that read a lot and report a little are context firewalls: the summary crosses over; the 30k tokens of grep output don't.
5. Compress tool outputs before they enter context. Test logs, JSON blobs, and directory listings compress 40–90% with zero information the model actually needs lost. Whether you do it with a proxy layer (this is the part where I mention that's what Lynkr does), a harness setting, or a wrapper script — do it somewhere.

The next time a model launch leads with context-window size, remember the 20B model with a good librarian beating GPT-4o with a hoard. Capacity isn't capability. Curation is.

---

Primary sources: Hindsight (arXiv:2512.12818) · benchmark data · Lost in the Middle (Liu et al.) · Active Context Compression (arXiv:2601.07190) · Memory in the Age of AI Agents — survey (arXiv:2512.13564) · Memory for Autonomous LLM Agents — survey (arXiv:2603.07670)