Context Rot: Why Your AI Agent Gets Dumber the Longer It Runs

My agent once spent nine minutes fixing a bug it had already fixed.

Turn 12: it patched a missing null check in auth.ts. Tests went green. I said nice, keep going.

Turn 38: it read a stack trace that was still sitting in the context window from turn 11, concluded the fix had never landed, and re-patched a file that already had the patch. The second patch conflicted with the first. Tests went red. It then started "debugging" the failure it had just manufactured.

The model wasn't broken. The code wasn't cursed. The context was rotten. Context rot is the single most under-discussed reason your AI coding agent gets dumber the longer it runs, and it kicks in way before you hit any token limit.

TL;DR

• Context rot is the degradation in output quality as a context window fills up, even at 20% capacity. It is not the model hitting a wall, it is the model drowning in its own history.
• Three mechanisms: attention gets diluted across more tokens, information in the middle of a long prompt gets recalled worse than the edges, and the model's own earlier mistakes become "facts" it stays consistent with.
• Needle-in-a-haystack benchmarks look great because finding one weird sentence is easy. Real agent work means aggregating 40 facts while ignoring 400 distractors, which is a different and much harder task.
• The top offender in coding agents is stale file snapshots: the agent reasons about the version of the file it read 30 turns ago, not the one on disk now.
• Fix it by curating the working set: fresh sessions per task, a handoff file on disk, sub-agents as context firewalls, re-read before edit, and truncate tool output aggressively.

What is context rot?

Context rot is the measurable drop in an LLM's accuracy and instruction-following as the input context grows, independent of whether the context fits in the window. A model that nails a task with 5k tokens of context can fumble the identical task with 100k tokens of context, even when those extra 95k tokens contain the answer.

The mental model most developers have is a hard drive: 200k tokens of space, fill it up, and everything in there is equally available until it's full. The real behavior is closer to a room full of people all talking at once. Adding more people doesn't make the room smarter. It makes the one useful voice harder to hear.

Why does my AI agent get worse in long sessions?

Four things compound, and they hit at different times.

1. Attention dilution. Every attention head has to spread a fixed budget of probability mass across every token in the context. Add 50k tokens of npm test output and the tokens describing your actual requirements get a thinner slice. The model doesn't "forget" your instruction. It weighs it against 50k competitors.

2. Position effects. Recall is not flat across the context. Content at the very beginning and very end of a long prompt tends to be used far more reliably than content in the middle. Your system prompt is safe. The turn-1 requirement buried under 60 tool calls is not.

3. Self-poisoning. This is the vicious one. Models are trained to be consistent with their context. When the agent writes "the bug is in the retry handler" at turn 9 and is wrong, that sentence is now in the context as an assertion. Turn 30 does not treat it as a hypothesis, it treats it as an established finding. One bad guess quietly becomes the frame for every subsequent step.

4. Stale state. The context contains a snapshot of config.py from turn 6. The file on disk has been edited four times since. Nothing in the transcript flags the snapshot as expired, so the model reasons over a file that no longer exists. This is where my nine-minute loop came from.

My context window is 1M tokens. Doesn't that solve it?

No, and the benchmark that convinced you it does is measuring the easy version of the problem.

Needle-in-a-haystack tests plant one distinctive sentence in a pile of unrelated text and ask the model to find it. Models are excellent at this, because the needle is lexically weird and has essentially zero competitors. Passing it tells you retrieval works when the target is obvious.

Agent work is the opposite shape. You need the model to hold twelve related facts, notice that two of them contradict, ignore three hundred plausible-looking distractors that are semantically close to the answer, and then act. Difficulty scales with the similarity between the signal and the noise, not with the raw token count.

And a bigger window makes the failure mode worse in practice, because it removes the forcing function. When your window was 8k, you were compelled to summarize. At 1M, you just keep appending garbage until quality quietly falls off a cliff you can't see.

What actually poisons an agent's context?

Ranked by how much damage I see them do in real coding sessions:

1. Raw test and build output. A single failing Jest run can be 8k tokens of stack traces you already read and resolved. It stays forever. Pipe it through grep, or ask for the last 30 lines.
2. Failed attempts. Three abandoned approaches sitting in the transcript are three wrong mental models the agent keeps re-litigating. Worse, they read as prior work, so it avoids "repeating" a path that would actually work now.
3. Whole-file dumps. cat on a 1200-line file to fix one function is 15k tokens where 200 would do.
4. Stale snapshots of files that have since changed.
5. Politeness debris. Long confirmations, restated plans, "you're absolutely right" turns. Individually tiny, collectively a tax on every subsequent token.

How do I stop context rot?

Stop treating context as an append-only log. Treat it as a working set you curate.

Start a fresh session per task. The cheapest fix, and the one people resist most because restarting feels wasteful. It isn't. A clean 5k-token context outperforms a polluted 120k one at basically every task.

Write a handoff file. Before you reset, have the agent write NOTES.md: what's done, what's next, key decisions, the three files that matter. The file system is the durable memory. The context window is scratch space.

Use sub-agents as context firewalls. Let a sub-agent burn 60k tokens grepping through the repo and return a 400-token answer. The search noise never touches your main context. This is the highest-leverage pattern in agent design right now and most people use sub-agents for parallelism instead, which is the smaller win.

Force a re-read before every edit. Instruct the agent to read the current file immediately before modifying it. Kills the stale-snapshot class of bug outright.

Truncate tools at the source. head -50, grep -n, --reporter=dot. Cheaper than any summarization step.

Restate the goal at the end. Recency is a real advantage. Dropping a one-line "current objective" at the bottom of a long session measurably re-anchors behavior.

Why doesn't everyone just prune the context?

Because pruning fights the billing model, and money usually wins.

Prompt caching makes an unchanged prefix dramatically cheaper and faster to re-send. Append-only conversations preserve that prefix perfectly. The moment you edit or drop something from the middle, you invalidate the cache from that point forward and pay full freight on the rebuild.

So the economically obvious move is to never delete anything, which is exactly the behavior that produces context rot. Tools optimize for cost per turn, and quality per turn silently degrades. Anywhere you see a system that keeps growing its prompt and never compacts, that trade-off is why.

The right call: prune at task boundaries, not mid-task. You eat one cache miss and get a clean window for the next unit of work.

So does context rot mean long context is useless?

No. Context rot means long context is a resource with a quality gradient, not a container with a capacity limit. Your AI agent gets dumber the longer it runs because attention is diluted across more tokens, mid-context information is recalled less reliably, its own wrong guesses harden into assumptions, and file snapshots go stale without any signal that they have. The fix is not a bigger window, it is curation: fresh sessions per task, state written to disk instead of held in the transcript, sub-agents that absorb search noise, a forced re-read before every edit, and hard truncation on tool output. Judge context by signal density, not size. A 5k-token context of exactly the right material beats 200k tokens of everything you've ever said, every single time.

Comments

[Reid Marlow]

The prompt cache billing trap is where most agent setups fall apart. Seeing ninety percent cache hit rates on a growing prefix makes long sessions look cheap on paper. The hidden cost shows up around turn thirty when the model treats an old traceback from turn eight as an ongoing constraint and starts reconciling imaginary diffs.

Dumping raw tool output directly into the session history is usually the trigger. I pipe raw command stdout and full file reads into local scratch files on disk, then feed the agent only the exit status and a bounded summary. Rebuilding the context at a clear milestone triggers a cache miss, but an extra fraction of a cent on fresh input tokens is far cheaper than watching an agent spend fifteen minutes debugging state that already changed on disk.