Context Compaction Patterns: When to Summarize, Truncate, or Retrieve

Context Compaction Patterns: When to Summarize, Truncate, or Retrieve

A context window fills up mid-session. Something has to leave. Most systems answer that problem with one move: summarize everything and hope the important parts survive. It's the same instinct that made "just add more context" the reflexive answer to the budgeting problem — a single technique, applied everywhere, regardless of what's actually being thrown out.

Compaction isn't one operation. It's three, and they destroy information in completely different ways. Summarizing keeps the gist and loses the precision. Truncating keeps the recent and drops the old outright. Retrieving loses nothing permanently, it just stops sitting in front of the model until something asks for it again. Pick the wrong one for a given piece of context, and the failure doesn't announce itself as a compaction bug. It shows up as the model forgetting a decision it already made, misreading a file it already read, or confidently inventing an API it saw an hour ago.

The Mistake Hiding Inside "Just Summarize It"

Ask an engineer how their system handles a full context window and the answer is usually one sentence: it summarizes. That's the imposter definition of compaction, and it's wrong in a specific way. Summarization is one tool in a set of three, best suited to exactly one kind of content: information where the gist matters more than the wording, and where there's no cheaper way to get it back later.

Truncation is not a worse version of summarization. It's a different bet entirely, that the old content isn't coming back and doesn't need a lossy stand-in either, because it's either genuinely stale or cheaply re-obtainable from somewhere else. Retrieval is not a fallback for when summarization would lose too much. It's the option that applies whenever the real thing still exists somewhere addressable, and fetching it again costs less than carrying a compressed copy around indefinitely.

Treat all three as interchangeable synonyms for "make it smaller," and a system will compact the wrong things the wrong way. A four-line grep result and a forty-turn architectural discussion do not fail the same way when compressed. Building one pipeline for both is the actual root cause of context management that quietly breaks agent behavior weeks into production, long after anyone remembers the shortcut was taken.

The One Question That Decides Which Technique to Use

There's a single question that sorts almost every piece of discardable context into the right bucket: if this disappeared right now, could you get it back, cheaply, from somewhere else?

If yes, retrieve. The content lives in a file, a database, a vector index, a URL. The model doesn't need to carry it, it needs to know it exists and how to ask for it again. Compressing it into a summary is strictly worse than dropping it and re-fetching the original when it's actually needed, because a summary of a file is a lossy copy of something you already have a lossless copy of.

If no, but the value sits in the narrative rather than the exact wording, summarize. A ninety-minute debugging conversation isn't reconstructable from any external source. Nobody logged the reasoning trail. But the decisions made along the way, the constraints established, the dead ends already ruled out, survive compression into a few paragraphs. The exact phrasing of turn forty doesn't matter. The fact that turn forty ruled out a database migration does.

If no, and the content has already been superseded by something more recent, truncate. An old grep result against a file that's since been edited isn't worth summarizing. It isn't worth a placeholder either. It's just wrong now, and the right move is to let it go rather than spend tokens compressing something that's no longer true.

The Trigger Matters As Much As the Technique

Even the right technique executed at the wrong moment causes damage. Wait until a conversation is nearly out of room before summarizing it, and the summary is built from an already-degraded view: the model is compressing a conversation it's already struggling to reason over cleanly. Run the same operation earlier, well before pressure builds, and the summary comes from a clean, complete picture instead of salvage material.

The same logic holds for the other two techniques. Truncate too early, before content is genuinely stale, and something still in use disappears with no summary to fall back on. Retrieve too late, after the model has already tried to answer from what it half-remembers instead of the real source, and the retrieval fixes an error that already happened instead of preventing it. Category tells you which technique to use. Timing tells you whether that technique does what it's supposed to.

Same System, Three Different Answers

The clearest proof this isn't theoretical is a single coding agent making three different calls inside one session. Claude Code doesn't run one compaction function against everything that gets long. It runs three, for three different problems. Old tool results that are no longer needed, a grep from many turns ago against a file that's since changed, get cleared outright: no summary, no reference kept, because the content is already stale and reconstructing it would mean re-running a command against a file that no longer matches. Large but still-relevant tool output, a long file read, a verbose test run, gets a different treatment: the full content moves to disk, and only a path reference stays in the model's view, pulled back in full if the work touches it again. Conversation history gets a third treatment entirely, because there's no disk copy of a debugging conversation sitting anywhere: when the session nears its limit, the turns themselves get compressed into a structured summary that preserves decisions and task state rather than exact wording.

Three techniques, three categories, inside one product. Cursor draws a similar line somewhere else. Ask it about a codebase and it doesn't truncate the repository or summarize old files to make room, it retrieves. The codebase gets indexed with embeddings ahead of time, and a request pulls back the handful of chunks that are actually relevant through semantic search, the same underlying bet as Claude Code's disk-backed tool outputs: the source is stable and cheaper to fetch again than to carry around compressed.

What Breaks When You Pick Wrong

The failure mode of picking the wrong technique rarely looks like a crash. It looks like a subtly worse answer that's hard to trace back to its cause. Summarize a tool output that needed to be reasoned over exactly, a JSON schema, a diff, an error stack, and the model starts working from its own paraphrase of that structure instead of the structure itself. It will still sound confident. It will just be wrong in ways that are expensive to catch, because the failure surfaces downstream of the compaction, not at the moment it happened.

Truncate something that hadn't actually been superseded yet, and the model loses a fact it still needed, with no summary to fall back on and no signal that anything is missing. This is the quieter failure. There's no error message, just a gap where a constraint used to be, and the next few turns build on an incomplete picture before anyone notices the output is already wrong.

Most teams don't choose the wrong technique through bad judgment. They choose it through convenience. Summarize-everything is the easiest thing to implement, one function call regardless of what's being compressed. Truncate-the-oldest is the second easiest, because it needs no model call at all. Retrieval is the one that gets skipped most often, not because it's technically harder, but because it requires deciding, in advance, that some piece of context deserves to live outside the conversation entirely, addressable rather than carried. That decision is architecture. It has to be made before the system is under pressure, not improvised the moment the window fills up.

Every compaction is a bet that the part being kept matters more than the part being thrown away. Most systems make that bet the same way for everything they discard. The ones that hold up under real use make it three different ways, on purpose, category by category, before they're ever forced to.