Context compaction is silently destroying your LLM agent's memory

Linford Reyes — Tue, 11 Aug 2026 17:05:24 +0000

Context compaction is silently destroying your LLM agent's memory

Tl;dr: Long-session LLM agents lose governing rules, todos, and decisions when context compaction runs — and the loss is usually silent. I built a zero-dependency Python library (memory-anchor) that snapshots that state verbatim before compaction and re-injects it after, plus a CLI to audit how much a compaction actually destroyed.

The problem nobody measures

Run any agent long enough and it hits the context window. The fix everyone reaches for is compaction: old turns get summarized into a paragraph and the summary replaces them. Cheap summarizers — especially the fast flash models everyone uses to keep costs down — flatten detail. In my own agent's logs:

a governing rule got paraphrased (the agent started behaving differently and nobody noticed why)
a pending todo vanished ("what was I doing?")
a decision's rationale was rewritten (settled questions got re-litigated)
a verification path disappeared (half-verified work got reported as done)

That last one is the scary one. After a 35% compaction pass, my agent forgot it had delegated a subtask and never collected the result. Not a hallucination — a silent state loss.

Why existing memory systems don't cover this

The current memory stack (mem0, Letta, Hindsight, and friends) remembers facts — entities, preferences, retrieval chunks. None of them guarantee that the rules and work state governing this session survive a compaction byte-for-byte. Research-backed systems like MemGPT treat memory as a tiered store, and Claude's own compaction is a black box — you don't get to see what it threw away.

There are components everywhere, but no off-the-shelf, framework-agnostic answer. So I built one.

What it does

preserve(ctx) ─► manifest (4 lists, verbatim) ─► [summarizer runs] ─► recover(ctx) ─► recovery block injected at head of messages

Four small pieces, pure stdlib, zero dependencies:

StateManifest — rules / todos / decisions / progress, serialized to JSON, with incremental merge(): done todos never resurrect, superseded decisions never reappear.
MemoryStore — atomic (tmp+rename) local JSON persistence, per-session indexed.
RecoveryInjector — rebuilds the recovery block; L1 (immutable rules) is never trimmed.
CompactableMemory — a two-line facade:

from pathlib import Path
from memory_anchor import CompactableMemory

mem = CompactableMemory(base_dir=Path(".memory"))
mem.preserve(ctx)                              # before compaction
messages = mem.recover(ctx, messages, summary) # after compaction

The part I'm most proud of: auditing compaction damage

v0.3 added cam judge — point it at your pre-compaction manifest and the summary that replaced it, and it classifies every item as verbatim (survived), paraphrased (semantics preserved), or lost. It's a CI gate, not a vibes check:

cam judge --before manifest.json --after summary.txt --min-verbatim 90
# exit code 1 if fewer than 90% of items survived byte-for-byte

The numbers from my own system are not flattering to the status quo. Running the same 8KB briefing through two compressors:

compressor	verbatim survival	outcome
extractive (built-in)	~60%	2 governing rules silently lost
caveman (aggressive)	100%	all items survived byte-for-byte

The judge didn't just measure the damage — it exposed it. That extractive pass was silently eating rules every single run, and nothing in the pipeline was looking.

Try it

git clone https://github.com/44334433/memory-anchor
cd memory-anchor && pip install -e . && pytest   # 31 tests

Or script it into your cron/CI without Python:

cam before my-session --rule "R1|never paraphrase governing rules|100" \
  --todo "ship v0.2|pending|run the drill"
# ...compaction happens...
cam after my-session --messages messages.json --budget 2000

Honest limitations

This is not a semantic memory system. It preserves work state you explicitly declare — it won't infer what matters on your behalf (yet).
judge uses diff-based matching, not semantics. A truly reworded rule may count as "paraphrased" rather than "lost" — the threshold (45%) is a documented heuristic.
It's framework-agnostic by design: bring your own summarizer, hook the two calls wherever your pipeline compacts.

If your agent has ever "forgotten" something after compaction, you already know this pain. Anchor the state that must survive — and start measuring what your summarizer is destroying. Star/watch the repo if you want to see the framework adapters (LangChain, Claude Code, OpenHands) land — I'm holding off until real integrations ask for them.

DEV Community: Linford Reyes

Context compaction is silently destroying your LLM agent's memory

Context compaction is silently destroying your LLM agent's memory

The problem nobody measures

Why existing memory systems don't cover this

What it does

The part I'm most proud of: auditing compaction damage

Try it

Honest limitations