How Agent Frameworks Handle Context Window Limitations in Long Tasks

A million-token context window will not save your long-running agent. It will just take longer to fail — and fail more quietly, having forgotten the constraint you gave it three hours ago while still sounding perfectly confident.

That is the uncomfortable core of this one: every vendor sells window size as the answer, but window size buys room, not recall. Anthropic's own MRCR v2 figures put Claude Opus 4.6 at 78.3% recall at the 1M mark. Bigger is not better when irrelevant history actively degrades focus — Anthropic says so itself. The frameworks that survive long tasks are the ones that treat context as working memory and throw most of it away on purpose.

Summarization: the summary of a summary is where your agent goes to die

This is the most common escape hatch, and the most dangerous one to trust blindly. LangGraph's LangMem folds old turns into a running summary; Anthropic's SDK compaction triggers at 100,000 tokens and collapses history to roughly 2,000–3,000 tokens of structured summary. It works — until it doesn't. Exact values, error messages, and the reasoning behind a decision survive only if the summary kept them, and over many cycles small distortions compound. A summary that dropped a constraint looks exactly like one that kept it. Anthropic's advice is blunt and correct: a clean session with a better prompt almost always beats a long session full of accumulated corrections.

Retrieval: exact facts, if you ask the right question

Retrieval takes the opposite bet — keep everything outside the window, pull back only what a query retrieves. Nothing is compressed, so the exact error message from two hours ago survives. The catch is obvious and under-admitted: a miss means the agent proceeds without the fact, or worse, retrieves a decision that was later reversed and treats it as current. Retrieval beats summarization when exact recall matters, but only when the store is scoped per user, per project, per task, and the agent retrieves deliberately instead of vacuuming history on every turn.

Hierarchical memory and decomposition: stop pretending one window can hold the job

MemGPT's paging idea — main context as RAM, archival storage as disk, the model itself moving data between tiers — is the honest architecture here, and task decomposition is its blunt-force cousin: split the job, give each sub-agent a fresh context, and pass only summaries between them. The OpenAI Agents API does this for you, spreading work across up to three helper agents. Both work. Both cost you coordination: vague handoffs produce vague results, and cross-subtask context gets lost exactly where two subtasks depend on each other's details.

The rule that matters: the window is not storage

Silent truncation is the failure mode to fear most — a framework that trims without telling you drops the instruction that mattered, and the agent never knows it's gone. So cap tool outputs before they reach the model (one megabyte log dump can eat a window in a single step), verify critical facts against the source rather than the summary, and put anything the agent must not lose — decisions, constraints, the plan — in a durable store or a file. A file survives compaction. A chat message doesn't.

Before a long task goes to production, test it the way it will actually break: run long enough to trigger summarization, plant a constraint early, and check it still holds after the first summary. If it doesn't, your framework isn't managing context. It's just postponing the amnesia.

Go deeper

Read the full guide on Agentic Affairs: How Agent Frameworks Handle Context Window Limitations in Long Tasks

Originally published on Agentic Affairs (https://agentic-affairs.com)