DEV Community

AgentBasics
AgentBasics

Posted on Originally published at agentbasicshq.com

Context-Window Management for Agents: Poll the Pressure, Don't Catch the Error

#agents #llm #softwareengineering

Everyone handles context exhaustion by catching the token-limit error. That's handling the rare failure mode. The common one has no error at all: somewhere around 70-80% window fill, your agent quietly stops following the instructions from the top of the context. Nothing crashed. The output just got worse, and by the time it's obvious, you've burned an hour of turns on a confidently amnesiac agent.

The fix is treating context like a timeout: a budget you poll before it fails, with graduated actions instead of a cliff.

A cheap gauge beats an accurate one

You don't need a tokenizer call per loop iteration — you need a cheap, monotonic signal. Characters-per-content-type is enough:

_CPT = {"text": 4.0, "code": 3.2, "json": 2.8, "cjk": 1.6}

def estimate_tokens(content, kind="auto"):
    # auto-detect: CJK density, JSON-looking lines, code markers
    return len(content) / _CPT[kind]

pressure = ContextPressure(model_window=128_000, system_overhead=8_000)
pressure.add("user", prompt)
pressure.add("tool", big_search_result, kind="json")
p = pressure.reading()      # e.g. 0.73 — poll this every loop iteration
Enter fullscreen mode Exit fullscreen mode

Heuristic estimates drift per model, so calibrate once: send a known prompt, read the API's actual usage.prompt_tokens, and store the correction factor. After that the gauge tracks reality closely enough to act on — and it costs microseconds per iteration.

Graduated actions, not a cliff

The reading only matters if the loop reacts before the cliff. The ladder we run:

  • ≥ 0.70 — summarize old tool results. Tool outputs are the fat: a 40KB search result summarized to 1KB keeps 97% of the signal. Replace the message content in place, keep a pointer to the full artifact on disk.
  • ≥ 0.85 — drop oldest turns. Old tool calls whose results were already used are dead weight. Drop oldest-first, never the system prompt, never the current task.
  • ≥ 0.95 — force a state checkpoint. Write a structured summary of decisions + state to a scratchpad, then continue with a trimmed context built from it. This is the difference between "agent forgot everything" and "agent resumed from notes." 
if p >= 0.95:
    checkpoint = summarize_state(messages)   # decisions, facts, next step
    messages = rebuild_from(checkpoint)
elif p >= 0.85:
    drop_oldest_used_tool_turns(messages)
elif p >= 0.70:
    summarize_fat_tool_results(messages)
Enter fullscreen mode Exit fullscreen mode

Why 70% and not 99%

Because the failure you're preventing isn't the API error — it's the quality decay that starts well before it. Long-context models demonstrably follow early instructions worse as fill rises; the "lost in the middle" effect is real and it's not rescued by a bigger window. Acting at 70% is what keeps the instructions inside the agent's attention, not just inside the token limit.

This is the context_pressure module in AgentOps Autopilot (MIT, stdlib only, 24/24 tests). It's the last of the six reliability modules — the other five cover retries, JSON repair, turn budgets, failure journals, and receipt-auditing partial runs.

Back to all posts, or grab the toolkit at agentbasicshq.com.

Top comments (0)