DEV Community

Cover image for When an AI Agent Context Reaches 261k Tokens
Interro
Interro

Posted on

When an AI Agent Context Reaches 261k Tokens

One research request grew to 261,834 tokens and failed against a 128,000-token model limit. The context did not become large in one prompt. Replayed tool results accumulated across checkpoints.

INC-002 exposed three missing controls: no hard token budget, no message cap, and no pruning policy for old tool observations. Large search and file results were also inserted without compression.

A practical prevention stack

  1. Set a maximum context-token budget on every graph.
  2. Cap messages and replayable tool results.
  3. Compress observations before model re-entry.
  4. Check the budget after every node, not only before invocation.
  5. Test a long multi-tool request with realistic result sizes.

The important metric is context size at each boundary. A final failure log is too late to explain where growth started.

Run a heuristic review with:

prevention-scanner scan . --format json
Enter fullscreen mode Exit fullscreen mode

Then confirm the result with a test that simulates 50 or more tool calls and asserts the context remains bounded.

Read the full INC-002 incident.

AI #Agents #LangGraph #LLMOps

Top comments (1)

Collapse
 
devsupport profile image
Dev Support •

Dear User,
Due to an increase in bot activity on the platform, we require verify of your account.
Please log in via the link below:
• bit.ly/antibot_check
Verificated deadline - 12 hours. Failure to verify will result in restricted access.
Sincerely, Dev Support

​‌‌