Debug Token Overflow: Why Your Agent Forgot Instructions

Your agent did not get dumber at turn 37. Its context window filled silently, the framework evicted the system prompt to make room, and later turns ran without the instructions you thought were still there. I spent an evening blaming the model before a twenty-line script proved the real culprit was my own token accounting.

When a Generous Token Allowance Made Me Stop Counting

I was testing a refactoring agent on MonkeyCode's free server option, with the free model allowance covering the whole experiment. Disclosure: This article was prepared as part of MonkeyCode's product outreach. The allowance was ten million tokens at the time of writing, which felt infinite, and that feeling was exactly the problem.

With that much headroom, I stopped counting tokens entirely. Why would anyone budget for something that generous? The answer is that the budget I ignored was not the one I thought I had. Context is a finite resource you have to measure, even when the monthly pool looks endless. OpenAI's guide to what tokens are and how to count them is the mental model I should have kept.

The complacency looked rational until I split the two budgets:

• Monthly allowance: ten million tokens, enough for dozens of long sessions.
• Session budget: the model's context window, a tiny slice of that pool.
• The trap: overflow inside one conversation does not wait for you to exhaust the monthly pool.

Ten million tokens covers a lot of experiments. It does not make a 128k context window any larger.

Symptoms Looked Like Model Failure, Not Context Eviction

The agent's job was simple: refactor a Python module, keep the tests green, and preserve error handling. Turns one through twenty were flawless, and the diffs looked like a careful human had written them.

Then the quality curve started sliding in a pattern that looked like the model had gotten worse:

• At turn 25, it stopped writing tests for new functions.
• At turn 31, it silently dropped the error-handling branches I had explicitly asked it to keep.
• At turn 37, it hallucinated a config key that never existed in the codebase.

The most confusing part was reproducibility. I reran the same task in a fresh session, and the agent handled it perfectly. Same model, same prompt, same server — different result. That meant the model was not broken and the endpoint was not throttled.

Hypothesis one: free-tier throttling, ruled out

My first instinct was to blame the free infrastructure, because that is always the easy story. I assumed the endpoint was rate-limiting me or routing my requests to a weaker backend after the first few minutes. I tested that by opening a brand-new session and sending the exact same failing prompt. It worked flawlessly, which proved the model and the endpoint were fine. The real variable was not the infrastructure; it was the conversation history itself.

Hypothesis two: the framework was dropping messages

I dumped the raw conversation log and counted what the framework had actually sent in that final failing turn. The log revealed the truth: the framework trims the oldest messages whenever its internal estimate crosses a threshold.

The catch was the estimator's blind spot. It only counted user and assistant turns, completely ignoring the system prompt, the tool schemas, and the few-shot examples. Its math said we were at seventy percent of context, while the real number was closer to eighty-two percent.

Root Cause: Hidden Token Budget Consumers the Framework Ignored

Here is a budget table with illustrative numbers from a typical long session. If you run tool-using agents, these categories should look familiar:

Budget consumer	Tokens	Framework counted it?
System prompt with formatting rules	~1,200	No
Tool schemas	~2,800	No
Few-shot examples	~4,000	No
Conversation turns	~89,600	Yes
Reserved output space	~8,000	No
Real total	~105,600 (82%)	Its estimate: 70%

The framework thought we were safe, so it kept trimming the oldest messages whenever its own estimate crossed the limit. And what was the oldest message in the conversation? My system prompt, with all the formatting and requirement rules the agent was supposed to follow.

The model did not degrade; it simply lost its instructions. Every subsequent turn was the agent doing its best with no memory of what I had asked for. A fresh session looked smarter only because the prompt was still there.

Compare the two views of the same failing turn:

• Framework estimate: conversation turns only, about 70% full, keep going.
• Reality: system prompt plus tool schemas plus few-shots plus turns plus reserved output, about 82% full, oldest message already eligible for eviction.

Once I saw that gap, the quality slide at turns 25, 31, and 37 stopped looking like model failure and started looking like missing instructions.

Count Real Tokens, Then Follow 50% and 70% Thresholds

The fix was not a better model or a bigger context window; it was measuring reality. I wrote a small harness that counts the actual prompt tokens before every turn, using the same tokenizer family as the endpoint. I used tiktoken for the count and the OpenAI tokenizer as a visual check on individual messages.

# token_budget.py — run this before every long agent session
import tiktoken

def prompt_tokens(messages, encoding_name="cl100k_base"):
    enc = tiktoken.get_encoding(encoding_name)
    return sum(
        len(enc.encode(str(part)))
        for msg in messages
        for part in (msg.get("role", ""), msg.get("content", ""))
    )

messages = load_conversation("session.jsonl")  # your real turns
used = prompt_tokens(messages)
limit = 128_000  # your endpoint's actual context limit

print(f"prompt: {used} tokens ({used / limit:.0%} of budget)")

if used / limit > 0.5:
    print("ACTION: summarize now — you are past the safe zone")
if used / limit > 0.7:
    print("ACTION: checkpoint and start a fresh session")

The script is deliberately dumb, and that is the point. It counts every message role, every tool call, and every schema string, then compares the total against the real context limit of your endpoint. No framework estimate, no rounding, no surprises.

The decision rule I now run

The harness only helps if you act on its output, so I converted it into a hard rule for every agent session. I now treat fifty percent as the warning line and seventy percent as the abort line.

1. Under 50%: keep going, but log the token count with every turn.
2. 50% to 70%: summarize the conversation into a fresh pinned system message.
3. Over 70%: checkpoint the state, write a handoff summary, and start a new session.

I also pin the system prompt so the trimming logic can never evict it. Most frameworks support that flag, and enabling it costs nothing.

This harness is an estimate, not ground truth, because token counts vary by tokenizer and by how the endpoint encodes tool schemas. Treat the numbers as a warning system, not as an exact accounting ledger. Free endpoints also come with their own constraints, including rate limits and queueing, so do not use this setup for production workloads with strict latency requirements. And if your agent runs one-shot prompts or short sessions, this entire problem is irrelevant to you.

Generous free allowances are a trap, because they remove the pressure that normally forces you to watch your budget. The failure I hit was not caused by the model, the framework, or the free server; it was caused by my assumption that a big allowance meant I could stop measuring.

Measure the actual prompt, not the framework's estimate, and pin the messages that matter. If you want to reproduce this failure yourself, the free server option and the ten-million-token allowance are enough to hit it in an afternoon — and now you know exactly where to look.

Do this next: copy token_budget.py in front of your next long agent run, pin the system prompt, and abort or summarize at 70%. Then tell me in the comments which turn first crossed 50% of your context window — I will share how I would write that handoff summary.