AutoGen's hidden token tax: why a 3-agent chat costs 15× what you expect
Cost-audit series, episode 2. This series began with an AI agent that burned 136M tokens overnight →.
AutoGen is Microsoft's multi-agent framework. It's genuinely good at orchestrating agents that hand off work to each other. But its default memory model has a cost shape that surprises almost every team that hits it in production.
This audit shows you exactly where the tokens go, with line numbers.
The setup: a 3-agent RoundRobin chat
The canonical AutoGen pattern is a RoundRobinGroupChat with N agents taking turns on a task. Here's the minimal version from the docs:
from autogen_agentchat.agents import AssistantAgent
from autogen_agentchat.teams import RoundRobinGroupChat
from autogen_agentchat.conditions import MaxMessageTermination
planner = AssistantAgent("planner", model_client=client, system_message="You plan.")
coder = AssistantAgent("coder", model_client=client, system_message="You code.")
reviewer = AssistantAgent("reviewer", model_client=client, system_message="You review.")
team = RoundRobinGroupChat(
[planner, coder, reviewer],
termination_condition=MaxMessageTermination(max_messages=10),
)
await team.run(task="Build a web scraper for Hacker News.")
Three agents, 10 turns total (~3–4 turns each). Seems cheap. It isn't.
The default context: unbounded, per-agent
Every AssistantAgent gets its own UnboundedChatCompletionContext by default:
# autogen-agentchat/src/autogen_agentchat/agents/_assistant_agent.py, __init__ (L708)
if model_context is not None:
self._model_context = model_context
else:
self._model_context = UnboundedChatCompletionContext()
UnboundedChatCompletionContext.get_messages() returns self._messages — the full list, no cap, no truncation:
# autogen-core/.../model_context/_unbounded_chat_completion_context.py (a ~20-line file)
async def get_messages(self) -> List[LLMMessage]:
"""Get at most `buffer_size` recent messages."""
return self._messages
(The docstring says "at most buffer_size" — that's a copy-paste artifact from BufferedChatCompletionContext. There is no buffer. It returns everything.)
The handoff tax: every agent sees every message
When an agent's turn arrives, on_messages_stream adds all incoming messages to its own context before calling the LLM:
# _assistant_agent.py, in on_messages_stream (STEP 1: "Add new user/handoff messages
# to the model context")
await self._add_messages_to_context(
model_context=model_context,
messages=messages, # ← the full message_thread from the group manager
...
)
And _add_messages_to_context appends each one:
# _assistant_agent.py, static method _add_messages_to_context
await model_context.add_message(llm_msg)
...
await model_context.add_message(msg.to_model_message())
The group manager (BaseGroupChatManager) maintains a single _message_thread and appends every response to it:
# _base_group_chat_manager.py
self._message_thread: List[BaseAgentEvent | BaseChatMessage] = []
...
await self.update_message_thread(delta) # called after every agent response
So at turn T, the agent receiving the baton gets T-1 messages added to its already-growing context. Its context now contains everything it has ever seen.
The math: O(N × T²) total tokens
Let's be precise. Define:
- T = total turns in the conversation
- N = number of agents
- m = average tokens per message (system prompt + response, ~300 tokens is realistic for a coding task)
Each agent speaks every N turns. When agent i speaks on turn t, its context contains all t-1 prior messages (because it has been accumulating them since turn 1).
Tokens consumed by agent i on turn t:
context_tokens(t) = (t - 1) × m
Total tokens for agent i across all its turns (it speaks at turns N, 2N, 3N, … up to T):
Σ (kN - 1) × m for k = 1 to T/N
≈ m × N × (T/N)² / 2
= m × T² / (2N)
Total tokens across all N agents:
N × m × T² / (2N) = m × T² / 2
The N cancels. Total cost scales as T² regardless of how many agents you add.
Worked example: 10 turns, 3 agents, 300 tokens/message
| Turn | Agent | Context size (messages) | Tokens in this call |
|---|---|---|---|
| 1 | planner | 0 prior + system | ~300 |
| 2 | coder | 1 prior + system | ~600 |
| 3 | reviewer | 2 prior + system | ~900 |
| 4 | planner | 3 prior + system | ~1,200 |
| 5 | coder | 4 prior + system | ~1,500 |
| 6 | reviewer | 5 prior + system | ~1,800 |
| 7 | planner | 6 prior + system | ~2,100 |
| 8 | coder | 7 prior + system | ~2,400 |
| 9 | reviewer | 8 prior + system | ~2,700 |
| 10 | planner | 9 prior + system | ~3,000 |
| Total | ~16,500 tokens |
Naïve expectation (10 calls × 300 tokens each): 3,000 tokens
Actual: ~16,500 tokens — 5.5× more.
At 20 turns it's ~63,000 tokens vs 6,000 expected — 10.5× more.
At 30 turns: ~139,500 tokens vs 9,000 — 15.5× more.
The multiplier grows linearly with T. This is the same O(T²) shape as ConversationBufferMemory in LangChain — but AutoGen's version is per-agent, so it's easy to miss in per-call logs.
Why per-call logs hide this
If you're watching your LLM provider's per-call token counts, you see something like:
call 1: 300 tokens ✓ cheap
call 2: 600 tokens ✓ fine
call 3: 900 tokens ✓ ok
...
call 10: 3,000 tokens ← this one looks expensive
Each call looks like a modest increase. The cumulative total — 16,500 — only shows up when you sum across the run. Most observability dashboards show per-call costs, not per-run totals. The runaway is invisible until the bill arrives.
The fix: cap the context
AutoGen ships two bounded alternatives, named in the AssistantAgent.__init__ docstring (around L1034 of _assistant_agent.py): BufferedChatCompletionContext (limits message count) and TokenLimitedChatCompletionContext (limits tokens):
Option 1: BufferedChatCompletionContext (sliding window)
from autogen_core.model_context import BufferedChatCompletionContext
coder = AssistantAgent(
"coder",
model_client=client,
model_context=BufferedChatCompletionContext(buffer_size=5), # last 5 messages
)
Cost shape becomes O(T × buffer_size) — linear. For buffer_size=5 and 30 turns: ~42,000 tokens vs 139,500 unbounded. 3.3× cheaper.
Option 2: TokenLimitedChatCompletionContext (token budget)
from autogen_core.model_context import TokenLimitedChatCompletionContext
coder = AssistantAgent(
"coder",
model_client=client,
model_context=TokenLimitedChatCompletionContext(token_limit=2000),
)
Caps the context at a fixed token budget. More predictable than a message count because message sizes vary.
Which to use?
| Scenario | Recommendation |
|---|---|
| Short tasks (≤10 turns) | Default is fine; monitor cumulative cost |
| Long tasks (>10 turns) | BufferedChatCompletionContext(buffer_size=8–12) |
| Strict cost budget | TokenLimitedChatCompletionContext(token_limit=N) |
| Need full history | Default + add per-run cost alerting (see below) |
Detecting this in CI before it hits production
The pattern is detectable statically: any file that instantiates AssistantAgent without a model_context= argument is using the unbounded default.
# Flag unbounded AssistantAgent instantiations
grep -rn "AssistantAgent(" src/ | grep -v "model_context="
For dynamic detection — measuring actual token growth across a run — this is exactly what tokenscope does: it instruments LLM calls, tracks per-run cumulative cost, and can block a CI build when a PR's token delta exceeds a threshold.
The wartzar-bee/ci-guardrail GitHub Action wraps tokenscope into a one-line workflow addition:
- uses: wartzar-bee/ci-guardrail@v1
with:
token_threshold: 50000 # block if PR adds >50k tokens/run
github_token: ${{ secrets.GITHUB_TOKEN }}
Summary
| Naïve expectation | Actual (unbounded) | With BufferedContext(5) | |
|---|---|---|---|
| 10 turns, 3 agents | 3,000 tokens | ~16,500 tokens | ~12,000 tokens |
| 20 turns, 3 agents | 6,000 tokens | ~63,000 tokens | ~27,000 tokens |
| 30 turns, 3 agents | 9,000 tokens | ~139,500 tokens | ~42,000 tokens |
The default UnboundedChatCompletionContext is correct for short tasks and full-history use cases. It becomes a cost trap in long multi-agent conversations. The fix is one constructor argument — but you have to know to add it.
The broader pattern: every major agent framework defaults to unbounded context because it's the safest correctness choice. Cost is a second-class citizen in the default config. That's the gap this series documents.
Next in the series: CrewAI — the delegation overhead. How hierarchical agent trees multiply your token bill.
tokenscope on npm · wartzar-bee/ci-guardrail · @wartzarbee on dev.to
Top comments (0)