I woke up last Tuesday to a $14 OpenAI bill. For a single night. I'd left an AI agent running — a background task that was supposed to summarize some docs and file GitHub issues. Instead, it got stuck in a retry loop, burning through GPT-4 tokens for six hours straight.
Sound familiar? If you're building with AI agents, autonomous workflows, or even just long-running LLM chains, unmonitored token consumption is the new forgotten while(true) loop.
The Problem Nobody Talks About
Everyone's excited about agentic AI. Give your agent tools, let it reason, let it act. But here's what the tutorials skip: agents make decisions, and decisions cost tokens. Every retry, every chain-of-thought step, every tool call with a fat context window — that's money evaporating.
The worst part? You won't notice until the invoice hits. Most LLM dashboards update with a delay. By the time you see the spike, the damage is done.
What I Changed
After that $14 wake-up call, I built three guardrails into every agent workflow:
1. Set Hard Token Budgets Per Task
Before any agent runs, I define a ceiling. "This summarization job gets 50K tokens max." If it hits the limit, it stops and logs what happened instead of retrying forever.
# Simple budget guard
if total_tokens > BUDGET_LIMIT:
logger.warning(f"Token budget exceeded: {total_tokens}/{BUDGET_LIMIT}")
return partial_result
2. Monitor Token Burn in Real Time
This was the game-changer. I started using TokenBar — it sits in my Mac menu bar and shows live token counts across all my API providers. I can glance up and see if something's burning hotter than expected while it's happening, not 24 hours later on a billing page.
The real value isn't the total — it's the rate. If I see tokens climbing faster than expected, I know something's wrong before it gets expensive.
3. Log Per-Step Costs, Not Just Totals
Every agent step now logs its own token count. When I review runs, I can see exactly which step went off the rails. Usually it's one of these:
- Context stuffing: Passing full documents when a summary would do
- Retry storms: Failed tool calls that trigger increasingly long error-handling chains
- Unnecessary reasoning: Using GPT-4 for tasks that GPT-3.5 handles fine
The Bigger Lesson
AI agents are powerful, but they're not free. Treating token usage like you treat CPU or memory — as a resource to monitor and budget — is the difference between a useful agent and an expensive mistake.
The developers who'll thrive in the agentic era aren't just the ones who build the cleverest chains. They're the ones who know exactly what those chains cost and can spot waste before it compounds.
If you're running any kind of autonomous AI workflow, start tracking tokens today. Your future self (and your wallet) will thank you.
What's your worst runaway agent story? I'd love to hear how others handle token budgets — drop a comment below.
Top comments (0)