Last Tuesday I opened my usage dashboard and saw a number that made me close it again. My ten-million-token allowance had dropped by roughly a fifth in a single day, and nothing in my logs suggested anything had gone wrong. No errors, no crashes, no slow responses, no failed jobs. The pipeline just consumed tokens the way a teenager consumes a family data plan.
I was running a document-summarization batch against MonkeyCode's free model access and the free server option, mostly to see whether a cheap pipeline could survive real workloads. Disclosure: This article was prepared as part of MonkeyCode's product outreach. The allowance sounded generous on day one, and it probably is, if you know where every token goes. I did not.
The symptom: a number that didn't add up
I had one obvious suspect, and it turned out to be innocent. The batch processed about 1,200 support tickets that day, and my rough estimate said that should cost somewhere around 400,000 tokens. The dashboard said the real number was closer to 2.1 million, and that gap was too large for rounding errors.
How do you audit a consumption problem when every individual call looks reasonable? You stop guessing and start measuring.
Step one: instrument every call
The first mistake was treating the usage field as a debugging tool instead of a data source. Every response from the API already contains prompt_tokens, completion_tokens, and total_tokens, and I had been ignoring all three. I wrote a small wrapper that records them into SQLite on every call, along with a caller tag and a timestamp.
import sqlite3
from datetime import datetime, timezone
SCHEMA = """
CREATE TABLE IF NOT EXISTS usage_log (
ts TEXT NOT NULL,
caller TEXT NOT NULL,
model TEXT NOT NULL,
prompt_tokens INTEGER NOT NULL,
completion_tokens INTEGER NOT NULL,
total_tokens INTEGER NOT NULL,
finish_reason TEXT
);
"""
def log_usage(conn, caller, response):
usage = response.get("usage", {})
conn.execute(
"INSERT INTO usage_log VALUES (?, ?, ?, ?, ?, ?, ?)",
(
datetime.now(timezone.utc).isoformat(),
caller,
response.get("model", "unknown"),
usage.get("prompt_tokens", 0),
usage.get("completion_tokens", 0),
usage.get("total_tokens", 0),
response.get("choices", [{}])[0].get("finish_reason"),
),
)
conn.commit()
The wrapper took twenty minutes to write, and it immediately turned a mystery into a spreadsheet. If you are not logging usage on every single call, you are flying blind, and a free tier is the worst place to be blind.
Step two: aggregate by caller and by day
SQLite makes the next step almost too easy. Two queries turned my silent pipeline into a ranked list of suspects.
-- Who ate the most tokens?
SELECT caller, SUM(total_tokens) AS tokens, COUNT(*) AS calls
FROM usage_log
GROUP BY caller
ORDER BY tokens DESC;
-- Daily burn rate
SELECT substr(ts, 1, 10) AS day, SUM(total_tokens) AS tokens
FROM usage_log
GROUP BY day
ORDER BY day;
The first query produced a ranking that embarrassed me. The second produced a projection that scared me: at the current burn rate, the allowance would last about nine days, not the month I had assumed.
Root cause: three leaks I never counted
None of these leaks produced an error, which is exactly why they survived for so long. Here is what the ledger revealed.
A polling loop with a bloated system prompt. A health-check function ran every five minutes and sent an 1,800-token system prompt on every call. That single function consumed about 38% of the weekly total, and its results were almost always identical to the previous check.
Verbose completions with no max_tokens. I never set max_tokens on the summarization endpoint, so the model happily wrote introductions, conclusions, and closing remarks for every ticket. Completion tokens were running about three times higher than my estimate.
Duplicate work from an upstream loop. A list-processing bug sent the same 300 tickets through the summarizer twice. The calls succeeded, the results were identical, and the tokens were gone.
| Leak | Share of weekly tokens | Fix |
|---|---|---|
| Polling loop with 1,800-token system prompt | 38% | Cache results, shrink the prompt |
| Verbose completions (no max_tokens) | 31% | Set max_tokens, tighten instructions |
| Duplicate tickets from upstream loop | 21% | Deduplicate before sending |
| Everything else | 10% | Keep logging |
The fix: a budget guard that fails before the meter does
Measuring the leak was satisfying, but preventing the next one required a guard. I added two layers: a cheap token estimator that runs before every request, and a daily cap that raises an exception instead of silently spending.
class BudgetGuard:
def __init__(self, daily_cap: int):
self.daily_cap = daily_cap
self.used_today = 0
def estimate(self, messages) -> int:
# Rough heuristic: about 4 characters per token for English text.
# The API's usage field is the source of truth; this only blocks
# obviously oversized requests before they are sent.
return sum(len(m.get("content", "")) for m in messages) // 4
def check(self, messages) -> None:
estimated = self.estimate(messages)
if self.used_today + estimated > self.daily_cap:
raise RuntimeError(
f"daily budget exceeded: {self.used_today} + {estimated} > {self.daily_cap}"
)
After every successful call, I add the real total_tokens to used_today, so the guard learns the truth instead of trusting my estimate. The polling loop's system prompt went from 1,800 tokens to 140, and max_tokens went from unset to 256.
What this workflow cannot do
This ledger tells you how many tokens you spent, but it cannot tell you whether you wasted them. A call that produces a wrong answer consumes exactly the same tokens as a call that produces a correct one, and no budget guard will catch that. If your pipeline needs quality guarantees, pair the meter with output validation, ground-truth checks, or human review.
Token estimation is also inherently fuzzy. Different tokenizers count differently, and my heuristic is calibrated for English prose, so treat the estimator as a tripwire rather than an accounting system.
Who should skip this
If your workload is a handful of interactive calls per day, this entire apparatus is overkill. A SQLite table and a budget guard add complexity, and complexity has its own cost. Build the meter when the allowance is a real constraint, when multiple services share one key, or when you cannot explain last week's consumption. If none of those apply, just set max_tokens and move on.
The dashboard told me the allowance was shrinking, but it never told me why. The meter did, and the fix took an afternoon. The next time your quota evaporates, log first and blame the model later.
If you want to poke at your own consumption patterns, MonkeyCode's free model access and free server give you a cheap playground, and the logging pattern above works on any OpenAI-compatible endpoint. I wrote the whole thing in about a hundred lines of Python, and the spreadsheet it produced was worth more than the code.
Top comments (0)