Token quotas are marketing numbers until you measure your burn rate, and most developers never take that second step. I have spent the last few weeks building reproducible harnesses for free coding models, and the number that matters most is never printed in the README. It is the tokens consumed per completed task, measured on your own repository, with your own prompts and your own failure modes. MonkeyCode is an open source coding assistant whose current offer includes 10 million free tokens and a free server option.
Disclosure: This article was prepared as part of MonkeyCode's product outreach.
That sounds generous until you ask where the budget actually disappears. My position is deliberately one-sided: stop comparing quota sizes and start measuring burn rate per task. A 10M token allowance is either a hundred small tasks or twenty large ones, and you cannot know which until you measure it. This post gives you a 20-minute burn test, a decision table, and the reasons I now trust telemetry over advertising.
The number that tells you nothing
Raw token counts are nearly useless for planning because they hide four variables that only appear during real work:
- Context window pressure. Long conversations silently consume budget before the model writes a single line.
- Tool call overhead. Every function call, file read, and shell command burns tokens that never appear in the diff.
- Retries and self-correction. A model that fails twice spends three times the tokens of a model that succeeds once.
- System prompt weight. A verbose agent prompt can eat ten percent of every request before your task starts.
Vendors quote token totals because a single large number is easy to print and hard to verify, but you can verify it with a small script. The verification is cheaper than you think, and it takes about twenty minutes of your afternoon.
Three metrics I trust instead
I stopped tracking the quota and started tracking three derived numbers that describe actual engineering output:
- Tokens per completed task. Divide total consumption by tasks finished, not requests sent.
- Tool call overhead. The share of the budget spent on tool-calling boilerplate rather than code.
- Cost per kept hunk. Tokens divided by the lines you actually keep after review.
These three numbers turn a vague allowance into a capacity plan you can defend in a team meeting. They also reveal which parts of your workflow are burning the free tier without producing any engineering value.
A 20-minute burn test you can run today
The script below is an illustrative adapter, not a guarantee that any specific CLI flag exists in your tool. The measurement pattern is the point, so adapt it to whatever agent tooling you already use today.
#!/usr/bin/env bash
# burn-test.sh — illustrative; adapt to your own agent CLI
set -euo pipefail
TASK="${1:-Refactor this function to use async/await}"
LOG="burn-$(date +%Y%m%d-%H%M).log"
# Run the agent against a real repo task and capture its usage report
your-agent run "$TASK" --repo ./sample-repo --json > "$LOG"
# Extract the numbers your tool actually reports
python3 - "$LOG" <<'PY'
import json, sys
data = json.load(open(sys.argv[1]))
steps = data.get("steps", [])
total = sum(s.get("tokens", 0) for s in steps)
tool_calls = [s for s in steps if s.get("tool")]
print(f"steps: {len(steps)}")
print(f"total tokens: {total}")
print(f"tool calls: {len(tool_calls)}")
print(f"tokens/tool call: {total / max(len(tool_calls), 1):.0f}")
PY
Run it on a task that takes at least ten minutes, because short tasks hide the overhead that long tasks expose. Record the output, then run the same task twice more and take the median of the three runs. Three runs are enough to see whether your burn rate is stable or chaotic across similar workloads.
Reading the results
| Burn rate per task | What 10M tokens buys you | Verdict |
|---|---|---|
| 50K | ~200 tasks | Comfortable for daily agent use |
| 150K | ~66 tasks | Usable, but watch long conversations |
| 400K+ | ~25 tasks | You are paying for retries and context bloat |
The table is arithmetic, not a product promise, and your real numbers will differ from these examples. That is exactly why you should run the test on your own repository before you plan anything.
Why the free server changes the math
Here is where my opinion hardens into something you can argue with. A token quota you spend from your laptop is a different product from the same quota on a hosted server. Measurement requires a stable environment, and a free server option gives you exactly that for long-running experiments. You can leave an agent alive, collect the burn log, and iterate without babysitting a local process or paying for a cloud VM.
The server turns a one-off experiment into a repeatable measurement loop, which is the whole point of this workflow. I am deliberately not quoting hardware, uptime, or model names here, because those details change and you should read the repository for current terms. What I can say is that the combination removes the two excuses that usually kill burn testing: cost anxiety and environment drift.
Who should not use this approach
This workflow is not for everyone, and I want the limitations to be loud before you adopt it:
- If your work is short interactive edits, a 20-minute burn test is overkill; a simple token counter in your editor is enough.
- If you need guaranteed uptime or strict data isolation, a free tier is not a contract, and you should not build production on it.
- If you cannot tolerate vendor lock-in, keep the harness tool-agnostic so you can point it at any agent that prints a JSON usage report.
The burn test answers a budgeting question, not a correctness question, so run it alongside the smoke tests and gatekeeper checks from my earlier posts. It is a complement to those harnesses, not a replacement for them.
The position, restated
Stop asking how many tokens you get and start asking how many complete tasks you can finish before the meter moves. While the community argues about whether AI assistants make us faster, the burn log settles the argument with numbers. The 10M number is only meaningful when you know your burn rate, and the honest way to know it is to measure it yourself. If you want to run this test against a free 10M-token setup with a free server, the MonkeyCode open source project is a reasonable place to start. The harness above works with any agent that reports its usage in JSON, so your measurement survives even if you switch tools. Measure first, trust the README second, and let your burn log make the final decision.
MonkeyCode provides free models that can run this workflow.
Top comments (0)