A 10M Token Budget Is a Constraint. Measure It First.

Every AI vendor sells tokens now, and every free tier is an allowance. Ten million tokens sounds generous, but it is a constraint, not a gift. A popular argument this week says constraints make better engineers. I mostly agree, with one condition. You must measure where the limit bites. Without measurement, the constraint just frustrates you.

MonkeyCode is an open-source project built around free access. It offers free model access and a free server option. Disclosure: This article was prepared as part of MonkeyCode's product outreach. I did not benchmark their hardware or quotas. I built a probe that measures your real token burn. The probe works against any OpenAI-compatible endpoint. MonkeyCode's free tier is a reasonable place to point it.

A free tier breaks in three predictable places. Context bloat. Retry loops. Hidden latency. Context bloat is the quiet killer. Every turn resends the same files into the context window. A 40,000-token context eats four percent of a 10M budget per run. Retry loops double the damage. A failed request still bills the input tokens. Latency makes you impatient, and impatience causes retries. Retries cost tokens twice.

The fix is a budget probe. It sends one task and reads the usage field from the response. The usage field reports prompt tokens and completion tokens. No guessing. No vendor dashboards. Just arithmetic.

#!/usr/bin/env python3
"""budget_probe.py — measure real token cost of one task."""
import argparse
import json
import time
import urllib.request


def probe(endpoint, api_key, model, system, user, max_tokens=512):
    payload = {
        "model": model,
        "messages": [
            {"role": "system", "content": system},
            {"role": "user", "content": user},
        ],
        "max_tokens": max_tokens,
        "temperature": 0,
    }
    request = urllib.request.Request(
        endpoint,
        data=json.dumps(payload).encode("utf-8"),
        headers={
            "Authorization": f"Bearer {api_key}",
            "Content-Type": "application/json",
        },
    )
    start = time.monotonic()
    with urllib.request.urlopen(request, timeout=120) as response:
        data = json.load(response)
    elapsed = time.monotonic() - start
    usage = data.get("usage", {})
    total = usage.get("total_tokens", 0)
    return {
        "latency_s": round(elapsed, 2),
        "prompt_tokens": usage.get("prompt_tokens", 0),
        "completion_tokens": usage.get("completion_tokens", 0),
        "total_tokens": total,
        "runs_per_10m_budget": 10_000_000 // total if total else 0,
        "output_preview": data["choices"][0]["message"]["content"][:200],
    }


if __name__ == "__main__":
    parser = argparse.ArgumentParser()
    parser.add_argument("--endpoint", required=True)
    parser.add_argument("--api-key", required=True)
    parser.add_argument("--model", required=True)
    parser.add_argument("--task", required=True)
    args = parser.parse_args()

    system = "You are a staff engineer. Answer concisely. No filler."
    result = probe(args.endpoint, args.api_key, args.model, system, args.task)
    print(json.dumps(result, indent=2))

The script does three things. It sends a fixed task to the endpoint. It measures end-to-end latency. It divides the 10M budget by the total tokens. The output tells you how many runs you can afford. Run it three times per task and take the median. Medians survive slow-network noise.

Save the script as budget_probe.py. Then run three probe tasks. A code review. A test generation. A refactor. Each task uses the same system prompt. Each task reports its real cost.

python3 budget_probe.py \
  --endpoint https://your-endpoint/v1/chat/completions \
  --api-key "$API_KEY" \
  --model your-model \
  --task "Review this diff for race conditions: <paste diff>"

python3 budget_probe.py \
  --endpoint https://your-endpoint/v1/chat/completions \
  --api-key "$API_KEY" \
  --model your-model \
  --task "Write pytest cases for this function: <paste function>"

python3 budget_probe.py \
  --endpoint https://your-endpoint/v1/chat/completions \
  --api-key "$API_KEY" \
  --model your-model \
  --task "Refactor this 500-line module into smaller functions: <paste module>"

Here are realistic planning numbers, not vendor benchmarks. Your context length and output verbosity will shift them. Treat them as a starting point for your own table.

Task	Input tokens	Output tokens	Runs per 10M
Review a 200-line diff	6,000	800	~1,470
Generate tests for one function	2,500	900	~2,940
Refactor a 500-line file	18,000	2,500	~487
Explain a legacy module	40,000	1,200	~242

These numbers assume a single turn. Agent loops multiply them. A three-turn refactor pays the input cost three times. Your budget disappears in dozens of runs, not thousands. That is the first hard lesson.

The free server changes the second constraint. Compute stops costing money and starts costing patience. Test concurrency the same way you test tokens. Run the probe with one request, then two, then four. Measure latency at each step.

# concurrency_probe.py — run N probes in parallel and print latencies
import concurrent.futures
import os
import subprocess
import sys

n = int(sys.argv[1])
cmd = [
    "python3", "budget_probe.py",
    "--endpoint", "https://your-endpoint/v1/chat/completions",
    "--api-key", os.environ["API_KEY"],
    "--model", "your-model",
    "--task", "Explain this function: <paste function>",
]

def run(_):
    out = subprocess.check_output(cmd, text=True)
    return next(line.strip() for line in out.splitlines() if "latency_s" in line)

with concurrent.futures.ThreadPoolExecutor(max_workers=n) as pool:
    for line in pool.map(run, range(n)):
        print(line)

python3 concurrency_probe.py 1
python3 concurrency_probe.py 2
python3 concurrency_probe.py 4

A free server usually serializes requests. Your jobs queue, and queueing is fine for batch work. It is fatal for interactive sessions.

Here is the debugging workflow when a run fails. Check the HTTP status first. A 429 means you hit a rate limit. Wait and retry with exponential backoff. A timeout means the server is busy, not broken. Reduce the task size and try again. A truncated output means you hit the max output token limit. Raise max_tokens or split the task. Each failure has a signature. Learn the signatures before you blame the model.

The free tier holds for three workloads. Personal automation. CI smoke tests. Weekend projects. It breaks for three others. Production traffic. Long agent loops. Strict latency requirements. The dividing line is not model quality. It is budget and patience.

Do not use this approach for customer-facing services. Do not use it for agents that re-read a whole repository. Do not use it when a failed request costs real money. Use it when you can afford to wait and retry. That is the honest boundary of any free tier.

A free tier is a budget with a deadline. Treat it like one. Measure the burn rate before you trust the output. The probe takes ten minutes to run. Point it at MonkeyCode's free model access and free server. See your real numbers. Then decide if the constraint helps you.