I Wrote a Token Budget Probe for a Free Model Server. The Numbers Matter.

Free model servers feel like a gift. Then you wire one into a real pipeline and your token balance evaporates silently at 3 AM. I've seen it happen to people who trusted the word "free" more than they trusted their own monitoring. So this time I built a probe before building a product.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode offers a free server and a token allowance, currently described as 10 million tokens. That's a real claim worth testing. But claims are not evidence. Every free tier has a boundary, and the only way to find it is to measure.

My goal wasn't to benchmark raw speed or write a poetry generator. I wanted to answer one question: can I predict how many tokens a workload will burn before I run it? Because if you can't predict that, you can't budget. And without a budget, your "free" model will surprise you exactly when you need reliability.

The reproducible artifact here is a small Python probe. It sends a few prompts of different shapes, reads the usage object, and logs latency and rate-limit headers. You can adapt it to any OpenAI-compatible endpoint. MonkeyCode's free server fits that shape, but the probe works everywhere.

import time
import requests
from dataclasses import dataclass

API_URL = "https://your-monkeycode-endpoint.example/v1/chat/completions"
API_KEY = "replace-me"

@dataclass
class ProbeResult:
    prompt_tokens: int
    completion_tokens: int
    total_tokens: int
    latency: float
    status: int
    error: str | None = None

def probe(prompt, max_tokens=256, temperature=0.2):
    payload = {
        "model": "your-model-name",
        "messages": [{"role": "user", "content": prompt}],
        "max_tokens": max_tokens,
        "temperature": temperature,
    }
    headers = {
        "Authorization": f"Bearer {API_KEY}",
        "Content-Type": "application/json",
    }
    start = time.perf_counter()
    try:
        r = requests.post(API_URL, headers=headers, json=payload, timeout=60)
        latency = time.perf_counter() - start
        if r.status_code != 200:
            return ProbeResult(0, 0, 0, latency, r.status_code, r.text[:200])
        data = r.json()
        usage = data.get("usage", {})
        return ProbeResult(
            usage.get("prompt_tokens", 0),
            usage.get("completion_tokens", 0),
            usage.get("total_tokens", 0),
            latency,
            r.status_code,
        )
    except Exception as e:
        latency = time.perf_counter() - start
        return ProbeResult(0, 0, 0, latency, 0, str(e))

The probe alone is boring. The interesting part is the scenario matrix you run it through. I used three prompts: a short factual question, a medium code-generation task, and a long context dump with 200 lines of throwaway code. Each prompt gets the same max_tokens cap. Then I ran the same set at three different times and recorded every number.

PROMPTS = [
    ("short", "Explain the difference between a process and a thread in 2 sentences."),
    ("medium", "Write a Python function that validates an IPv4 address without using ipaddress."),
    ("long", "Refactor this 200-line data pipeline. " + "x = 1\n" * 200),
]

for name, prompt in PROMPTS:
    result = probe(prompt)
    print(f"{name}: total={result.total_tokens} prompt={result.prompt_tokens} "
          f"completion={result.completion_tokens} latency={result.latency:.2f}s "
          f"http={result.status}")

The first thing you'll notice is that the numbers are never as clean as the README promises. Free servers share infrastructure, so latency and token counts can drift between runs. That's not a lie; it's a characteristic. The more important metric is the completion-to-prompt ratio. If you send a 500-token prompt and ask for 50 tokens of JSON, a healthy response should return a ratio near 0.1. If you see 0.3 or worse, the model is padding its output, and your token budget is silently leaking.

I also captured response headers on a second pass. Some free tiers expose X-RateLimit-Remaining or similar. That header tells you more than any blog post. If it never changes, the limit is probably enforced elsewhere. If it drops after a single long context call, you know exactly where your allowance went.

def probe_with_headers(prompt):
    # same payload as probe(), but return (result, r.headers)
    ...

Now here's the part that surprised me: with a 10 million token allowance, the raw number sounds huge. But a single long-context task with a large max_tokens cap can eat tens of thousands of tokens in one call. Suddenly 10 million is a month of serious internal tooling, not a weekend toy. My probe showed me that the real cost driver wasn't the prompt length; it was the completion length. Models leak tokens when you let them ramble, and free tiers are rambling machines.

The fix is boring but effective: set max_tokens to the minimum you actually need, not the default. Add a token meter to your own app before the request, and log usage after it. Treat the server's reported usage as ground truth, but verify it with your own input counter. If the difference grows over time, the endpoint is either truncating silently or adding invisible system tokens to every message. Both are worth knowing before you ship.

Who should not use this approach? If you're just playing with prompts interactively, building a probe is like setting up a fire alarm in a matchbox. Overkill. But if you're writing an agent, a code-review bot, or a CI backfill service, the probe is your seatbelt. You don't wear it because you expect a crash every minute. You wear it because the one time you forget, the crash is expensive.

I'm not here to tell you MonkeyCode is the best server on the market. I can't verify that. What I can tell you is that a free tier is only useful if you understand its boundaries. The probe gives you a way to map those boundaries in an afternoon. Run it, save the logs, and then decide with data instead of hype.

The next time someone says "it's free," ask them: free per what? Per token, per request, per second, or per heart attack? My probe is my answer to that question. Build yours.