Free Tokens Are a Deadline

A solo founder wants to ship a changelog triage tool before Monday. The scenario below is illustrative, not a case study. The tool reads release notes, groups them, and posts a digest to a webhook. The hosting budget is zero dollars.

Free model access looks like an infinite runway. It is not. It is a deadline with an unknown date.

That single reframing changes the whole design. An allowance is a countdown, not a wallet.

Free access has no calendar

An agent that retries a failed tool call three times bills the full prompt three times. Loops are the expensive part, not long answers. A ten-million-token allowance is generous for humans and tight for agents.

The operator of MonkeyCode states two things: a free model allowance of that size, and a free server option. Those are operator-supplied claims, and they will change. Read the project's own page before designing a business around either one.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The practical takeaway is narrow. Treat the allowance as a finite resource with a meter, and treat the free server as a staging tier that can hold real traffic for a while. Neither one is a promise of permanence.

The recent wave of model-generated side projects makes this worse. More generation means more retries, more context, and more forgotten background jobs at 3 a.m.

Write the ledger before the first request

The cheapest guard is a file that only grows. Every response carries a usage object. Append it, then read it back before the next call.

# ledger.py — append-only usage ledger with a hard stop.
# The cap is self-imposed. Keep headroom under any real allowance.
import json, os, time

LEDGER = os.environ.get("MC_LEDGER", "usage.jsonl")
CAP = int(os.environ.get("MC_TOKEN_CAP", "8000000"))


def spent() -> int:
    if not os.path.exists(LEDGER):
        return 0
    total = 0
    with open(LEDGER) as fh:
        for line in fh:
            total += json.loads(line)["total_tokens"]
    return total


def gate(estimated: int) -> None:
    if spent() + estimated > CAP:
        raise RuntimeError("allowance reached: degrade or stop")


def record(model: str, usage: dict) -> None:
    with open(LEDGER, "a") as fh:
        fh.write(json.dumps({
            "ts": time.time(),
            "model": model,
            "prompt_tokens": usage["prompt_tokens"],
            "completion_tokens": usage["completion_tokens"],
            "total_tokens": usage["total_tokens"],
        }) + "\n")

The call site stays boring. The example below assumes an OpenAI-compatible request shape. Check the current docs for the base URL and the live model list before copying it.

import os, requests
from ledger import gate, record

BASE = os.environ["MC_BASE_URL"]
MODEL = os.environ["MC_MODEL_ID"]
KEY = os.environ["MC_API_KEY"]


def ask(prompt: str, est: int = 800) -> str:
    gate(est)
    r = requests.post(
        f"{BASE}/chat/completions",
        headers={"Authorization": f"Bearer {KEY}"},
        json={"model": MODEL, "messages": [{"role": "user", "content": prompt}]},
        timeout=60,
    )
    r.raise_for_status()
    data = r.json()
    record(MODEL, data["usage"])
    return data["choices"][0]["message"]["content"]

A weekly glance at the ledger is enough reporting. Ten lines of standard library beat a dashboard nobody opens.

python - <<'PY'
import json
tot = {}
for line in open("usage.jsonl"):
    d = json.loads(line)
    tot[d["model"]] = tot.get(d["model"], 0) + d["total_tokens"]
for k, v in sorted(tot.items(), key=lambda x: -x[1]):
    print(f"{v:>9,}  {k}")
PY

Route work by cost, not by preference

The right question is not which model is best. It is which work can tolerate a slow, imperfect answer. Route accordingly.

Workload	Where it runs	Why
Nightly digest, batch, cached	Free model	Latency is irrelevant; a human re-reads the output
Issue label suggestions	Free model, low confidence flagged	A wrong label costs one click
Customer-facing chat	Paid tier or nothing	A free tier carries no uptime promise
Prompts containing secrets	Nowhere	Prompts leave the machine

Caching is the highest-leverage trick. Hash the input, store the output, and never pay twice for the same release note.

Put it on the free server honestly

A free server is fine for a tool with a forgiving audience. It is not fine for a product that promises five nines. Configure the reverse proxy, then let the process restart itself.

ship.example.com {
    reverse_proxy 127.0.0.1:8000
    encode gzip
}

[Unit]
Description=changelog triage
After=network.target

[Service]
WorkingDirectory=/srv/app
EnvironmentFile=/srv/app/.env
ExecStart=/srv/app/.venv/bin/uvicorn app:api --host 127.0.0.1 --port 8000
Restart=always
RestartSec=3

[Install]
WantedBy=multi-user.target

A health check every five minutes tells the truth about uptime. Log the failures instead of hiding them.

Decide what happens on day one of the shortage

The worst outcome is a silent failure. Build the ladder before the allowance runs out.

Step one: serve cached answers. Step two: switch to a smaller model. Step three: queue the job and tell the user it is delayed. Step four: return a clear 503 with an honest message.

Each step is ten lines of code. Together they turn a cliff into a slope.

Limits and who should skip this

"Free" here means no invoice today, not no cost ever. A free server has no SLA, no autoscaling, and usually one region. Model access without a contract means the endpoint can change under you.

Skip this approach if the product carries customer data under a compliance regime, if latency is part of the promise, or if the business plan assumes the allowance never ends. Solo builders shipping a small, forgiving tool are the right audience. Everyone else should price the paid tier first.

The project's own documentation is the only current source for the allowance and the server option, so start there before the first request.