Price the Run, Not the Token

Your spend graph has two meters and the free one only covers half of them. Tokens are the meter you can see on a dashboard. Seconds are the meter your deadline actually reads. Cap the first and ignore the second, and you will ship late while the chart looks perfectly healthy.

This is the failure mode that shows up whenever free capacity is in the mix. Free tokens make attempts cheap, so people stop counting attempts. Attempts are not free in time.

Every attempt re-buys the context

A retry is not a replay from the last good word. In most agent loops you append the failure or the validator output and send the whole thing again. Attempt one costs your context. Attempt two costs your context plus the first error. Attempt three costs context plus two errors, and so on.

The token cost of a run therefore grows faster than the number of attempts, and the wall-clock cost grows with queue wait you do not control. Two retries on a shared free tier might be ninety seconds of generation and four minutes of waiting. That is fine for a nightly job. It is fatal for anything a person is staring at.

Record both meters on every attempt

The cheapest fix is a ledger. One line per attempt, written before you decide anything about the result. No SDK assumptions — if your transport exposes token counts, log them; if it does not, log characters and label the field as an estimate.

# runledger.py — one JSONL line per model attempt.
import json, time, pathlib, hashlib

LEDGER = pathlib.Path("runs/ledger.jsonl")
LEDGER.parent.mkdir(exist_ok=True)

def attempt(task_id, prompt, client, accept, **gen_kw):
    rec = {
        "task": task_id,
        "prompt_sha": hashlib.sha256(prompt.encode()).hexdigest()[:12],
        "prompt_chars": len(prompt),
    }
    started = time.monotonic()
    resp = client.generate(prompt, **gen_kw)      # your transport, free or paid
    gen_s = time.monotonic() - started
    ok = accept(resp)
    rec.update(
        gen_s=round(gen_s, 2),
        wall_s=round(time.monotonic() - started, 2),
        prompt_tokens=getattr(resp.usage, "prompt_tokens", None),
        completion_tokens=getattr(resp.usage, "completion_tokens", None),
        accepted=bool(ok),
    )
    with LEDGER.open("a") as fh:
        fh.write(json.dumps(rec) + "\n")
    return resp, ok

Note what is not in that record: the model output. You are logging cost, not content. Keep the outputs in your own store so the ledger stays small enough to read.

Let the cap fall out of the arithmetic

Most teams pick a retry cap by feel — three, five, whatever the last person typed. You can derive it instead. If each failure appends roughly e tokens to an initial context of c, the cumulative token cost of n attempts is n*c + e*n*(n-1)/2. Invert that against your ceiling.

def attempt_cap(context_tokens, error_tokens, ceiling_tokens):
    """How many attempts fit under the ceiling, including re-sends."""
    spent = 0
    n = 0
    while True:
        spent += context_tokens + n * error_tokens   # attempt n+1 re-sends n errors
        if spent > ceiling_tokens:
            return n
        n += 1

print(attempt_cap(context_tokens=8000, error_tokens=400, ceiling_tokens=120000))

Run that with your real context size and the answer is usually smaller than the number you were using. That is the whole point. The cap is a budget decision, not a personality trait.

Aggregate by accepted task, not by call

A per-call dashboard flatters itself: every call succeeded, therefore the system is efficient. Group by task and charge the failed attempts to the task, because that is where the money went.

import json, collections

runs = collections.defaultdict(lambda: {"tok": 0, "s": 0.0, "n": 0, "ok": False})
for line in open("runs/ledger.jsonl"):
    r = json.loads(line)
    g = runs[r["task"]]
    g["tok"] += (r["prompt_tokens"] or 0) + (r["completion_tokens"] or 0)
    g["s"] += r["wall_s"]
    g["n"] += 1
    g["ok"] = g["ok"] or r["accepted"]

done = [g for g in runs.values() if g["ok"]]
print("accepted:", len(done), "of", len(runs))
print("tokens/accepted:", sum(g["tok"] for g in done) // max(len(done), 1))
print("seconds/accepted:", round(sum(g["s"] for g in done) / max(len(done), 1), 1))
print("attempts/accepted:", round(sum(g["n"] for g in done) / max(len(done), 1), 2))

The third number is the one that tells you whether to raise your acceptance test or lower your cap. If attempts per accepted task drifts above two, your prompts are guessing, not working, and no amount of free capacity will fix a guessing loop.

When free capacity is the wrong bet

Free capacity is a fine trade when you are spending queue time instead of money. It is a bad trade the moment the queue time lands on someone else's critical path.

Job shape	Free capacity	Committed capacity
Someone is watching a spinner	rarely worth it	yes
Overnight batch, due by morning	yes, with a derived cap	only if the deadline is contractual
Long context with deep retries	only if the cap is enforced	yes, because each retry re-sends everything
Burst of one-off jobs	yes, accept the jitter	yes, if jitter breaks a downstream SLA

Read the second column carefully. It is not a downgrade. Trading seconds for tokens is a legitimate engineering choice — as long as you know which currency you spent and you wrote it down.

Limits of this method

This ledger measures only what your transport exposes. If token counts are missing, your totals are estimates and should be labelled that way. Queue wait varies by hour of day, so a single afternoon sample will mislead you; collect for at least a week before you set a cap in production. The numbers are yours and they do not transfer — another team's tokens-per-task tells you nothing about your prompts.

Do not use this approach if you have a hard external deadline, a sub-second interactive surface, or a compliance rule that keeps prompts inside your own network. And if your acceptance test is "looks fine to me," skip the ledger entirely; without a real test, accepted is a constant and the third metric becomes noise.

Where the free tier actually fits

There is a project worth knowing about here: MonkeyCode is an open-source coding agent project, and its operator states that new accounts receive a 10,000,000-token allowance on its free models along with a free server option. Treat that as operator-supplied and check the current project page before you plan a budget around it, because allowances change.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The honest way to use it is as the measurement environment, not the production answer. Point the ledger at free models, run your own task set for a week, and you get real tokens-per-accepted-task and seconds-per-accepted-task numbers without paying for the observation. The free server option means the collector and a cron for the aggregator can live somewhere that is not your laptop.

Then read your own table. If seconds per accepted task is fine for your workload, keep spending the cheap currency. If it is not, you now have a number to argue with instead of a feeling. If you want to try it, the free models and the free server are a low-cost place to start — and if you self-host the ledger instead, that works too, because the point was never the provider. The point was pricing the run.