Your Hourly Rate Makes Free Tokens Expensive

Free model calls look like a gift until you convert queue time into payroll. If a person is blocked, the cheap path is often the expensive one. Tokens can be free. Calendars never are.

You already treat the rest of the stack this way. A complimentary CI runner that sits for half an hour is not complimentary. A “zero-cost” staging box that steals an afternoon from two reviewers is a cost center with a friendly label. Inference is the same machine wearing a different sticker.

This note is an ops worksheet, not a vendor bake-off. You will price wait, classify traffic, and decide when free capacity is the wrong bet. The code is a calculator you fill with your numbers. It is not a benchmark I ran, and it does not bless any model, quota, or hardware story.

Payroll is the unit, not tokens

Token invoices are easy to screenshot. Engineering wait is not. That mismatch is how teams keep a free path in production long after it has started taxing the sprint.

Picture overflow parking at a venue. The lot is free. The walk is not. If the show starts in twelve minutes, you pay for the garage next to the door. LLM routing has the same shape. Batch jobs can walk. Interactive work cannot.

Translate the walk into money before you argue about token price. A fully loaded hourly rate of $H and a human blocked for W hours costs H * W. If that number beats what you would have spent on a paid, low-latency call, the free path lost. Quietly. On the calendar.

You do not need a finance system to see it. You need a clock, a log line, and honesty about who is waiting.

Three kinds of traffic, one common mistake

Most pipelines mix three shapes and then pretend they share a budget.

Interactive work has a person in the loop: a chat reply, an IDE hint, a support draft. The meter is attention. A five-second hitch is visible. A forty-second queue is a meeting that never starts.

CI-gated work has a merge, a deploy, or a release manager on the other side. Nobody is staring at a spinner, but the branch is frozen. Delay here multiplies. One slow job holds a reviewer, then a second reviewer, then the release window.

Batch work can sleep. Eval sweeps, offline summarization, prompt experiments, overnight red-team runs. This is the only shape where “free and uneven” is usually a rational bet.

The common mistake is routing all three through the same free overflow lot because the sticker says zero. That is not thrift. That is refusing to look at the invoice you already pay in stand-ups.

A worksheet you can actually run

Do not debate this in the abstract. Time a path, tag who is blocked, and compute a break-even. The script below is a proposed worksheet. Plug in measurements from your traces. Leave paid price at zero if you do not have one yet; the wait column will still tell you whether the free path is eating payroll.

#!/usr/bin/env python3
"""break_even.py — payroll vs paid inference. Worksheet, not a benchmark."""
from __future__ import annotations

import argparse
import json
from dataclasses import dataclass


@dataclass
class Path:
    name: str
    kind: str  # interactive | ci_gate | batch
    calls_per_day: float
    tokens_per_call: float
    free_p95_s: float
    paid_p95_s: float
    humans_blocked: float
    hourly_rate_usd: float
    paid_usd_per_1k: float

    def wait_hours_per_day(self) -> float:
        extra = max(0.0, self.free_p95_s - self.paid_p95_s)
        return (self.calls_per_day * extra * self.humans_blocked) / 3600.0

    def payroll_usd(self) -> float:
        return self.wait_hours_per_day() * self.hourly_rate_usd

    def paid_usd(self) -> float:
        return (self.calls_per_day * self.tokens_per_call / 1000.0) * self.paid_usd_per_1k

    def free_is_wrong(self) -> bool:
        if self.kind == "batch" and self.humans_blocked <= 0:
            return False
        if self.kind == "interactive" and self.free_p95_s > 8:
            return True
        if self.kind == "ci_gate" and self.free_p95_s > 30:
            return True
        return self.payroll_usd() > self.paid_usd() and self.paid_usd() > 0


def load_paths(raw: str) -> list[Path]:
    data = json.loads(raw)
    return [Path(**row) for row in data]


def main() -> None:
    p = argparse.ArgumentParser()
    p.add_argument("--config", required=True, help="JSON array of Path objects")
    args = p.parse_args()
    paths = load_paths(open(args.config, encoding="utf-8").read())
    print(f"{'path':22} {'kind':12} {'payroll/day':>12} {'paid/day':>10} {'verdict'}")
    for path in paths:
        verdict = "PAY" if path.free_is_wrong() else "FREE_OK"
        print(
            f"{path.name:22} {path.kind:12} "
            f"{path.payroll_usd():12.2f} {path.paid_usd():10.2f} {verdict}"
        )


if __name__ == "__main__":
    main()

A starter config looks like this. Replace every number. Especially hourly_rate_usd and the two p95 fields. If you leave fiction in those slots, the script will launder it into a confident-looking table.

[
  {
    "name": "pr-bot-review",
    "kind": "ci_gate",
    "calls_per_day": 80,
    "tokens_per_call": 2500,
    "free_p95_s": 47,
    "paid_p95_s": 4,
    "humans_blocked": 1.5,
    "hourly_rate_usd": 95,
    "paid_usd_per_1k": 0.003
  },
  {
    "name": "ide-inline-complete",
    "kind": "interactive",
    "calls_per_day": 400,
    "tokens_per_call": 800,
    "free_p95_s": 12,
    "paid_p95_s": 1.2,
    "humans_blocked": 1,
    "hourly_rate_usd": 95,
    "paid_usd_per_1k": 0.003
  },
  {
    "name": "nightly-eval-sweep",
    "kind": "batch",
    "calls_per_day": 2000,
    "tokens_per_call": 1500,
    "free_p95_s": 90,
    "paid_p95_s": 6,
    "humans_blocked": 0,
    "hourly_rate_usd": 95,
    "paid_usd_per_1k": 0.003
  }
]

Run it as a boring command, not a ritual.

python3 break_even.py --config paths.json

You still need real p95s. A proposed timing harness, pointed at whatever endpoint you already use, is enough to stop guessing. Twenty sequential calls will not give you a journal-grade distribution. They will tell you if you are in seconds or in coffee-refill territory.

ENDPOINT="${LLM_URL:?set LLM_URL}"
: > /tmp/latencies.txt
for i in $(seq 1 20); do
  /usr/bin/time -f "%e" -o /tmp/latencies.txt -a \
    curl -sS -o /dev/null -w "" --max-time 120 "$ENDPOINT" || echo 120 >> /tmp/latencies.txt
done
sort -n /tmp/latencies.txt | awk 'END {print "n="NR}
  {a[NR]=$1}
  END {
    p50=a[int(NR*0.50)]; p95=a[int(NR*0.95)];
    print "p50=" p50, "p95=" p95
  }'

If you cannot name humans_blocked for a path, it is batch or you have not watched the work. Watch it for one afternoon. Count the people who cannot proceed until the call returns. That integer is more important than the token price you were arguing about in Slack.

How to read a PAY verdict

PAY does not mean “the free tier is bad.” It means this path should not live there. Keep the free route for the nightly sweep. Move the PR bot. Move the inline complete. The interesting decision is almost never all-or-nothing.

A FREE_OK on batch is not a moral victory either. Batch still fails. It just fails while you sleep. If tomorrow’s launch depends on last night’s sweep finishing before 9:00, that batch path quietly became CI-gated. Change the kind. Re-run the worksheet. The classification is the product.

Thresholds in the script are proposals: eight seconds for interactive, thirty for a gate. Your product may be stricter. A trading UI and a weekly digest do not share a nervous system. Edit the function. Do not treat my constants as policy.

Where a free lab still belongs

You still need somewhere to measure without parking the measurement on the production meter. That is the remaining honest use of free capacity: a scratch lane for the timing harness, the worksheet, and the prompt junk you should never send through a customer path.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode currently offers free model access and a free server option. Use those as a lab if they fit the workflow above. Do not use them as an SLA. The worksheet does not care which lobby you stood in. It cares whether a human was standing with you.

If you try that lab, keep the same discipline you would use anywhere else. Environment variables for the URL. No secrets in the gist. No assumption that today’s free lane will be tomorrow’s. Free is a snapshot of capacity, not a contract with your roadmap.

Limitations, said plainly

This method prices wait, not quality. A paid call that hallucinates faster is not a win. The script also ignores retries, cache hits, and the engineering hours you will spend maintaining two routes. Those are real costs. I left them out so the first version stays something you will actually run.

It cannot see quota cliffs, preemption, or a sudden policy change on a free lane. If your “p95” is a luck sample from a quiet Tuesday, you will under-price risk. Sample on a busy day. Sample when your region is awake. If you cannot sample, you do not have a p95. You have a hope.

It also assumes you have a paid alternative with a known price per thousand tokens. If you do not, the payroll column is still useful as a “how much are we donating to the queue” number. It is not a break-even. Do not fake a paid price to force a verdict.

Currency is another lie people sneak in. $95/hour is a stand-in. Use your fully loaded cost, not the salary number from a job post. If you do not know that number, ask whoever signs the contractor invoices. Guessing low makes free look smarter than it is.

Who should not take this bet the other way

If you are a solo hobbyist with no payroll and no customer waiting, stop. Free capacity is the correct bet. Go build. The overflow lot is for you.

If every call is already reserved capacity you pay for whether you use it or not, this worksheet is noise. You are deciding utilization, not routing.

If the path is safety-critical, regulated, or cannot tolerate a silent model swap, do not put it on a free lane in order to “save” tokens. That is not cost ops. That is hiding an availability decision inside a budget conversation.

And if your team will not log latency, this article will not save you. The arithmetic is the easy part. The hard part is admitting that the stand-up delay is the invoice.

Price the wait. Keep free capacity for work that can sleep. Move anything with a human or a release gate off the overflow lot the moment the worksheet says PAY. That is the whole policy. The rest is logging.

If you want a throwaway lane for the timing harness, MonkeyCode’s free model access and free server option are one place to run the same script. Treat the output as a lab notebook, then route production like you mean it.