Your Free Lane Is Cheap Until Rework Tax Beats the Token Bill

A nightly batch finished at 02:40 instead of 23:10. The token invoice still read zero. Nobody shipped anything extra that night.

That is the trap in free capacity. Price is a line item. Rework is a schedule item.

This post builds a small gate around the second one. You will get a ledger, a formula, thresholds, a failure drill, and a rollback path. All numbers below are illustrative, not measured on your workload.

The number nobody puts on the dashboard

Most teams track tokens in. Very few track repair time out. The gap is where free lanes quietly get expensive.

Four costs hide behind a zero invoice:

• Queue wait, measured as submit-to-start p95.
• Retry inflation, measured as attempts per run.
• Repair minutes, measured per accepted change.
• Rollback minutes, measured per failed batch.

You cannot gate on a cost you never log. So log the run, not the request.

Step 1: Count accepted changes, not requests

A request that returns 200 is not an output. An output that a reviewer rewrites is not a delivery.

Define three terminal states up front:

1. accepted — merged, no edits beyond whitespace.
2. repaired — merged after a human edit or a second pass.
3. rejected — discarded, or rolled back after deploy.

Accepted change is your unit of value. Everything else is overhead you are paying for twice.

Step 2: Write an append-only ledger

SQLite is enough here. You want one file you can query, copy, and delete.

# ledger.py — append-only run ledger, no ORM, no service
import sqlite3, time

DDL = """
CREATE TABLE IF NOT EXISTS runs (
  run_id        TEXT PRIMARY KEY,
  lane          TEXT NOT NULL,      -- free | paid | self_hosted
  job           TEXT NOT NULL,
  submitted_at  REAL NOT NULL,
  started_at    REAL,
  finished_at   REAL,
  attempts      INTEGER DEFAULT 0,
  tokens_out    INTEGER DEFAULT 0,
  status        TEXT,               -- accepted | repaired | rejected | rolled_back
  repair_min   REAL DEFAULT 0,
  rollback_min REAL DEFAULT 0
);
CREATE TABLE IF NOT EXISTS attempts (
  run_id TEXT, attempt INTEGER, http_status INTEGER,
  started_at REAL, ended_at REAL, tokens INTEGER,
  PRIMARY KEY (run_id, attempt)
);
"""

def connect(path="cost.db"):
    con = sqlite3.connect(path)
    con.executescript(DDL)
    return con

def open_run(con, run_id, lane, job):
    con.execute(
        "INSERT OR IGNORE INTO runs (run_id, lane, job, submitted_at) VALUES (?,?,?,?)",
        (run_id, lane, job, time.time()),
    )
    con.commit()

def start_run(con, run_id):
    con.execute("UPDATE runs SET started_at=? WHERE run_id=?", (time.time(), run_id))
    con.commit()

def record_attempt(con, run_id, attempt, http_status, started, ended, tokens):
    con.execute(
        "INSERT OR REPLACE INTO attempts VALUES (?,?,?,?,?,?)",
        (run_id, attempt, http_status, started, ended, tokens),
    )
    con.execute("UPDATE runs SET attempts=? WHERE run_id=?", (attempt, run_id))
    con.commit()

def close_run(con, run_id, status, tokens_out, repair_min=0.0, rollback_min=0.0):
    con.execute(
        """UPDATE runs SET finished_at=?, status=?, tokens_out=?, repair_min=?, rollback_min=?
           WHERE run_id=?""",
        (time.time(), status, tokens_out, repair_min, rollback_min, run_id),
    )
    con.commit()

The lane in this walkthrough can run on MonkeyCode's free model access and its free server option. Disclosure: This article was prepared as part of MonkeyCode's product outreach. Allowance, limits, and terms are operator-stated and can change, so read the current project page before you size anything against them.

One rule keeps this honest: status is written by the reviewer, not the client. If the client self-reports acceptance, your rework rate is fiction.

Step 3: Compute effective cost per accepted change

Money is one term. Time is the other two.

-- effective_cost.sql
WITH base AS (
  SELECT lane,
         COUNT(*)                                        AS runs,
         SUM(CASE WHEN status='accepted' THEN 1 ELSE 0 END) AS accepted,
         SUM(CASE WHEN status IN ('repaired') THEN 1 ELSE 0 END) AS repaired,
         SUM(tokens_out)                                 AS tokens_out,
         SUM(COALESCE(finished_at, submitted_at) - submitted_at)/60.0 AS wall_min,
         SUM(repair_min + rollback_min)                  AS human_min,
         SUM(attempts)                                   AS attempts
  FROM runs GROUP BY lane
)
SELECT lane, runs, accepted, repaired,
       ROUND(1.0*repaired/NULLIF(accepted,0), 3)      AS repair_ratio,
       ROUND(1.0*attempts/NULLIF(runs,0), 2)          AS attempts_per_run,
       ROUND(wall_min/NULLIF(runs,0), 1)              AS wall_min_per_run,
       ROUND((tokens_out/1000.0 * :tok_rate
              + wall_min * :lane_rate
              + human_min * :human_rate) / NULLIF(accepted,0), 2) AS cost_per_accepted
FROM base;

Bind your own rates. A free lane sets :tok_rate and :lane_rate to zero, which is exactly why you keep :human_rate non-zero.

Illustrative shape of the output, not measured:

lane	attempts/run	repair_ratio	cost_per_accepted
free	2.4	0.42	human-dominated
paid	1.1	0.18	mixed

The lesson is not that free loses. It is that free loses on this workload, at this repair ratio, today.

Step 4: Gate admission on the ratio, not the price

Write thresholds with a reason next to each one. A threshold without a rationale is a vibe.

• repair_ratio > 0.25 over two consecutive windows → stop routing new work to that lane.
• attempts_per_run > 1.5 → fix the client before adding jobs.
• queue_age_p95 > 0.4 * deadline_slack → shed load or split the batch.
• cost_per_accepted up 30% week over week → reopen the routing decision.

Deadline slack beats utilisation as a queue signal. Utilisation can sit at 70% and still miss a 03:00 cutoff.

The gate itself is a few lines. Keep it boring and keep it out of the queue worker.

# gate.py — unexecuted sketch, wire to your own ledger query
FREE_MAX_REPAIR = 0.25
FREE_MAX_ATTEMPTS = 1.5

def choose_lane(repair_ratio, attempts_per_run, slack_ratio):
    if slack_ratio > 0.4:
        return "shed", "queue_age over 40% of slack"
    if repair_ratio > FREE_MAX_REPAIR:
        return "paid", "repair ratio above 25%"
    if attempts_per_run > FREE_MAX_ATTEMPTS:
        return "hold", "retry inflation, fix client first"
    return "free", "within all thresholds"

Read LANE_GATE from the environment so you can flip routing without a deploy. LANE_GATE=off must mean off, not "mostly off".

Step 5: Drill the retry storm before it drills you

Free capacity plus a naive retry loop equals a wall-clock bill you never see. Test the cap locally with a test double, not a real provider.

# drill.py — test double; no real model calls
import os, random, time

def call_model(prompt, fail_rate=float(os.getenv("STRESS_FAIL_RATE", "0"))):
    time.sleep(float(os.getenv("STRESS_LATENCY_MS", "0")) / 1000.0)
    if random.random() < fail_rate:
        return {"http_status": 429, "tokens": 0}
    return {"http_status": 200, "tokens": len(prompt.split())}

Run it with a bounded loop and assert the ceiling.

STRESS_FAIL_RATE=0.4 STRESS_LATENCY_MS=800 python drill.py --jobs 20 --max-attempts 3

Expected output: 20 runs, attempts_per_run <= 3, and at least one run marked rejected rather than retried forever. If attempts per run climbs toward the cap, your backoff is too shallow, not your provider too slow.

When free capacity is the wrong bet

Skip this pattern if any of these are true:

• Your deadline is fixed and short, like a market-open report.
• Rework is reviewed by the most expensive engineer you have.
• Output feeds a customer-facing contract with a hard SLA.
• You cannot write a status column because nobody reviews the output.

Free capacity is a great bet for exploratory batches, low-stakes summarisation, and load you can cancel mid-flight.

It is a bad bet wherever repair time silently outranks token price.

Rollback and cleanup

Do these in order, and rehearse once on a dry batch.

1. Set LANE_GATE=off and confirm the worker keeps draining old jobs.
2. Drain the free lane and cancel anything still queued.
3. Re-route in-flight work to the fallback lane at a capped concurrency.
4. Keep cost.db for 30 days, then delete it with the batch workspace.
5. Record the rollback in the ledger as rolled_back, with minutes spent.

Step 5 is the one teams skip. Without it, your next cost review has a hole exactly where the incident happened.

The takeaway

Price told you the lane was free. Repair time told you what it cost.

Log accepted changes. Gate on the ratio. Drill the retry cap. Then decide.

Start the gate on ten jobs, not a thousand. Publish your rework rate, not your token savings.

Comments

[Alex Shev]

Rework tax is the metric teams forget because it is distributed across retries, reviews, support, and lost trust. I’d price the workflow by successful verified outcome, not by model call, and include the human minutes needed to catch or repair a bad result. That makes a more expensive but dependable lane easier to justify.