Free Capacity Fails the Deadline Test, Not the Price Test

Tuesday, 02:40. Nine hundred batch jobs queued. Deadline at 09:00.

At 200 jobs, the free lane looked perfect. Zero spend, clean output.

At 900 jobs, p95 queue age hit 34 minutes. Retry attempts tripled.

Nobody changed the price. The deadline changed.

That is the trap with free capacity. You keep evaluating it on cost. It fails on time.

So stop asking "is it free?" Ask "does it still finish before the deadline?"

This post gives you a preflight check for that second question. It runs in seconds. It exits non-zero when the lane no longer fits.

The price test is easy. The deadline test is not

Free capacity is a shared resource. You get it cheap because you get it later.

That is not a bug. It is the trade. The queue absorbs spikes you did not create.

Your batch does not care about the reason. It cares about deadline slack.

Three numbers decide the lane choice:

1. p95 queue age — how long a job waits before a worker starts it.
2. Retry amplification — total attempts divided by accepted artifacts.
3. Deadline slack — deadline minus projected finish, in minutes.

You can measure the first two from a canary. The third is arithmetic.

Most teams measure none of them. They measure price. Then they miss the date.

Build a job event ledger first

You cannot compute slack without timestamps. Emit one JSON line per lifecycle event.

{"job_id":"j-0042","event":"queued","ts":1762480800,"lane":"free"}
{"job_id":"j-0042","event":"started","ts":1762481100}
{"job_id":"j-0042","event":"retry","attempt":2,"ts":1762481400}
{"job_id":"j-0042","event":"started","ts":1762481460}
{"job_id":"j-0042","event":"finished","status":"ok","ts":1762481820,"tokens_in":1840,"tokens_out":610}

Two rules keep this ledger honest.

• Write the queued event before you submit, not after.
• Log tokens_in and tokens_out on finish, never estimated.

Retries are first-class events. If you hide them, amplification disappears from your data.

The preflight script

Save this as lane_preflight.py. Standard library only. No network calls.

#!/usr/bin/env python3
"""lane_preflight.py - does the free lane still fit the deadline?"""
import argparse, json, math, sys
from statistics import quantiles


def load(path):
    with open(path) as fh:
        return [json.loads(l) for l in fh if l.strip()]


def p95(values):
    if not values:
        return 0.0
    if len(values) < 2:
        return float(values[0])
    return quantiles(values, n=20)[-1]


def roll_up(events):
    jobs = {}
    for e in events:
        j = jobs.setdefault(e["job_id"], {
            "attempts": 0, "tokens": 0, "queued": None,
            "started": [], "finished": None, "status": None,
        })
        kind = e["event"]
        if kind == "queued" and j["queued"] is None:
            j["queued"] = e["ts"]
        elif kind == "started":
            j["attempts"] += 1
            j["started"].append(e["ts"])
        elif kind == "finished":
            j["finished"] = e["ts"]
            j["status"] = e.get("status", "ok")
            j["tokens"] += e.get("tokens_in", 0) + e.get("tokens_out", 0)
    return jobs


def main():
    ap = argparse.ArgumentParser()
    ap.add_argument("events")
    ap.add_argument("--remaining", type=int, required=True)
    ap.add_argument("--concurrency", type=int, default=4)
    ap.add_argument("--deadline", type=int, required=True, help="epoch seconds")
    ap.add_argument("--now", type=int, default=None)
    ap.add_argument("--max-amp", type=float, default=1.5)
    args = ap.parse_args()

    jobs = roll_up(load(args.events))
    done = [j for j in jobs.values() if j["status"] == "ok"]
    if not done:
        sys.exit("no accepted jobs yet: run a canary first")

    queue_age = p95([j["started"][0] - j["queued"] for j in done if j["queued"]])
    duration = p95([j["finished"] - j["started"][-1] for j in done])
    attempts = sum(j["attempts"] for j in jobs.values())
    amp = attempts / len(done)
    tokens_per_artifact = sum(j["tokens"] for j in done) / len(done)

    now = args.now or max(j["finished"] for j in done)
    waves = math.ceil(args.remaining / args.concurrency)
    projected = now + queue_age + waves * duration * amp
    slack_min = (args.deadline - projected) / 60.0

    print(f"accepted jobs          : {len(done)}")
    print(f"p95 queue age (s)      : {queue_age:.0f}")
    print(f"p95 job duration (s)   : {duration:.0f}")
    print(f"retry amplification    : {amp:.2f}x")
    print(f"tokens / artifact      : {tokens_per_artifact:.0f}")
    print(f"projected finish (s)   : {projected:.0f}")
    print(f"deadline slack (min)   : {slack_min:.1f}")

    verdict = 0
    if slack_min < 0:
        print("VERDICT: free lane misses the deadline")
        verdict = 1
    if amp > args.max_amp:
        print(f"VERDICT: amplification {amp:.2f}x exceeds {args.max_amp}x")
        verdict = 1
    sys.exit(verdict)


if __name__ == "__main__":
    main()

Run it before dispatch, not after:

python3 lane_preflight.py canary.jsonl \
  --remaining 860 \
  --concurrency 4 \
  --deadline 1762506000 \
  --max-amp 1.5
echo "exit=$?"

Expected output

Numbers below are synthetic, from a made-up canary. They show shape, not a benchmark.

accepted jobs          : 40
p95 queue age (s)      : 2040
p95 job duration (s)   : 96
retry amplification    : 1.37x
tokens / artifact      : 2410
projected finish (s)   : 1762509922
deadline slack (min)   : -65.4
VERDICT: free lane misses the deadline
exit=1

Note what happened. Amplification stayed under threshold. Price stayed zero. Slack went negative.

The verdict is a time verdict. Budget verdicts miss it entirely.

Turn the numbers into a lane decision

Slack	Amplification	Action
> 90 min	< 1.2x	Keep the free lane, run full batch
0–90 min	< 1.2x	Keep free, cut scope, re-check every 15 min
0–90 min	≥ 1.5x	Split batch: canary on free, rest elsewhere
< 0 min	any	Do not start. Fix inputs or buy time
< 0 min	≥ 2.0x	Stop. The retries are the problem, not the lane

The last row matters most. Retry amplification above 2x is usually a bad prompt or a failing dependency.

Moving that batch to a faster lane just burns money faster.

Where free model access and a free server actually help

Here is where I use it, and where I do not.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

MonkeyCode is an open-source project offering free model access and a free server option. Both are operator-supplied availability claims, so check the current terms yourself before you plan around them.

I use the free server for the boring half of this workflow:

• Run the canary batch and write the JSONL ledger.
• Execute lane_preflight.py on the same host as the events.
• Re-run the check every 15 minutes during a long batch.

That keeps measurement off your paid workers. It also keeps the ledger next to the data.

I never treat the free tier as a latency SLO. It is a capacity source with unknown queue depth.

One more thing: quotas move. I will not quote a token allowance here, because a stale number is worse than no number. Read the current figure on the project page, then plug the real consumption into your ledger.

Do not bet the whole batch on a canary you did not measure

Commit the first 5% of jobs. Read the ledger. Then decide.

If your canary runs 40 jobs, you have 40 samples. That is enough for a p95 that will still be wrong. Treat it as a smoke signal, not a guarantee.

Failure modes this check will not catch

Be honest about coverage. The script measures time. It misses these.

• Disk. Free capacity still writes artifacts. Cap retention before dispatch.
• Lane thrash. A job that flaps between lanes double-counts tokens. Pin the lane after one ambiguous retry.
• Non-idempotent retries. Amplification looks cheap when duplicates silently overwrite output.
• Clock skew. Timestamps from different hosts break queue age. Sync clocks first.

Each one is a separate control. None of them are fixed by a faster lane.

Rollback and cleanup

Prepare the exit before you start the batch.

1. Tag every job with lane=free and a batch_id correlation field.
2. Keep a cancel script that filters on batch_id and stops queued jobs.
3. Snapshot the ledger before any cleanup. It is your postmortem evidence.
4. Re-dispatch the remainder with the same job IDs. Idempotent sinks only.
5. Cap artifact retention at 7 days until the batch is accepted by a human.

If the deadline slips anyway, revert to the paid lane and rerun the preflight with --now set to the real clock. Do not rerun the whole batch blind.

Who should skip this approach

Be honest, most request-path systems should not use free capacity as a first option.

• Hard latency SLOs. Any p99 commitment under a minute. Queue age alone will break it.
• Non-idempotent side effects. Retries will corrupt state before you notice.
• Data residency requirements. You cannot verify the path, so do not send the data.
• Tiny batches. Under 20 jobs, the measurement cost exceeds the savings.

If you are in those groups, keep the ledger and drop the free lane. The tracking is still worth it.

The habit worth keeping

Write the deadline into your config. Compute slack from real events. Let the check block the dispatch.

Free capacity is a good bet when the deadline is loose and retries are rare. It is a bad bet when both are unknown.

If you want to try this loop, the free server option is a reasonable place to run the canary and the preflight script. Measure your own numbers before you trust anyone's, including mine.