If the Job Has a Deadline, Spare Capacity Is the Wrong Bet

If the job has a deadline, spare capacity is the wrong bet. Price is not a clock. Free model access and a free server are fine for work you can abandon. They are the wrong default for a deploy freeze, a customer callback, or a CI gate that already owns the calendar.

Standby boarding is the right picture. You might get on. You might not. That is a reasonable deal when you can take the later flight. It is a bad deal when a meeting starts at 4 p.m. whether you are in the building or not. Deadline work is that meeting. Spare tokens are the standby list.

Quiet failures are the expensive ones. You hook a generator onto the last hour before merge because the prompt is ready and the branch looks clean. Then the first completion drags, the fifth drags more, and the minutes you have left belong to the freeze, not to you. You ship a half-finished diff, or you miss the window. Neither outcome is "free."

Keep the lab. Change the routing. Work you can drop can wait on whatever machine happens to be idle. Work with a clock needs a reservation, and sometimes the reservation is simply refusing to start.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode is an open-source project that offers free model access and a free server option. That pair is useful as a lab lane for probes, dry runs, and jobs you are willing to kill. It is not a substitute for reserved capacity when a human, a freeze, or a customer already booked the hour.

Cheap generation makes this split more urgent, not less. When producing a patch is easy, teams try to squeeze one more rewrite into the last slice of a release train. That is how unverified text becomes inventory you must either rubber-stamp or throw away under a clock. Spare capacity will happily help you create that pile. It will not help you review it in time.

Treat the calendar as the budget, then make the pipeline ask permission. The script below is an unexecuted template. Point it at your endpoint, your deadline, and your expected call count. It does not pick a model for you. It only asks whether remaining wall clock can absorb a slow probe times the work you still want to do.

# deadline_preflight.py — example template, not a live benchmark
from __future__ import annotations

import argparse, json, os, sys, time, urllib.error, urllib.request


def epoch_deadline(raw: str) -> float:
    raw = raw.strip()
    if raw.endswith("s"):
        return time.time() + float(raw[:-1])
    if raw.endswith("m"):
        return time.time() + float(raw[:-1]) * 60
    return float(raw)


def probe(url: str, token: str | None, timeout: float) -> float:
    body = json.dumps({
        "messages": [{"role": "user", "content": "Reply with the single word pong."}],
        "max_tokens": 8,
    }).encode()
    headers = {"Content-Type": "application/json"}
    if token:
        headers["Authorization"] = f"Bearer {token}"
    req = urllib.request.Request(url, data=body, headers=headers, method="POST")
    t0 = time.perf_counter()
    with urllib.request.urlopen(req, timeout=timeout) as resp:
        resp.read(256)
    return time.perf_counter() - t0


def main() -> int:
    p = argparse.ArgumentParser(description="Fail closed if spare capacity cannot beat a deadline.")
    p.add_argument("--url", default=os.environ.get("LLM_URL", ""))
    p.add_argument("--deadline", default=os.environ.get("JOB_DEADLINE", ""),
                   help="unix epoch, or relative like 12m / 90s")
    p.add_argument("--calls", type=int, default=int(os.environ.get("EXPECTED_CALLS", "8")))
    p.add_argument("--safety", type=float, default=float(os.environ.get("SAFETY_FACTOR", "3")))
    p.add_argument("--timeout", type=float, default=20.0)
    p.add_argument("--slack", type=float, default=30.0, help="seconds to keep for cancel/upload")
    args = p.parse_args()
    if not args.url or not args.deadline:
        print("deadline_preflight: LLM_URL and JOB_DEADLINE are required", file=sys.stderr)
        return 3

    until = epoch_deadline(args.deadline)
    remaining = until - time.time()
    if remaining <= args.slack:
        print(f"do-not-start: {remaining:.1f}s left before slack", file=sys.stderr)
        return 3

    try:
        latency = probe(args.url, os.environ.get("LLM_TOKEN"), args.timeout)
    except (urllib.error.URLError, TimeoutError, json.JSONDecodeError) as exc:
        print(f"switch-reserved: probe failed ({exc})", file=sys.stderr)
        return 2

    need = latency * args.calls * args.safety + args.slack
    print(json.dumps({
        "probe_s": round(latency, 3),
        "remaining_s": round(remaining, 1),
        "need_s": round(need, 1),
        "calls": args.calls,
    }))
    if need > remaining:
        print("switch-reserved: spare path cannot clear the clock", file=sys.stderr)
        return 2
    return 0


if __name__ == "__main__":
    sys.exit(main())

Exit 0 means the spare path is still plausible. Exit 2 means move the job to a reserved endpoint or skip generation and ship the human-written change. Exit 3 means do not start: the window is already smaller than shutdown slack. Wire that into CI so a late pipeline fails closed instead of launching a twenty-call rewrite with six minutes left.

# example CI gate — unexecuted
export JOB_DEADLINE="12m"
export EXPECTED_CALLS=12
export LLM_URL="$SPARE_ENDPOINT"   # lab / free-capacity URL
python deadline_preflight.py
case $? in
  0) python generate_notes.py --lane spare ;;
  2) python generate_notes.py --lane reserved ;;
  *) echo "skipping generation; clock owns this job"; exit 0 ;;
esac

Notice the last branch exits 0 for the pipeline. Missing a generator is usually cheaper than missing the freeze. If your culture treats "the bot must comment" as mandatory, you will keep burning the only minutes that matter. Change that culture before you tune the probe.

Add a second check for prompt bloat. A 12k-character paste of logs is not context. It is a long occupation of a shared endpoint, and it stretches every later call you still think you can afford. Truncate or refuse before you spend the hour.

# prompt_budget.py — example template
from pathlib import Path
import sys

MAX_CHARS = 6000  # lab default; measure against YOUR deadline, not a slogan
path = Path(sys.argv[1])
text = path.read_text(encoding="utf-8")
if len(text) > MAX_CHARS:
    sys.stderr.write(
        f"refuse: {path} is {len(text)} chars; spare capacity plus a deadline cannot carry this dump\n"
    )
    sys.exit(3)
print(text[:MAX_CHARS], end="")

Run a probe when the office is quiet and again when everyone is merging. Spare capacity changes shape during the day. A morning number is not a promise at 3:55 p.m. If the second probe is several times slower, that is your signal to keep deadline work off that lane, not a puzzle to solve with a longer retry loop.

Who should ignore this gate? Anyone whose job has no clock. Batch summarization overnight, eval sweeps, prompt experiments, and docs you can publish tomorrow belong on spare capacity. Forcing a deadline preflight onto that work just adds ceremony. Also skip this pattern if the payload cannot leave your network. A free server you do not control is the wrong place for customer data, secrets, or anything that would be awkward in a support ticket. The script above sends a tiny ping; it is still a network call. Point it only at endpoints you are allowed to use.

The probe can lie. A cheap first token does not guarantee the twentieth call. Safety factors are guesses. If you need p95 math, you need a reserved path and a real load test, not a gist in a blog post. This workflow is a tripwire. It stops you from starting. It does not make idle hardware behave like a contract.

Split the board in two columns and keep them honest. Column A is abandonable: evals, rewrites you can throw away, probes against free model access, experiments on a free server. Column B is dated: release notes before a freeze, customer-facing summaries, CI comments that block merge. Column B either pays for a clock or it waits until you have one. Mixing the columns is how zero-price capacity becomes a late-night incident.

If you want a lab lane for column A, try MonkeyCode's free model access and free server option on jobs you can drop without apology. Keep column B off that lane. The clock is the cost. The token invoice is just the receipt you notice last.