Let Warmup Cast the Only Vote

A Tuesday standup ended early when an agent-written HTTP handler landed in a disposable branch and looked complete. Unit tests were green, names were tidy, and the generated comments read like a postmortem that had already decided on success. A teammate then ran a twenty-second warmup against the listening port while the author still narrated the design. Concurrent clients turned the p95 figure into wet paper, and the room learned that polish is not a clock.

That scene now repeats wherever coding assistants are scored mainly by screenshots of passing tests. A ninety-minute spike can carry only one hypothesis, and the hypothesis has to be falsifiable by wall time. Ordinary functional assertions praise the narrative the model already preferred to emit during the session. A short warmup praises nothing except the process that remains upright after overlapping requests arrive.

Treat the spike as a fire drill rather than a dress rehearsal for a launch weekend. Drills help only when the alarm is specific, the exit path is timed, and the building is not a slide deck. Running the drill on a free remote server keeps laptop thermal noise from impersonating a latency budget. Browser tabs, indexing jobs, and Docker Desktop contention all counterfeit delay with a straight face.

The hypothesis for this worked example is deliberately narrow and slightly unkind to optimistic demos. After ninety minutes the handler must hold p95 under one hundred fifty milliseconds for thirty concurrent clients. The window is a thirty-second warmup, and any miss kills the spike without further appeal. Error rate must stay at zero during that same window, because retries would hide the collapse.

No second hypothesis is allowed to ride along inside a README or an architecture footnote. Before any assistant is invited to type, the contract is written as JSON that a script can parse. The file becomes the judge, and extra commentary from the model is demoted to a private diary. Teams that skip this file usually discover at minute eighty-eight that they have been debating taste.

{
  "hypothesis_id": "warmup-p95-v1",
  "method": "GET",
  "path": "/inventory/next",
  "concurrency": 30,
  "duration_seconds": 30,
  "p95_ms_max": 150,
  "error_rate_max": 0.0,
  "spike_budget_seconds": 5400,
  "verdict": "ship_or_kill"
}

A minimal handler is enough to show the shape of the drill without dragging in a framework weather system. The following Python example is a labeled proposal rather than a measured service in any environment. It uses only the standard library so dependency noise cannot steal minutes from the ninety-minute envelope. Readers should treat the twelve-millisecond sleep as a confessed stand-in for a cheap datastore roundtrip.

# proposal: spike_handler.py -- not a production service
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
import json
import time

STOCK = {"sku": "SKU-9", "qty": 40}

class Handler(BaseHTTPRequestHandler):
    def log_message(self, fmt, *args):
        return

    def do_GET(self):
        if self.path != "/inventory/next":
            self.send_error(404)
            return
        time.sleep(0.012)  # stand-in for a cheap datastore roundtrip
        body = json.dumps(STOCK).encode("utf-8")
        self.send_response(200)
        self.send_header("Content-Type", "application/json")
        self.send_header("Content-Length", str(len(body)))
        self.end_headers()
        self.wfile.write(body)

if __name__ == "__main__":
    ThreadingHTTPServer(("0.0.0.0", 8080), Handler).serve_forever()

The sleep is a confession rather than an optimization that anyone should copy into production. Real stores, locks, and JSON codecs will replace it, and the warmup will notice the substitution. The spike is not an invitation to admire a nap that looks fast on an idle core. The spike exists to keep that nap from stretching past a hundred milliseconds under overlap.

The warmup client should stay equally boring so dashboards do not smuggle in a second hypothesis. Percentile charts, trace backends, and flame graphs are useful later and poisonous during minute twelve. The script records status codes and latency samples, then prints one boolean that matches the contract. Exit status one means kill, and exit status zero means the handler may be discussed as a candidate.

# proposal: warmup_vote.py -- prints PASS or KILL
import json
import sys
import time
import urllib.error
import urllib.request
from concurrent.futures import ThreadPoolExecutor, as_completed

def load_contract(path):
    with open(path, encoding="utf-8") as handle:
        return json.load(handle)

def once(url, timeout):
    started = time.perf_counter()
    try:
        with urllib.request.urlopen(url, timeout=timeout) as response:
            response.read()
            status = response.status
    except (urllib.error.URLError, TimeoutError):
        status = 599
    return status, (time.perf_counter() - started) * 1000.0

def percentile(values, p):
    if not values:
        return 9999.0
    ordered = sorted(values)
    idx = int(round((p / 100.0) * (len(ordered) - 1)))
    idx = max(0, min(len(ordered) - 1, idx))
    return ordered[idx]

def main(contract_path, base_url):
    spec = load_contract(contract_path)
    url = base_url.rstrip("/") + spec["path"]
    deadline = time.perf_counter() + spec["duration_seconds"]
    samples = []
    timeout = max(1.0, spec["p95_ms_max"] / 1000.0 * 4)

    def worker():
        rows = []
        while time.perf_counter() < deadline:
            rows.append(once(url, timeout))
        return rows

    with ThreadPoolExecutor(max_workers=spec["concurrency"]) as pool:
        futs = [pool.submit(worker) for _ in range(spec["concurrency"])]
        for fut in as_completed(futs):
            samples.extend(fut.result())

    latencies = [ms for _, ms in samples]
    errors = sum(1 for status, _ in samples if status >= 400)
    p95 = percentile(latencies, 95)
    err_rate = errors / max(1, len(samples))
    ok = p95 <= spec["p95_ms_max"] and err_rate <= spec["error_rate_max"]
    print(json.dumps({
        "n": len(samples),
        "p95_ms": round(p95, 2),
        "error_rate": round(err_rate, 4),
        "verdict": "PASS" if ok else "KILL",
    }))
    sys.exit(0 if ok else 1)

if __name__ == "__main__":
    main(sys.argv[1], sys.argv[2])

A shell envelope keeps the human honest about the ninety minutes when the narrative begs for another pass. When the budget elapses the spike is over even if the story still wants one more refactor. GNU timeout is the adult in the room, because moods do not close branches on their own. The caller should wrap the whole group so leftover servers cannot leak into the next experiment.

#!/usr/bin/env bash
# proposal: run_spike.sh
# invoke as: timeout --signal=TERM 5400 ./run_spike.sh
set -euo pipefail
CONTRACT="${1:-spike_contract.json}"
BASE_URL="${2:-http://127.0.0.1:8080}"
python spike_handler.py &
SERVER_PID=$!
cleanup() { kill "$SERVER_PID" 2>/dev/null || true; }
trap cleanup EXIT
sleep 1
python warmup_vote.py "$CONTRACT" "$BASE_URL" | tee -a spike_verdict.ndjson

Engineers iterate by editing the handler, restarting the script, and appending another JSON line until the outer timeout fires. The last line in spike_verdict.ndjson is the only ballot that counts, even when earlier lines looked friendlier. Calling timeout --signal=TERM 5400 ./run_spike.sh makes ship-or-kill a property of the clock rather than a mood. Without that wrapper, spikes become weekend novels that keep a testing hobby in the footnotes.

Isolation still matters after the scripts exist and the contract file looks official inside git. A laptop compiling an unrelated tree will convict an innocent handler by stealing scheduler time. A quiet laptop will pardon a guilty handler that only works because nothing else is awake. That is why a free remote server belongs in the method when the hypothesis concerns latency rather than syntax.

MonkeyCode's free model access and free server option fit this isolation problem without turning the article into a catalog. Disclosure: This article was prepared as part of MonkeyCode's product outreach. The only product facts used here are that free model access and a free server option exist for this workflow. Model names, quotas, instance sizes, duration, and permanence stay out of scope because they are not required to run the boolean.

The assistant may draft the handler, yet it may not own the contract, the warmup, or the timeout. Mixing those duties is how pretty transcripts quietly replace boolean evidence after the clock stops. After the process exits, keep the JSON verdict beside the contract file in the same commit. Delete the chat log if it starts to feel like an appendix that wants a vote it did not earn.

Several failure modes appear before minute ninety if the team watches the wrong lights on the box. A warmup that begins before the server accepts sockets records a fake massacre of 599 responses. A warmup that reuses one TCP client hides thread-safety bugs that separate users will still find. A p95 copied from a vendor slide will kill honest handlers that were never meant to be edge caches.

Each of those mistakes is a second hypothesis smuggled into the first while the clock still appears calm. Cold start is another thief, because the first second after listen() is not the service anyone will ship. Discarding a short discarded prefix of samples keeps the verdict about steady overlap rather than boot. The contract can mention that prefix, but it cannot mention a second product goal in the same breath.

This approach is the wrong tool for capacity planning, multi-region failover, and anything needing an hour of boring generated traffic. It is also wrong when the service talks to a shared staging database whose other tenants are the real experiment. People chasing a launch SLO should graduate to a proper load lab after this spike ships a candidate. People chasing a blog screenshot should skip the spike, because the timeout will not flatter the narrative.

The closing move is dull on purpose so the spike cannot grow a sequel inside the same budget. If the verdict is PASS, merge the handler and the contract together so later agents cannot widen the path. If the verdict is KILL, archive the branch and write one sentence about which number moved. Either way the ninety minutes purchased a boolean, not a feeling about whether the model seemed helpful.

Readers who already time-box work this way can drop the same warmup onto a free server and keep the boolean instead of the transcript. The server is infrastructure for isolation, not a prize, and the contract file remains the only applause. If the boolean flips, believe the warmup rather than the model's explanation of why the number should not count.