Unverified Assumptions End the Spike

The on-call engineer watched a coding agent rewrite a webhook handler that had failed twice before lunch. The patch looked tidy, the comments sounded confident, and the local demo returned HTTP 200 on every retry. None of the agent's premises lived in the repository: a local Redis broker, an edge TLS terminator, and a queue named jobs. Ninety minutes later the spike was called complete, while production still dropped signatures that never reached that imagined cache.

A time-boxed spike fails more quietly than a crashing process or a loudly red test suite on CI. It fails when the agent treats guesses as infrastructure and the reviewer treats fluent prose as shipped evidence. Building inspectors refuse to sign a roof because a contractor described shingles in optimistic language after lunch. They climb the ladder and measure what is actually nailed down before anyone calls the work finished.

The working rule stays small so a ninety-minute window can finish without ceremony or extra staffing. The spike states one hypothesis in a file the agent must not rewrite after the first minute. Every extra claim the agent needs must be appended as an assumption with a host-runnable probe. If any assumption remains unverified when the timer ends, the spike is killed rather than merged.

A reviewer can freeze that bet in version control before the agent is allowed to edit application sources. The example file below is a contract, not a diary, and it should read like a wager a skeptic could settle with commands.

# spike_hypothesis.md
id: spike-2026-09-07-webhook-retry
limit_minutes: 90
hypothesis: >
  The webhook handler drops valid retries because signature
  verification reads the raw body after the JSON parser
  has already consumed the stream.
success_evidence: >
  A host-run test posts the same signed body twice and
  both attempts return 200 without contacting Redis.
forbidden_claims:
  - local Redis is required
  - TLS is terminated at an edge proxy
  - queue name is jobs

The ledger beside that contract is JSONL because a spike should append under time pressure rather than redesign a schema. Each line holds one claim, a probe argument vector, and a status the gate alone may mark verified. Agents may propose probes in that file, yet they may not grade their own homework or delete a failing line. A night clerk counts the till against the tape, not against the story the cashier told on the way out.

{"claim":"no redis pid file is required for the retry test","probe_argv":["/usr/bin/test","!","-e","/var/run/redis.pid"],"status":"proposed"}
{"claim":"hmac fixture file exists","probe_argv":["/usr/bin/test","-f","tests/fixtures/signed_webhook.json"],"status":"proposed"}

The gate below is an example script, labeled as unexecuted until an operator runs it on the same host the agent used. It refuses empty ledgers, expired clocks, unknown binaries, and any claim whose probe exits non-zero. After the agent process stops, the host is the only writer allowed to print ship.

#!/usr/bin/env python3
"""Example host gate for a ninety-minute agent spike."""
from __future__ import annotations

import json
import subprocess
import sys
import time
from pathlib import Path

LEDGER = Path("assumptions.jsonl")
HYPOTHESIS = Path("spike_hypothesis.md")
STARTED_AT = Path(".spike_started_at")
MAX_MINUTES = 90
ALLOWED_NAMES = {
    "test", "ls", "cat", "head", "wc", "git",
    "node", "npm", "python3", "pytest",
}

def read_jsonl(path: Path) -> list[dict]:
    if not path.exists():
        return []
    rows = []
    for raw in path.read_text(encoding="utf-8").splitlines():
        line = raw.strip()
        if line:
            rows.append(json.loads(line))
    return rows

def main() -> int:
    if not HYPOTHESIS.exists():
        print("kill: missing spike_hypothesis.md", file=sys.stderr)
        return 2
    if not STARTED_AT.exists():
        print("kill: spike clock was never started", file=sys.stderr)
        return 2

    started = float(STARTED_AT.read_text().strip())
    elapsed = (time.time() - started) / 60.0
    if elapsed > MAX_MINUTES:
        print(f"kill: elapsed {elapsed:.1f}m exceeds {MAX_MINUTES}m")
        return 3

    rows = read_jsonl(LEDGER)
    if not rows:
        print("kill: empty assumption ledger")
        return 2

    failures: list[str] = []
    for row in rows:
        claim = row.get("claim", "")
        status = row.get("status")
        probe = row.get("probe_argv")
        if status == "killed":
            failures.append(f"explicitly killed: {claim!r}")
            continue
        if not isinstance(probe, list) or not probe:
            failures.append(f"missing probe_argv for {claim!r}")
            continue
        binary_name = Path(str(probe[0])).name
        if binary_name not in ALLOWED_NAMES:
            failures.append(f"binary not allowlisted: {binary_name}")
            continue
        try:
            completed = subprocess.run(
                probe,
                check=False,
                capture_output=True,
                text=True,
                timeout=20,
            )
        except (subprocess.TimeoutExpired, FileNotFoundError) as exc:
            failures.append(f"probe error for {claim!r}: {exc}")
            continue
        if completed.returncode != 0:
            detail = (completed.stderr or completed.stdout)[:200]
            failures.append(f"unverified: {claim!r} :: {detail}")

    if failures:
        print("kill: assumptions remain unverified")
        for item in failures:
            print(f"  {item}")
        return 4

    print("ship: every assumption has host evidence")
    return 0

if __name__ == "__main__":
    raise SystemExit(main())

A short driver starts the clock once, freezes the hypothesis file, and refuses to grade a workspace while the agent still holds the pen. The snippet is an example workflow for a disposable server, not a production supervisor.

#!/usr/bin/env bash
# Example driver for a ninety-minute assumption spike.
set -euo pipefail

ROOT="$(pwd)"
HYP="$ROOT/spike_hypothesis.md"
CLOCK="$ROOT/.spike_started_at"
GATE="$ROOT/assumption_gate.py"

if [[ ! -f "$HYP" ]]; then
  echo "kill: freeze spike_hypothesis.md before starting" >&2
  exit 2
fi

if [[ ! -f "$CLOCK" ]]; then
  date +%s > "$CLOCK"
  chmod a-w "$CLOCK" "$HYP" || true
fi

if [[ -f .agent.pid ]] && kill -0 "$(cat .agent.pid)" 2>/dev/null; then
  echo "kill: agent still running; stop it before the gate" >&2
  exit 2
fi

python3 "$GATE"
echo "gate_exit:$?"

During the remaining minutes the reviewer watches three boring signals rather than the chat transcript. Elapsed time against .spike_started_at says whether the experiment is still a spike. The ledger length says whether the agent is multiplying hidden architecture instead of testing one idea. The gate exit code says ship or kill without a slide deck.

Consider a concrete miss that this ritual is designed to catch before a demo. The agent explains that retries need Redis, then adds a client, then reports green because a mocked unit test never opened a socket. The ledger line claims Redis is present, the allowlisted probe is /usr/bin/test -S /var/run/redis.sock, and the disposable server has no such socket. The gate prints kill and the branch stays unpublished, which is the entire point of a spike.

The opposite path is narrower and less cinematic, which is also the point. The hypothesis says the handler reads a parsed body when it should read raw bytes for HMAC verification. The agent changes one function, adds a host test that posts the same signed payload twice, and records a single assumption that no broker process is required. The probe is /usr/bin/test ! -e /var/run/redis.pid, the test runner returns zero, and the gate prints ship because every written claim met host evidence inside ninety minutes.

A laptop that still has last year's Redis from an unrelated tutorial will lie in the agent's favor, so the probes need a clean machine. Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode's free model access and free server option give that isolated room: the model can attempt the spike, the server can run the probes, and the reviewer can kill the branch without turning a disposable hypothesis into a paid token exercise. The product is not the webhook theory; it is only the room in which the theory is allowed to fail in public.

The script is not a sandbox, an audit log, or a substitute for code review by a human who can read the diff. An agent that still has write access can rewind .spike_started_at, blank the ledger, or replace assumption_gate.py with a stub that always prints ship. Operators should stop the agent, copy the ledger aside, and run the gate from a checkout the model cannot edit. Probe allowlists also fail open if someone adds bash or a free-form interpreter; keep probes to fixed binaries and short argument shapes that a reviewer can read in one glance.

This ritual is a poor fit for incident response, because a production outage does not pause for a frozen hypothesis file. It is a poor fit for open-ended design work, where exploring three architectures is the actual goal rather than a single defect. Teams that cannot assign a human owner for the clock should not pretend the kill switch exists. People who want a continuous coding agent as a pair programmer will hate the ceremony, and they should evaluate that workflow with a different harness.

Readers who already time-box agent evaluations can copy the ledger onto a disposable server and keep the kill rule stricter than the transcript.