The Spike's Exit Packet

Picture a branch named spike/assistant-trial that is eleven days old, whose last commit message reads wip, and whose author has already moved to another ticket. The spike did produce output, in the form of half-finished scripts and a long thread and a vague memory of something being faster, but nothing that a second engineer can pick up and re-run. A time box guarantees that the work stops; it says nothing about whether the stop leaves anything usable behind.

The exit interview is the missing half of spike discipline. Instead of treating minute ninety as a cliff, the spike should end by producing a packet that a stranger can re-run on a clean machine and use to reach the same verdict. That requirement is cheap when it is designed at minute zero and nearly impossible to reconstruct afterwards, because the evidence that matters tends to be the evidence the spike already deleted.

Pre-register the kill criterion as a number

A hypothesis that no shell script can falsify is not ready to be spiked. Before the first command runs, write the claim, the trial command, and both thresholds into hypothesis.md and commit it, because that commit is the pre-registration and it is the only thing preventing the goalposts from moving at minute eighty-five. Thresholds have to be counts rather than adjectives: "the assistant fixes the bug" cannot fail, while "three of five fixtures pass their own check script" can. The template below also caps extensions at one, which matters more than any other line in the file.

#!/usr/bin/env bash
# spike-init.sh <spike-id> — run from the repository root, minute 0
set -euo pipefail
ID="${1:?usage: spike-init.sh <spike-id>}"
PACKET="spikes/$ID"
mkdir -p "$PACKET/evidence"
cat > "$PACKET/hypothesis.md" <<'TPL'
# Claim
One sentence, falsifiable, about one assistant on one fixture set.
# Trial
The exact command that produces evidence, runnable unattended from repo root.
# Kill criterion
Fewer than 3 of 5 fixtures pass their own check script.
# Ship criterion
At least 4 of 5 pass, zero infrastructure failures, median run under 6 minutes.
# Extension cap
One extension of 20 minutes, allowed only with a written reason in decision.md.
TPL
{ date -u +'started_utc=%Y-%m-%dT%H:%M:%SZ'; hostname; } | tee "$PACKET/evidence/clock.txt"

The clock line looks trivial and is not. Wall-clock minutes are not comparable across hosts, so the packet records the start time and the host alongside whatever version string the tool volunteers about itself. When a tool reports no version at all, the honest move is to write that absence down as a reproducibility gap instead of pretending the run is pinned to something.

Split the hosts: disposable evidence, durable code

The second decision is where the trial runs, and it is usually made by accident. A spike that borrows the developer's laptop inherits dotfiles, cached credentials, and nine half-configured toolchains, while a spike that borrows production inherits something considerably worse. The cleaner arrangement uses two hosts: the repository and the packet live where they will be reviewed, and the trial itself runs on a disposable machine that is destroyed at minute ninety and rebuilt from the packet whenever somebody disputes the result.

Free capacity changes the arithmetic of that arrangement without changing its conclusions. MonkeyCode offers free model access and a free server option, which means the disposable half of the split does not need a budget conversation before the clock starts. Disclosure: This article was prepared as part of MonkeyCode's product outreach. The availability claims here are operator-supplied and time-sensitive, so check the current project page for quotas and hardware before planning around them, and treat any figure you remember from an older post as stale until you have re-read it.

The durable half stays deliberately small. The driver below assumes exactly one thing about the assistant under evaluation, which is that it can be invoked as a command reading a prompt on stdin, and that indirection is what keeps the packet portable when flags and defaults change underneath it.

One command, five fixtures, no product flags

#!/usr/bin/env bash
# run_trial.sh <spike-id> — run from the repository root, minutes 5..80
set -uo pipefail
ROOT="$(pwd)"
ID="${1:?usage: run_trial.sh <spike-id>}"
OUT="$ROOT/spikes/$ID/evidence"
: "${ASSISTANT_CMD:?set ASSISTANT_CMD to the command that invokes the assistant}"
mkdir -p "$OUT" "$ROOT/.spike-wt"
while IFS= read -r fixture; do
  name="$(basename "$fixture")"
  wt="$ROOT/.spike-wt/$name"
  if ! git worktree add --detach "$wt" HEAD >/dev/null 2>&1; then
    printf '%s\t%s\t%s\t%s\n' "$name" worktree_failed - - >> "$OUT/runs.tsv"
    continue
  fi
  start=$(date +%s)
  ( cd "$wt" && eval "$ASSISTANT_CMD" < "$ROOT/$fixture/prompt.txt" > "$OUT/$name.out" 2>&1 )
  assistant_rc=$?
  ( cd "$wt" && bash "$ROOT/$fixture/check.sh" > "$OUT/$name.check" 2>&1 )
  check_rc=$?
  dur=$(( $(date +%s) - start ))
  printf '%s\t%s\t%s\t%s\n' "$name" "$assistant_rc" "$check_rc" "$dur" >> "$OUT/runs.tsv"
  git worktree remove --force "$wt"
done < <(ls -d fixtures/*/)

Each fixture directory holds a prompt.txt and a check.sh that exits zero only when the behaviour under test is correct, so the assistant's own claims about success never enter the evidence. Every run happens in a detached worktree at HEAD, which keeps the trial reproducible from the packet alone and stops experimental edits from contaminating the fixtures. Failures of the harness, such as a worktree that will not attach, are logged as infrastructure rows and invalidate the run rather than counting against the tool.

The exit gate reads the table, not the vibes

Minute ninety is for one command, and that command is not allowed to interpret anything. It counts rows, compares them to the pre-registered threshold, and writes a single line that becomes the spike's verdict.

#!/usr/bin/env bash
# verdict.sh <spike-id> <required-passes> — minutes 90..95, the exit gate
set -euo pipefail
ID="${1:?usage: verdict.sh <spike-id> <required-passes>}"
NEED="${2:?usage: verdict.sh <spike-id> <required-passes>}"
RUNS="spikes/$ID/evidence/runs.tsv"
total=$(awk -F'\t' 'NF==4' "$RUNS" | wc -l | tr -d ' ')
passes=$(awk -F'\t' '$3==0' "$RUNS" | wc -l | tr -d ' ')
infra=$(awk -F'\t' '$2=="worktree_failed"' "$RUNS" | wc -l | tr -d ' ')
verdict=KILL
if [ "$passes" -ge "$NEED" ] && [ "$infra" -eq 0 ]; then verdict=SHIP; fi
printf 'passes=%s/%s required=%s infra=%s verdict=%s\n' \
  "$passes" "$total" "$NEED" "$infra" "$verdict" | tee "spikes/$ID/verdict.txt"

The mapping from counts to decisions is intentionally dull, and dullness is the point, because a table cannot be talked into a different answer at the end of a long afternoon.

Passes vs. required	Infrastructure rows	Median run vs. cap	Verdict
at or above	0	at or under	ship, keep the packet
at or above	0	over by more than half	ship with a note, or one capped extension
exactly one short	0	any	extend once by 20 minutes with a written reason
two or more short	any	any	kill, delete the branch, keep the packet
any	above 0	any	kill as inconclusive; the harness failed, not the tool

What the packet must contain

At minute ninety-five the spike owns five files: hypothesis.md written before the clock, evidence/clock.txt, the raw evidence/runs.tsv with the per-fixture output, verdict.txt from the gate, and decision.md that a human writes in one paragraph. The paragraph is the only place where judgement enters, and it is allowed to say that the numbers were met while the experience was miserable, or that the numbers were missed for a reason worth one more spike.

The second-engineer test decides whether the packet is worth committing. Hand the directory to somebody who was not present, ask them to re-run the gate on a fresh host from the packet alone, and compare the verdicts. Agreement is the bar for a ship decision; a one-fixture wobble means the trial is not reproducible yet, and the strongest verdict the spike can honestly claim is an extension.

Where this breaks

Five fixtures are a smoke test and not a benchmark, so the packet proves that evidence can be produced cheaply and says almost nothing about sustained quality over a month. Ninety minutes measures time-to-first-evidence and tends to reward tools that are fast at a narrow task while flattering tools that are slow but careful not to be. Wall-clock durations from a free server are relative rather than absolute, because the hardware is not the hardware anyone ships to, and the recorded host is the only honest way to compare one packet with another.

Free-tier terms and assistant versions both drift, which gives every packet a shelf life that nobody stamps on the cover. Anyone comparing tools for procurement, or running anything that needs real credentials and production data, should not put those workloads on a shared disposable host at all, and should not mistake a spike for a trial with statistical confidence.

If the disposable half of the split is the part that keeps blocking a first trial, the free server option in MonkeyCode is a reasonable place to put it: write the kill criterion first, start the clock, and let the packet make the argument for you.