DEV Community: Harper Zhu

Split the Clock and Repeat the Spike

Harper Zhu — Wed, 16 Sep 2026 03:46:08 +0000

On a gray Tuesday a payments team let an agent chase a flaky CSV importer while a wall clock sat beside the keyboard. Forty minutes later the unit suite was green and the chat log sounded like a victory lap around the office. A reviewer merged the branch because the spike had a hypothesis, a deadline, and a passing test command. Production broke that night when a fixture filename leaked into a parser branch that nobody had frozen.

Single-run spikes still dominate agent evaluation because a green bar feels like evidence after a tense hour. The bar is a snapshot, not a decision, and snapshots lie when the environment drifts between attempts. A ninety-minute spike needs a ship-or-kill rule that survives luck, cached files, and an agent that memorizes fixtures.

Public conversation this month keeps returning to whether coding agents already outrun the tests used to grade them. Other threads warn that treating a fluent session as engineering confuses a demo with a change that can be replayed. Those arguments stay theatrical until a team records one hypothesis and demands two agreeing runs inside one clock.

The workflow below keeps a ninety-minute box and adds a variance gate that a single green run cannot satisfy. Before the clock starts, the public interface of the change is written by a human and hashed on disk. The agent may implement behind that interface, yet any edit to the contract file kills the spike immediately. The same hypothesis then runs twice in freshly unpacked directories, and disagreement between those runs is also a kill.

Both confirming runs must finish inside the original ninety minutes, including package install time and the test command. If the first attempt consumes the whole budget, the spike fails because no second sample exists to compare. That rule sounds harsh until a team remembers that irreproducible speed is how fixture leaks reach production.

A copyable template uses three artifacts: a frozen contract module, tests that import only that module, and a gate script. The script is an unexecuted example for operators to adapt, not a measured claim about any particular model. Replace the placeholder agent command with the runner the team already trusts, then keep the comparison logic intact.

# contract.py — freeze and hash this file before the clock starts.
from typing import Protocol


class CsvBatchImporter(Protocol):
    def parse_rows(self, payload: bytes) -> list[dict[str, str]]:
        """Return row dicts keyed by the header names in the first line."""
        ...

    def reject_reason(self, payload: bytes) -> str | None:
        """Return a stable error token, or None when parse_rows would succeed."""
        ...

The protocol is intentionally small so the spike cannot hide a redesign inside a passing suite. Header keys and a stable reject token are ordinary product rules rather than a published research benchmark. A human writes this file, commits it, and copies its hash into the evidence directory before any agent process starts.

# test_importer.py — bind only to the frozen protocol and a stable reject token.
from importer import DefaultImporter

SAMPLE = b"id,amount\n1,10.00\n2,bad\n"


def test_parse_rows_keeps_header_keys():
    rows = DefaultImporter().parse_rows(b"id,amount\n1,10.00\n")
    assert rows == [{"id": "1", "amount": "10.00"}]


def test_reject_reason_is_stable_token():
    reason = DefaultImporter().reject_reason(SAMPLE)
    assert reason == "invalid_amount"

These tests refuse to import helper modules the agent might invent for convenience during a lucky first run. They also pin the reject token so a second run cannot rename errors and still look successful. If DefaultImporter is missing, the suite fails, and that failure is a valid kill rather than a reason to extend the clock.

#!/usr/bin/env bash
# spike_gate.sh — unexecuted template for a 90-minute two-run spike.
set -euo pipefail

SPIKE_MINUTES="${SPIKE_MINUTES:-90}"
HYPOTHESIS="${HYPOTHESIS:-DefaultImporter satisfies CsvBatchImporter and reject tokens stay stable}"
CONTRACT="${CONTRACT:-contract.py}"
AGENT_CMD="${AGENT_CMD:-echo replace-me-with-the-team-agent-runner}"
TEST_CMD="${TEST_CMD:-python3 -m pytest -q test_importer.py}"
ROOT="$(pwd)"
START_EPOCH="$(date +%s)"
DEADLINE="$((START_EPOCH + SPIKE_MINUTES * 60))"
STAMP="$(date -u +%Y%m%dT%H%M%SZ)"
EVIDENCE="$ROOT/spike_evidence_$STAMP"
mkdir -p "$EVIDENCE"

remaining() {
  echo "$((DEADLINE - $(date +%s)))"
}

if [[ ! -f "$CONTRACT" ]]; then
  echo "kill: missing $CONTRACT" >&2
  exit 2
fi

CONTRACT_SHA="$(sha256sum "$CONTRACT" | awk '{print $1}')"
printf '%s\n' "$HYPOTHESIS" > "$EVIDENCE/hypothesis.txt"
printf '%s\n' "$CONTRACT_SHA" > "$EVIDENCE/contract.sha256"
git archive --format=tar HEAD > "$EVIDENCE/clean_tree.tar"

run_one() {
  local name="$1"
  local dir="$EVIDENCE/$name"
  mkdir -p "$dir"
  tar -xf "$EVIDENCE/clean_tree.tar" -C "$dir"

  local budget
  budget="$(remaining)"
  if (( budget < 120 )); then
    printf '%s\n' "{\"name\":\"${name}\",\"status\":\"timeout_before_start\",\"test_rc\":1,\"contract_sha256\":\"${CONTRACT_SHA}\",\"requirements_sha256\":\"\"}" > "$dir/result.json"
    return 0
  fi

  if [[ "$name" == "run_a" ]]; then
    budget="$((budget / 2))"
  fi

  local test_rc new_sha req_sha
  pushd "$dir" >/dev/null
  set +e
  timeout --preserve-status "$budget" bash -lc "$AGENT_CMD" >agent.log 2>&1
  timeout --preserve-status 120 bash -lc "$TEST_CMD" >test.log 2>&1
  test_rc=$?
  set -e
  new_sha="$(sha256sum "$CONTRACT" | awk '{print $1}')"
  req_sha=""
  if [[ -f requirements.txt ]]; then
    req_sha="$(sha256sum requirements.txt | awk '{print $1}')"
  fi
  python3 - "$name" "$test_rc" "$new_sha" "$req_sha" "$budget" >result.json <<'PY'
import json, sys
name, test_rc, new_sha, req_sha, budget = sys.argv[1:6]
print(json.dumps({
    "name": name,
    "status": "finished",
    "test_rc": int(test_rc),
    "contract_sha256": new_sha,
    "requirements_sha256": req_sha,
    "budget_seconds": int(budget),
}))
PY
  popd >/dev/null
}

run_one run_a
run_one run_b
python3 "$ROOT/compare_runs.py" \
  "$EVIDENCE/run_a/result.json" \
  "$EVIDENCE/run_b/result.json" \
  "$CONTRACT_SHA"

The archive step matters more than the timeout arithmetic, because a dirty working tree is how laptop caches impersonate skill. Using git archive exports HEAD without uncommitted helper files the agent dropped during a previous attempt. Each run unpacks that tarball into its own directory so log files cannot leak from run_a into run_b. Half of the remaining seconds go to the first run so the second run is not leftover theater.

The template assumes GNU timeout from coreutils, which treats the budget as a hard stop rather than a hint. Operators who dislike splitting the clock can keep two directories, yet they no longer have a ninety-minute spike. A longer box is a different experiment, and mixing those experiments is how lucky sessions get relabeled as process.

chmod +x spike_gate.sh compare_runs.py
git add contract.py test_importer.py spike_gate.sh compare_runs.py
git commit -m "Freeze importer contract before the spike clock"
SPIKE_MINUTES=90 ./spike_gate.sh
echo "gate_exit=$?"

Those commands are the whole ritual: freeze the contract, commit the gate, start the clock, and inspect the exit status. Nothing in the chat log can replace the evidence directory that the script leaves beside the repository root. If the exit status is nonzero, the branch stays unmerged even when one of the agent logs reads like a confident summary.

#!/usr/bin/env python3
"""compare_runs.py — unexecuted template. Ship only when both isolated runs agree."""
from __future__ import annotations

import json
import sys
from pathlib import Path


def load(path: str) -> dict:
    return json.loads(Path(path).read_text(encoding="utf-8"))


def main() -> int:
    if len(sys.argv) != 4:
        sys.stderr.write("usage: compare_runs.py run_a.json run_b.json expected_contract_sha\n")
        return 2
    run_a = load(sys.argv[1])
    run_b = load(sys.argv[2])
    expected = sys.argv[3]
    reasons: list[str] = []
    for label, row in ("run_a", run_a), ("run_b", run_b):
        if row.get("status") == "timeout_before_start":
            reasons.append(f"{label} never started")
        if int(row.get("test_rc", 1)) != 0:
            reasons.append(f"{label} tests exited {row.get('test_rc')}")
        if row.get("contract_sha256") != expected:
            reasons.append(f"{label} mutated the contract")
    if run_a.get("contract_sha256") != run_b.get("contract_sha256"):
        reasons.append("contract hashes diverged")
    if run_a.get("requirements_sha256") != run_b.get("requirements_sha256"):
        reasons.append("requirements hashes diverged")
    if run_a.get("test_rc") != run_b.get("test_rc"):
        reasons.append("test exit codes diverged")
    if reasons:
        print("kill")
        for item in reasons:
            print(item)
        return 1
    print("ship")
    print("both runs passed on the frozen contract")
    return 0


if __name__ == "__main__":
    raise SystemExit(main())

compare_runs.py is the entire decision procedure, written as code so a chat summary cannot override it. Ship requires matching contract hashes, matching requirement hashes, and matching zero test statuses from both directories. Any other combination prints kill and a reason; the spike ends without a planning meeting attached to the exit code.

Teams that lack a spare empty machine often run both directories on a short-lived remote shell so a laptop cache cannot contaminate the tarball. Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode provides free model access and a free server option that can host those two isolated runs while the contract hash stays under human control. The gate still belongs to the compared JSON files, and it remains useful if that product mention is ignored.

Reading the evidence directory should feel like reading a receipt rather than a blog post about the session. hypothesis.txt states the only claim under test, and result.json files show whether both clocks produced the same numbers. If reviewers need a narrative, they can add one after the gate, never before it, and never as a substitute for the second run.

The approach is a poor fit for production incidents, where the clock is an outage rather than a learning box. It also fails when the hypothesis requires browsing unknown public APIs, because a frozen contract cannot describe a surface the team has not seen. Visual work without a stable interface, and refactors that must rewrite the public types, should use a different review path than a ninety-minute spike.

Two agreeing runs can still share a systematic hole if both tests encode the same fixture mistake. Matching requirement hashes still allow both runs to add the same unjustified dependency, which isolation will not catch. Isolation cuts environmental luck, and it does not invent missing assertions or turn a weak protocol into a strong one. Teams that cannot control network egress or package mirrors should treat a green gate as a weak signal rather than a merge stamp.

A spike that cannot be repeated inside the same clock is a demo, and demos can teach without shipping. The payments team did not need a longer meeting; they needed a second empty directory and a hash that the agent was forbidden to touch. Freeze the contract, split the remaining minutes, and let disagreement kill the branch before a fixture name reaches production.

Give Your Spike an Exit Code, Not a Retrospective

Harper Zhu — Mon, 14 Sep 2026 21:33:02 +0000

A spike ends when the timer does, but the decision tends to end somewhere else entirely. Three engineers read the same notes after a ninety-minute box closes, and each of them walks away with a different verdict: one calls the probe a success, another keeps pointing at a timeout that never fully disappeared. The probe was probably fine; what was missing was a stop rule that could be executed instead of argued. Notes describe a spike, but they cannot decide one, and that gap is where most of the post-spike hour is spent.

The fix is unglamorous and small. Write the thresholds before the first probe runs, append every probe to a line-delimited ledger, and let a short script read both files and print a verdict with an exit code. This is a proposed workflow with illustrative code, not a report of a specific team's results, so treat the numbers below as placeholders you replace with your own. The point is that the last ten minutes of the spike become a command rather than a conversation.

Freeze the hypothesis before the clock starts

A decision rule has to exist before the evidence arrives, otherwise the evidence quietly rewrites the rule. The file below registers one hypothesis, the smallest number of probes that count as a sample, and the two thresholds that bound the outcome. Anything not written here cannot influence the verdict later, which is the whole reason to keep it this short.

{
  "spike": "cache-write-fence",
  "deadline_minutes": 90,
  "min_probes": 8,
  "ship_if": { "rate": 0.9 },
  "kill_if": { "failures_with_same_signature": 3 },
  "signature_fields": ["error_class", "stage"],
  "hypothesis": "the write fence prevents partial writes under concurrent retries"
}

The ship_if rule asks whether the mechanism works at all, while kill_if asks whether it fails in the same place three times. That second rule matters more than it looks, because repeated identical failures in a ninety-minute box are usually structural rather than incidental. Freezing the file with sha256sum spike/hypothesis.json > spike/hypothesis.sha256 gives you a cheap way to prove later that nobody tuned the thresholds to fit the results.

Keep a ledger of probes, not a diary

Prose notes mix observations with interpretation, and the checker needs only the former. A ledger keeps one JSON object per line, which means a probe can be appended by a human, a wrapper script, or a test runner without any coordination between them. The note kind records context such as an environment change, and it stays out of the pass rate because the checker filters on kind.

{"t":"2026-09-15T09:02:11Z","kind":"probe","id":"p1","ok":true,"ms":421,"error_class":null,"stage":"write"}
{"t":"2026-09-15T09:02:39Z","kind":"probe","id":"p2","ok":false,"ms":903,"error_class":"TimeoutError","stage":"fsync"}
{"t":"2026-09-15T09:04:02Z","kind":"note","text":"raised probe timeout to 1500ms; /tmp is on overlayfs"}
{"t":"2026-09-15T09:05:20Z","kind":"probe","id":"p3","ok":false,"ms":1502,"error_class":"TimeoutError","stage":"fsync"}

Two fields carry unusual weight here. The stage value is what turns three unrelated timeouts into one repeated signature, and ms is what lets a reader tell a real failure from a probe that was simply given too little time. Honest classification of error_class is the weakest link in the whole format, because a wrapper that labels everything nonzero_exit produces a signature that can never repeat.

Let a checker say SHIP, KILL, or STILL-OPEN

The script below prints one line and exits with a code that a shell or CI job can act on, which is the part a retrospective cannot offer. Exit codes are cheap to wire into habits: 0 means ship, 1 means kill, and 2 means the spike has not earned either verdict yet.

#!/usr/bin/env python3
"""spike_verdict.py - print SHIP, KILL, or STILL-OPEN from a pre-registered ledger."""
import json
import sys
from collections import Counter
from datetime import datetime, timezone
from pathlib import Path


def load_events(path):
    lines = Path(path).read_text().splitlines()
    return [json.loads(line) for line in lines if line.strip()]


def parse(ts):
    return datetime.fromisoformat(ts.replace("Z", "+00:00"))


def decide(hyp, events, now=None):
    now = now or datetime.now(timezone.utc)
    probes = [e for e in events if e.get("kind") == "probe"]
    start = min((parse(e["t"]) for e in events), default=now)
    minutes = (now - start).total_seconds() / 60
    passed = sum(1 for p in probes if p.get("ok"))
    rate = passed / len(probes) if probes else 0.0
    sigs = Counter(
        tuple(p.get(f) for f in hyp["signature_fields"])
        for p in probes if not p.get("ok")
    )
    signature, repeats = sigs.most_common(1)[0] if sigs else (None, 0)
    if len(probes) >= hyp["min_probes"] and rate >= hyp["ship_if"]["rate"]:
        return "SHIP", f"{passed}/{len(probes)} probes passed ({rate:.0%})"
    if repeats >= hyp["kill_if"]["failures_with_same_signature"]:
        return "KILL", f"{repeats} failures share signature {signature}"
    if minutes >= hyp["deadline_minutes"]:
        return "KILL", f"clock expired at {minutes:.1f} min, {passed}/{len(probes)} passed"
    return "STILL-OPEN", f"{passed}/{len(probes)} passed, {minutes:.1f} min used"


if __name__ == "__main__":
    hypothesis = json.loads(Path(sys.argv[1]).read_text())
    code, detail = decide(hypothesis, load_events(sys.argv[2]))
    print(f"{code}: {detail}")
    sys.exit({"SHIP": 0, "KILL": 1, "STILL-OPEN": 2}[code])

The order of the rules is a deliberate choice worth stating out loud. Evidence is evaluated before the clock, so a spike that reaches its ship threshold in minute eighty-four still ships, and an expired clock only kills a spike that never earned a verdict. A run against the sample ledger looks unremarkable, which is exactly the property you want from a decision tool.

$ python spike_verdict.py spike/hypothesis.json spike/events.jsonl
STILL-OPEN: 3/4 passed, 6.4 min used
$ echo $?
2

Record the clock honestly, including setup

A ninety-minute box that starts at the first successful probe is not a ninety-minute box, so the wrapper below opens the ledger at the moment the command starts and measures it with a hard timeout. Each invocation appends one probe line, which keeps manual bookkeeping out of the loop and makes the ledger append-only by construction.

#!/usr/bin/env bash
# usage: LEDGER=spike/events.jsonl PROBE_STAGE=fsync spike-run p3 -- ./probe.sh
set -u
LEDGER="${LEDGER:-spike/events.jsonl}"
ID="$1"; shift; [ "${1:-}" = "--" ] && shift
START=$(date -u +%s%3N)
OUT=$(mktemp)
timeout "${PROBE_TIMEOUT:-20s}" "$@" >"$OUT" 2>&1; RC=$?
END=$(date -u +%s%3N)
OK=$([ "$RC" -eq 0 ] && echo true || echo false)
ERR=$([ "$RC" -eq 124 ] && echo TimeoutError || echo nonzero_exit)
printf '{"t":"%s","kind":"probe","id":"%s","ok":%s,"ms":%s,"error_class":%s,"stage":"%s"}\n' \
  "$(date -u +%Y-%m-%dT%H:%M:%SZ)" "$ID" "$OK" "$((END-START))" \
  "$([ "$OK" = true ] && echo null || printf '"%s"' "$ERR")" "${PROBE_STAGE:-unknown}" >>"$LEDGER"
cat "$OUT"; exit "$RC"

Everything before that first probe is unpaid clock, and it is usually paid to provisioning: an endpoint, a credential, a machine that can hold the workload. The MonkeyCode project is one option that removes part of that tax, since it advertises free model access plus a free server option, with a free allowance described as ten million tokens. Disclosure: This article was prepared as part of MonkeyCode's product outreach. Those availability claims come from the project rather than from independent verification here, so confirm the current limits yourself before you design a spike around them; the wrapper does not care which endpoint answers, only that one answers inside the box.

What this workflow cannot do

A checker certifies that a rule was applied, not that the probe measured something meaningful, and that distinction limits where the pattern belongs. Latency and throughput questions need distributions rather than pass rates, since eight probes cannot separate an eighty-five percent mechanism from a ninety-five percent one. The wrapper's signature quality depends entirely on honest error_class values, and date +%s%3N is GNU-specific, so macOS users need gdate or a small Python replacement. Anyone whose spike is a single long run rather than repeated probes should skip the ledger and keep the hypothesis file, and teams that cannot freeze thresholds before the first probe will only generate confident nonsense.

A killed spike is still an asset when its ledger survives, because the next hypothesis can be diffed against the signatures that already failed. That is the quiet payoff of writing the stop rule first: the meeting disappears, the negative evidence stays searchable, and the next ninety minutes start from a fact instead of a memory. If repeated spikes are part of your week, the ledger plus the checker is smaller than the retrospective it replaces, and the free tier in the MonkeyCode repository is worth a look the next time provisioning eats the first ten minutes of your box.

A Hunk Map for a Ninety-Minute Spike

Harper Zhu — Sun, 13 Sep 2026 19:48:10 +0000

On a Tuesday afternoon, a staff engineer blocked ninety minutes for one agent spike on a flaky client. The hypothesis was narrow: the HTTP layer swallowed 429 bodies, so callers never observed a Retry-After header. Twenty minutes later the assistant had renamed two packages, extracted a logger, and invented a circuit breaker. The original timeout still failed in the same way, yet the working tree already looked like a small rewrite.

That scene repeats whenever a coding agent is scored on motion instead of mapped change. Passing tests can hide a novel of unrelated edits, and a calendar alert cannot reconstruct intent from a noisy diff. A ninety-minute spike needs a kill rule that reads the diff the way a reviewer would, hunk by hunk. The protocol below treats every unmapped hunk as a reason to stop, even when the suite is green.

Time-boxed agent work fails in a particular way that ordinary review often notices only after merge. The hypothesis stays buried in a chat scroll, while the tree accumulates helpers that belong on some other ticket. A kitchen timer cannot save a recipe once every spice in the rack has been poured into the pot. The host needs a ledger that binds each unified-diff hunk to the single claim under test, then refuses to ship a mismatch.

The proposed artifact is a host-owned hunk map, kept outside the agent's write path, and checked before merge. The map records one hypothesis sentence, the allowed paths, and a tag for every hunk the host will accept. After each agent turn the host regenerates the diff, matches hunk headers to the map, and either continues or kills. Unlisted files, extra hunks, or tags that ignore the hypothesis all count as unexplained work.

Some teams run the disposable checkout on a separate machine so the laptop remains the only writer of the map. Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode currently offers free model access and a free server option that can hold the isolated spike tree. The host still keeps the ledger local, and this article does not claim model names, quotas, or hardware details.

A ledger the agent cannot hold

Before the clock starts, the host freezes a baseline commit and creates two files the agent must not edit. One file states the hypothesis in a single sentence, and the other is the YAML map the checker will read. The commands below assume a clean git worktree dedicated to the spike, not the engineer's everyday branch. The workflow is a proposal with labeled scripts, not a claim that a particular team already ran these numbers.

git worktree add /tmp/spike-429 HEAD
cd /tmp/spike-429
git checkout -B spike/retry-after
mkdir -p .spike src tests

cat > .spike/hypothesis.txt <<'EOF'
HTTP client surfaces Retry-After from 429 responses to callers.
EOF

cat > .spike/hunk-map.yml <<'EOF'
hypothesis: "HTTP client surfaces Retry-After from 429 responses to callers."
deadline_minutes: 90
allowed_paths:
  - src/http_client.py
  - tests/test_http_client.py
hunks: []
EOF

chmod 0444 .spike/hypothesis.txt

The empty hunks list is intentional at minute zero, because no agent write has been accepted yet. The host will append tags only after reading a real diff, never in advance of the change. That delay is the difference between a plan and a receipt, since a pre-filled map invites fiction. A post-diff map forces the human to look at the hunks that actually landed in the tree.

Host-side commands and a checker

A small checker then becomes the ship-or-kill gate for the rest of the ninety minutes. The script is proposed host-side Python, run on the evidence machine, not inside the agent's tool loop. It parses unified hunks from git diff and requires each file plus header to appear in the YAML map. A tag must reuse a token from the hypothesis so a vague note like cleanup cannot launder extra work.

#!/usr/bin/env python3
# Proposed host-side checker. Label: unexecuted example for a ninety-minute spike.

import subprocess
import sys
from pathlib import Path

import yaml


def load_map(path):
    data = yaml.safe_load(path.read_text())
    if not data or not data.get("hypothesis"):
        raise SystemExit("hunk-map.yml missing hypothesis")
    return data


def git_diff():
    result = subprocess.run(
        ["git", "diff", "--unified=3", "HEAD"],
        check=True,
        capture_output=True,
        text=True,
    )
    return result.stdout


def parse_hunks(diff):
    hunks = []
    current = None
    for line in diff.splitlines():
        if line.startswith("diff --git "):
            parts = line.split(" b/", 1)
            current = parts[1] if len(parts) == 2 else None
            continue
        if line.startswith("@@ ") and current:
            body = line.split("@@")[1].strip()
            old_part, new_part = body.split()[:2]
            old_start = int(old_part.split(",")[0].replace("-", ""))
            new_start = int(new_part.split(",")[0].replace("+", ""))
            hunks.append((current, old_start, new_start))
    return hunks


def tokens(text):
    words = set()
    buf = []
    for ch in text.lower() + " ":
        if ch.isalnum() or ch == "-":
            buf.append(ch)
        else:
            word = "".join(buf)
            if len(word) > 3:
                words.add(word)
            buf = []
    return words


def main():
    spike_map = load_map(Path(".spike/hunk-map.yml"))
    allowed = set(spike_map.get("allowed_paths") or [])
    mapped = {}
    for entry in spike_map.get("hunks") or []:
        key = (entry["file"], int(entry["old_start"]), int(entry["new_start"]))
        mapped[key] = entry
    hyp_tokens = tokens(spike_map["hypothesis"])
    found = parse_hunks(git_diff())
    if not found:
        print("KILL: no diff against HEAD; the hypothesis was not exercised")
        return 2
    failures = []
    for file_name, old_start, new_start in found:
        if file_name not in allowed:
            failures.append("unallowed path " + file_name)
            continue
        key = (file_name, old_start, new_start)
        if key not in mapped:
            failures.append(
                "unmapped hunk %s @@ -%s +%s @@" % (file_name, old_start, new_start)
            )
            continue
        tag_tokens = tokens(str(mapped[key].get("tag", "")))
        if not (tag_tokens & hyp_tokens):
            failures.append("tag does not mention hypothesis tokens: " + file_name)
    found_keys = set(found)
    for key in mapped:
        if key not in found_keys:
            file_name, old_start, new_start = key
            failures.append(
                "stale map entry %s @@ -%s +%s @@" % (file_name, old_start, new_start)
            )
    if failures:
        print("KILL: unexplained diff")
        print("\n".join(failures))
        return 1
    print("MAP OK: every hunk is tagged to the hypothesis")
    return 0


if __name__ == "__main__":
    sys.exit(main())

A toy HTTP client makes the rule concrete without pretending to be a production benchmark of any model. The spike may touch only the client module and its test file, which the allowlist already named. If the agent adds a circuit breaker module, the checker fails even if every test passes. The host copies these fixtures into the worktree before starting the clock, then refuses further scope.

# src/http_client.py — baseline fixture for the proposed spike, not production code
from dataclasses import dataclass


@dataclass
class Response:
    status: int
    headers: dict
    body: str


def handle_response(resp: Response) -> dict:
    # Callers currently receive status and body only.
    return {"status": resp.status, "body": resp.body}

# tests/test_http_client.py — oracle the host runs after minute seventy
from src.http_client import Response, handle_response


def test_429_surfaces_retry_after():
    resp = Response(429, {"Retry-After": "12"}, "rate limited")
    out = handle_response(resp)
    assert out["retry_after"] == "12"

python3 -m pip install pyyaml pytest
python3 -m pytest tests/test_http_client.py -q
python3 .spike/check_hunk_map.py; echo $?
# After an agent turn, rebuild hunks from git diff --unified=3 HEAD
# then rewrite .spike/hunk-map.yml on the host laptop only.
git restore --source=HEAD --staged --worktree .   # kill path: drop unexplained writes

Minute zero through seventy belong to implementation, with a map check after every agent turn that writes files. Each passing check only means the current diff is explained, not that the hypothesis is true yet. At minute seventy the host stops tool writes and spends the remaining twenty minutes on tests and the ledger. If the map still has gaps, or the 429 test still fails, the spike is killed and the branch is deleted.

Suppose the agent edits handle_response to parse Retry-After and adds one test that first fails, then passes. The host runs git diff, records two hunks, and writes tags that mention Retry-After and the 429 path. A third hunk that rewrites logging in the same file is unexplained, so the checker exits nonzero and the clock continues only after revert. Green tests never override that revert, because the suite answers correctness while the ledger answers scope.

An accepted map after a clean two-hunk turn might look like the YAML below, rebuilt from the latest headers rather than edited by guesswork. The tags repeat hypothesis tokens on purpose, which is how a cleanup story fails the set intersection in the checker. Stale line numbers from an earlier turn are treated as unexplained work, not as a paperwork nuisance.

hypothesis: "HTTP client surfaces Retry-After from 429 responses to callers."
deadline_minutes: 90
allowed_paths:
  - src/http_client.py
  - tests/test_http_client.py
hunks:
  - file: src/http_client.py
    old_start: 14
    new_start: 14
    tag: "surface Retry-After from 429 responses"
  - file: tests/test_http_client.py
    old_start: 1
    new_start: 1
    tag: "callers observe Retry-After on 429"

Hunk headers drift as later edits shift line numbers, so the map is rebuilt from the latest diff each turn. The host does not patch old line numbers by hand, which would hide a second change inside a previous hunk. Allowed paths stay frozen from minute zero, which is how the protocol blocks package renames and extra modules. The baseline commit remains the only rollback target if the spike is killed at minute ninety.

When the protocol should be refused

The protocol does not measure model quality, latency, or cost, and it will not replace design review on large changes. Unified diff struggles with binary files, generated bundles, and reformats that touch every line of a module. A read-only bit on the map is only a convention, so the host should keep the ledger off the agent's writable disk. Teams that need a wide refactor should not use a ninety-minute mapped spike as a theatrical substitute for planning.

On-call engineers handling a live incident should not spend the first hour building a hunk ledger around an agent. Contributors who cannot run git worktrees, or who must add dependencies across twenty packages, will find the gate hostile. The method also misfires when the true question is product direction rather than a falsifiable runtime claim. Those cases need a design note and a human schedule, not a kill switch dressed up as agent evaluation.

A mapped spike is useful when the claim is small enough to die cleanly, and the host is willing to delete the branch. The ledger turns the last git diff into evidence instead of a souvenir of the agent's personality. Teams that want a disposable machine for the checkout can try the free server option and keep the hunk map on a trusted laptop.

Fingerprint Repeated Failures in a Ninety-Minute Spike

Harper Zhu — Sat, 12 Sep 2026 17:47:45 +0000

Late on a Thursday, a platform engineer watched an assistant rerun the same failing integration test while the scroll kept moving. The terminal looked busy, and the wall clock still had nearly an hour left on the ninety-minute spike. Each invocation printed the same missing column name, yet the process treated every retry as a fresh attempt. The room had the energy of progress and the substance of a stuck elevator between floors.

Time-boxed spikes fail in a quiet way when busy motion is mistaken for evidence of learning. A coding agent can burn the entire clock inside one failing command without any memory of prior failures. The useful analogy is a locksmith who tries the same key on the same door, then bills for the motion of the wrist. The spike needs a budget for identical failures, not only a calendar alarm on the wall.

The working rule fits inside a small shell wrapper that the coding agent never owns or edits. One hypothesis is written before any agent starts, and the wall clock is set to ninety minutes from that moment. A command fingerprint log sits outside the repository tree the agent is allowed to modify during the spike. If the same command returns a non-zero status three times, the spike ends even when minutes remain on the clock.

The hypothesis file is a contract, not a diary of hopeful notes written after the first failure. It names one change, one observable, and one kill condition that does not depend on the agent's self-report. A spike that cannot state those three items in a short file is not ready for a clock. The sample below is labeled as a template, because the numbers inside it are not measurements from a published trial.

# HYPOTHESIS.md
# Template for a ninety-minute spike. Not a result from a live run.

Statement: Adding an explicit NOT NULL migration will make the users.email
integration test pass without changing application query code.

Observable: pytest tests/test_users_email.py -q exits 0, and git diff --stat
stays inside db/migrations/ plus the test file.

Kill: The same failing command fingerprint is recorded three times, or the
ninety-minute deadline file expires, whichever arrives first.

Ship: Observable holds and the retry budget is unused or only partly used.

Public conversation around agent loops often celebrates another retry as a sign of professional grit. That reading is generous when the process is only a while loop with no memory of earlier exits. A fingerprint log is a cheap way to see the loop as a loop, then stop it with written evidence. The wrapper below is the mechanism, and the hypothesis file is the reason that mechanism may halt work.

The wrapper records a fingerprint for every command the operator or the agent launches through one entry point. The fingerprint is a hash of the working directory, the argument vector, and the integer exit status. A passing command with the same arguments does not consume the failure budget, because success is not a stuck loop. A failing command that matches an earlier failing fingerprint increments a counter that lives beside the log.

#!/usr/bin/env python3
"""Fingerprint a command for duplicate-failure accounting.
Template for a local lab. Not a measured benchmark.
"""
from __future__ import annotations

import hashlib
import json
import os
import sys


def fingerprint(cwd: str, argv: list[str], exit_code: int) -> str:
    payload = json.dumps(
        {
            "cwd": os.path.realpath(cwd),
            "argv": argv,
            "exit": int(exit_code),
        },
        separators=(",", ":"),
        ensure_ascii=True,
    )
    return hashlib.sha256(payload.encode("utf-8")).hexdigest()[:16]


if __name__ == "__main__":
    if len(sys.argv) < 4:
        sys.stderr.write("usage: fingerprint.py <cwd> <exit_code> <argv...>\n")
        sys.exit(2)
    cwd = sys.argv[1]
    code = int(sys.argv[2])
    argv = sys.argv[3:]
    print(fingerprint(cwd, argv, code))

Coarse fingerprints create false kills, and that failure mode is more common than it first appears. Hashing only the binary name treats pytest foo and pytest bar as one command, which collapses unrelated work. Hashing the full argument vector plus the working directory keeps those runs apart while still catching a stuck loop. Environment variables are left out on purpose, because leaking them into a log can copy secrets into the evidence bundle.

The start script uses a portable Python one-liner for the deadline so GNU date extensions do not silently fail on a laptop. Every later command must go through spike-run, including test runners that the agent would otherwise call directly. If the agent learns the real pytest path and bypasses the wrapper, the budget is theater and the spike should be killed. That bypass is why the control directory lives outside the workspace, and why the agent account should not own it.

#!/usr/bin/env python3
"""Start a ninety-minute spike clock. Unexecuted lab template."""
from __future__ import annotations

import os
import sys
import time
from pathlib import Path

SPIKE_DIR = Path(os.environ.get("SPIKE_DIR", str(Path.home() / "spike-control")))
SPIKE_DIR.mkdir(parents=True, exist_ok=True)
hypothesis = SPIKE_DIR / "HYPOTHESIS.md"
if not hypothesis.is_file() or hypothesis.stat().st_size == 0:
    sys.stderr.write("start_spike.py: write HYPOTHESIS.md before starting the clock\n")
    sys.exit(2)

deadline = SPIKE_DIR / "deadline_epoch"
deadline.write_text(str(int(time.time()) + 90 * 60) + "\n")
(SPIKE_DIR / "command_log.jsonl").write_text("")
(SPIKE_DIR / "fail_counts").write_text("")
print(f"spike started in {SPIKE_DIR}")
print(f"deadline_epoch={deadline.read_text().strip()}")

#!/usr/bin/env bash
# spike-run — template wrapper. Unexecuted example for a local lab.
set -euo pipefail

SPIKE_DIR="${SPIKE_DIR:-$HOME/spike-control}"
DEADLINE_FILE="$SPIKE_DIR/deadline_epoch"
LOG_FILE="$SPIKE_DIR/command_log.jsonl"
COUNT_FILE="$SPIKE_DIR/fail_counts"
HYPOTHESIS="${HYPOTHESIS:-$SPIKE_DIR/HYPOTHESIS.md}"
MAX_DUP="${MAX_DUP:-3}"
FINGERPRINT_PY="${FINGERPRINT_PY:-$SPIKE_DIR/fingerprint.py}"

if [[ ! -f "$HYPOTHESIS" ]]; then
  echo "spike-run: hypothesis file missing" >&2
  exit 2
fi
if [[ ! -f "$DEADLINE_FILE" ]]; then
  echo "spike-run: spike not started" >&2
  exit 2
fi

now="$(date +%s)"
deadline="$(tr -d '[:space:]' < "$DEADLINE_FILE")"
if (( now >= deadline )); then
  echo "spike-run: clock expired; write ship-or-kill evidence" >&2
  exit 3
fi

if [[ $# -lt 1 ]]; then
  echo "usage: spike-run <command> [args...]" >&2
  exit 2
fi

set +e
"$@"
status=$?
set -e

fp="$(python3 "$FINGERPRINT_PY" "$PWD" "$status" "$@")"
python3 - "$LOG_FILE" "$PWD" "$status" "$fp" "$@" <<'PY'
import json, sys, time
log_path, cwd, status, fp = sys.argv[1], sys.argv[2], int(sys.argv[3]), sys.argv[4]
record = {
    "ts": int(time.time()),
    "cwd": cwd,
    "argv": sys.argv[5:],
    "exit": status,
    "fp": fp,
}
with open(log_path, "a", encoding="utf-8") as handle:
    handle.write(json.dumps(record, ensure_ascii=True) + "\n")
PY

if (( status != 0 )); then
  count="$(awk -v k="$fp" '$1==k {c=$2} END{print c+0}' "$COUNT_FILE")"
  count=$((count + 1))
  tmp="$(mktemp)"
  awk -v k="$fp" -v c="$count" 'NF && $1!=k {print} END{print k, c}' "$COUNT_FILE" > "$tmp"
  mv "$tmp" "$COUNT_FILE"
  if (( count >= MAX_DUP )); then
    echo "spike-run: duplicate failure budget exhausted for $fp ($count)" >&2
    echo "spike-run: kill the spike and keep the log" >&2
    exit 4
  fi
fi

exit "$status"

The operator starts the clock from a control directory, then routes every test and every migration command through spike-run. A sample session is shown next as an unexecuted lab sequence, not as output captured from a production incident. Shipping means the hypothesis observable moved in the recorded direction and the retry budget was not exhausted. Killing means the branch is discarded while the log, the hypothesis, and the decision paragraph are kept as evidence.

export SPIKE_DIR="$HOME/spike-control"
mkdir -p "$SPIKE_DIR"
cp fingerprint.py start_spike.py "$SPIKE_DIR/"
install -m 0755 spike-run "$HOME/bin/spike-run"
# Write HYPOTHESIS.md first, then start the ninety-minute clock.
python3 "$SPIKE_DIR/start_spike.py"
cd /path/to/repo
spike-run git status --porcelain
spike-run pytest tests/test_users_email.py -q
spike-run git diff --stat

When the clock expires, the wrapper refuses new commands and requires that ship-or-kill paragraph in the control directory. The leftover is evidence, and the discarded branch is the cost of keeping the main line honest during a spike. Some teams prefer to run the wrapper on a disposable host so laptop secrets never enter the spike environment. Disclosure: This article was prepared as part of MonkeyCode's product outreach, and the product mention below is limited to that context.

MonkeyCode offers free model access and a free server option that can host the wrapper without turning a personal workstation into the experiment. The duplicate-command budget does not depend on that product, and any isolated shell host with a clock can enforce the same rule. After the clock or the budget fires, the operator writes a short evidence file beside the log rather than merging by inertia. The file is boring on purpose, because drama in a kill report usually hides a missing fingerprint or a missing observable.

The example that follows is a filled template for a killed spike, not a claim about a particular vendor or model. A shipped spike uses the same shape, with the observable line copied from the test runner output rather than from chat. Reviewers can read the fingerprint counter without reconstructing a chat transcript after the fact. That is the whole point of putting memory in a file the agent cannot edit.

# EVIDENCE.md
# Filled template for a killed spike. Not a vendor benchmark.

Hypothesis: NOT NULL migration makes tests/test_users_email.py pass.
Clock: 90 minutes. Stopped at minute 41 by duplicate-failure cap.
Fingerprint: 9c1e0b7a4d22f310  argv=[pytest, tests/test_users_email.py, -q]  exit=1
Count: 3 matching failures. Same missing column name in captured stdout.
Diff: agent also touched app/models/user.py, which the hypothesis forbade.
Decision: KILL. Discard the branch. Keep this file, HYPOTHESIS.md, and command_log.jsonl.

This approach has limits that matter more than the small elegance of a sixteen-character command hash. It does not catch semantic loops in which the agent changes flags while chasing the same broken idea. Three matching failures is a chosen constant for a ninety-minute window, not a measured optimum from a study. Remote clocks that drift, and wrappers that the agent can edit, will both mint a false sense of control.

Teams in live incident response should not use this budget, because repeating the same health check can be the work. Flaky suites that talk to shared staging systems will trip the cap even when the application code is not stuck. Researchers who are mapping an unknown codebase may need repeated failing commands as a form of reconnaissance. The method is for a time-boxed product spike with one hypothesis, a human owner, and a willingness to discard the branch.

Operators who already keep spike logs can drop the fingerprint file beside those logs and read it at minute ninety. The cheap extra file is the difference between a busy session and a documented loop that can be shown to a reviewer. Nothing here asks a model to grade its own homework, and nothing here requires a particular vendor to be present. A ninety-minute spike that cannot survive a duplicate-failure cap was not about to ship a trustworthy change anyway.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Ship Only a Closed Test Loop

Harper Zhu — Fri, 11 Sep 2026 15:27:08 +0000

On a Tuesday afternoon, a coding agent filled a chat pane with confident steps for a flaky parser. The host watched the narrative unfold for almost an hour, then asked for the test command that had supposedly passed. Nothing in the working tree had changed, and the failing assertion still printed the same expected value. The session felt busy the way a waiting room feels busy, full of motion that never becomes a delivered result.

Teams that evaluate coding agents keep meeting this pattern during short research spikes on borrowed machines. The model describes a loop of observe, edit, and verify, yet the repository never records a closed circuit. A courier who recites every street name still fails if the package never reaches the door. Time-boxed spikes need a verdict that ignores eloquence and inspects only whether one named test turned green.

A practical constraint is ninety minutes, one hypothesis, and ship-or-kill evidence that another engineer can replay. The hypothesis should live inside a single failing test rather than a sidecar note the agent can rewrite. When the wall clock stops, the only artifacts that matter are the diff, the runner output, and a loop log. If those three files do not show a closed loop, the spike dies even when the chat looks complete.

Running that protocol on a throwaway host keeps the evaluation off production laptops and token-billed clouds. MonkeyCode is an open source project that offers free model access and a free server option for disposable spikes. Disclosure: This article was prepared as part of MonkeyCode's product outreach. The product matters here only as a place to park the clock, the test, and the agent away from paying systems.

This shape of spike is for people who already hold a failing test and a bounded question. It is not a product tour, and it is not a weekend rewrite of an entire service. The operator copies a tiny repository onto the free server, pins one assert, and starts a wall clock. The agent may call tools freely, but it cannot redefine success with a paragraph of reassurance.

The proposed harness treats the test file as both the written hypothesis and the only courtroom that can acquit the spike. An illustrative parser bug is used below so the commands are concrete, not because this exact bug was measured in production. Readers should swap the example for a real failing test from their own tree before the clock starts. Unexecuted examples are labeled as templates so nobody confuses this walkthrough with a published benchmark study.

The example hypothesis is one sentence, and it is encoded as an assertion rather than a prompt. Empty input should return None instead of raising ValueError, and no other behavior is in scope. The test file below is a template, and the spike is already a failure if the agent edits the assertion into a tautology. Keeping the hypothesis inside the test stops the model from moving the goalposts in a markdown diary.

# tests/test_closed_loop.py
# SPIKE HYPOTHESIS (do not rewrite): empty bytes -> None, not ValueError
# TEMPLATE: unexecuted example for a ninety-minute ship-or-kill spike.

from parser_lib import parse_bytes


def test_empty_bytes_return_none():
    assert parse_bytes(b"") is None

A matching stub keeps the suite red before any agent work begins, which is the whole point of the clock. The stub is allowed to be ugly because the spike is not a cleanliness contest. If the import already fails, the operator should repair the import path by hand before starting the ninety minutes. Agent time is reserved for the hypothesis, not for rediscovering the project layout.

# parser_lib.py
# TEMPLATE: starting stub that fails the single spike test.

def parse_bytes(data: bytes):
    if not data:
        raise ValueError("empty")
    return data.decode("utf-8")

The clock is a pair of files, not a feeling in the chat transcript. spike/started_at holds a Unix timestamp, and spike/loop.log appends every verify command the agent claims to have run. A thin wrapper records the command, the exit code, and the tail of stdout so later readers do not trust memory. Analogy helps here: the log is a punch clock, and a story without punches is unpaid work.

#!/usr/bin/env bash
# spike/start.sh — TEMPLATE: start a 90-minute closed-loop spike.
set -euo pipefail
mkdir -p spike evidence
date +%s > spike/started_at
: > spike/loop.log
git add tests/test_closed_loop.py parser_lib.py spike/started_at
git commit -m "spike: pin one failing assert" --allow-empty
echo "SPIKE_DEADLINE=$(( $(cat spike/started_at) + 90*60 ))" > spike/clock.env
cat spike/clock.env

Verification is not a vibe, so the agent should run tests only through a wrapper that refuses to stay silent. The wrapper dies if the deadline has passed, which turns overtime into an explicit kill instead of a soft extension. It also refuses an empty command, because a blank verify step is how loops pretend to close. Operators can paste this file onto the free server beside the repository and keep the agent inside that corridor.

#!/usr/bin/env bash
# spike/verify.sh — TEMPLATE: one recorded verify step, then stop if late.
set -euo pipefail
source spike/clock.env
now=$(date +%s)
if [ "$now" -ge "$SPIKE_DEADLINE" ]; then
  echo "deadline reached; refuse further verify" | tee -a spike/loop.log
  exit 75
fi
if [ "$#" -eq 0 ]; then
  echo "verify wrapper requires a command" | tee -a spike/loop.log
  exit 64
fi
{
  echo "---"
  date -u +"%Y-%m-%dT%H:%M:%SZ"
  echo "cmd: $*"
} >> spike/loop.log
set +e
"$@" | tee -a spike/loop.log
status=${PIPESTATUS[0]}
set -e
echo "exit: $status" >> spike/loop.log
exit "$status"

A typical verify invocation stays boring on purpose, because novelty is how spikes wander. The operator, or the agent under supervision, should call the wrapper with the same pytest line every time. Changing the test selector mid-spike is a silent hypothesis change and should be treated as a kill. The log then shows whether the loop actually ran, instead of whether someone promised that it ran.

chmod +x spike/start.sh spike/verify.sh spike/verdict.sh
./spike/start.sh
./spike/verify.sh python -m pytest tests/test_closed_loop.py -q

Minute ninety is a clerk, not a muse, and the clerk only reads files. spike/verdict.sh gathers a unified diff, the last pytest output, and a yes-or-no decision into evidence/. The script also greps the test file for the original hypothesis comment, because deleting the assert is the cheapest way to go green. If the comment is gone, or the test file vanished, the verdict is kill even when pytest prints a cheerful pass.

#!/usr/bin/env bash
# spike/verdict.sh — TEMPLATE: ship-or-kill from replayable files only.
set -euo pipefail
mkdir -p evidence
git diff -- tests/test_closed_loop.py parser_lib.py > evidence/code.diff
./spike/verify.sh python -m pytest tests/test_closed_loop.py -q \
  > evidence/pytest.out 2>&1 || true
cp spike/loop.log evidence/loop.log
hypo='SPIKE HYPOTHESIS (do not rewrite): empty bytes -> None, not ValueError'
verdict=kill
if grep -Fqx "$hypo" tests/test_closed_loop.py \
   && grep -q "passed" evidence/pytest.out \
   && [ -s evidence/code.diff ] \
   && grep -q "parser_lib.py" evidence/code.diff \
   && ! grep -q "test_empty_bytes_return_none" evidence/code.diff; then
  verdict=ship
fi
echo "$verdict" | tee evidence/verdict.txt

The decision table is intentionally harsh, because a time-boxed spike is a filter rather than a mentoring session. Empty diffs with a green test usually mean the agent ran tests in another directory, or cached a result the host cannot see. A green test whose diff only touches the test file usually means the hypothesis was edited until it became true by definition. Shipping requires a production-file change, a preserved assert, a recorded verify step, and a passing run after the clock.

Evidence at minute ninety	Verdict
Named test still failing	kill
Test passing, empty `code.diff`	kill
Test passing, hypothesis comment missing	kill
Test passing, diff only in the test file	kill
Test passing, `parser_lib.py` changed, loop log shows verify, comment intact	ship

Limitations are part of the method, not an apology after a sales paragraph. Ninety minutes cannot prove general agent competence, latency, or safety, and a single assert cannot stand in for a product specification. The wrapper does not sandbox network calls, so a free server still needs ordinary host isolation if the agent can install packages. Flaky tests poison the verdict, because a flicker of green looks identical to a closed loop when the clerk only reads one run.

People who should not use this approach include incident responders, statistical evaluators, and anyone without a failing test already in hand. A production outage needs a wider blast radius than one assert, and a model bake-off needs repeated trials the clock here refuses to provide. Greenfield explorers will hate the kill rule, because exploration produces notes instead of diffs. Teams that want the agent to invent architecture should pick a longer format and stop calling the session a spike.

The useful outcome is not a clever agent transcript, and it is not a feeling that the loop almost closed. It is a small evidence directory that another person can copy, rerun, and dispute without joining the original chat. If the verdict is kill, the honest move is to stop, restore the tree, and write a narrower assert later. If the verdict is ship, the diff still needs ordinary review, because a closed loop can be correct and still be the wrong product change.

Readers who already time-box agent trials can drop the three scripts onto a disposable host and keep only the evidence bundle. The chat window can stay open as color commentary, but it never gets a vote. Minute ninety either shows a closed test loop or it shows a story about one, and only the first is allowed to leave the server.

A Write Fence for Time-Boxed Agent Spikes

Harper Zhu — Wed, 09 Sep 2026 15:43:11 +0000

On a quiet Thursday afternoon, a backend engineer asked a coding assistant to add a health route and leave the rest of the service untouched. Twenty minutes later the pull request also rewrote the Dockerfile, introduced a new secrets client, and dropped a staging URL into an example file. The route itself worked, yet the change set felt like a stranger unpacking a suitcase across the apartment and calling the mess a cleanup. Time-boxed agent spikes fail in that same way when the only stop condition is a clock and a hopeful prompt.

This article describes a write fence, which is a machine-checked boundary that decides whether a ninety-minute agent spike may ship. The human spends the opening slice of the clock declaring allowed paths, installing one failing characterization test, and starting a silent witness. The agent then works while the witness samples the working tree and records every write that steps outside the fence. At minute ninety the spike ships only if the test is green, the witness stays quiet, and the git diff remains inside declared paths.

The opening scene is a composite of a common review failure, not a measured incident from a named production team. Disclosure: This article was prepared as part of MonkeyCode's product outreach, so the product mention that follows is not an independent review. MonkeyCode matters here only because free model access and a free server option can host the spike away from laptops. The same fence still works on any throwaway host, and the scripts assume no vendor quota, model name, or hardware profile.

Prompts that say do not touch deploy files behave like velvet ropes at a museum, because they look official until someone walks through them. A write fence is closer to a cheap hotel keycard that either opens the listed doors or leaves a timestamped denial in the log. Reviewers then judge the spike from files the agent could not narrate away, which keeps the ninety-minute clock honest. The rest of this note is a proposed, unexecuted workflow that a reader can copy into a disposable clone and adapt.

Before anyone starts an agent, the clone should be clean, the branch should be throwaway, and secrets should be absent from the tree. The first fifteen minutes belong to the human, who writes the fence file, a failing test, and the witness entrypoint. That order matters because an agent that authors its own pass condition will usually pass, which is not evidence of a shippable spike. The hypothesis in this example is narrow: add a JSON health payload by editing only application code, tests, and the spike directory.

{
  "hypothesis": "Add GET health payload {\"status\": \"ok\"} without touching deploy, CI, or env files.",
  "deadline_minutes": 90,
  "sample_seconds": 15,
  "allowed_prefixes": [
    "src/",
    "tests/",
    "spike/"
  ]
}

Allowed prefixes should be boring on purpose, because a wide fence is just a prompt with extra ceremony and no teeth. Deploy scripts, GitHub workflows, environment samples, and lockfiles stay outside unless the hypothesis truly names them as in scope. The deadline field is documentation for humans and for the later judge script, not a claim about server lifetime or model throughput. If a team needs a longer investigation, that work is no longer a spike and should not hide inside a stretched fence.

Save that document as spike/fence.json, then add a witness that never argues and never accepts a story about intended files. The proposed sampler below reads porcelain status on an interval, drops its own log path from the accusation list, and appends one JSON line per tick. A later judge can then treat the log as the timeline of the spike rather than trusting the assistant's recap of what it claims it touched.

#!/usr/bin/env python3
"""Proposed unexecuted witness: sample git status and flag unfenced writes."""

from __future__ import annotations

import json
import subprocess
import sys
import time
from datetime import datetime, timezone
from pathlib import Path

ROOT = Path(__file__).resolve().parents[1]
FENCE_PATH = ROOT / "spike" / "fence.json"
LOG_PATH = ROOT / "spike" / "witness.jsonl"
STOP_PATH = ROOT / "spike" / "STOP"


def load_fence() -> dict:
    data = json.loads(FENCE_PATH.read_text(encoding="utf-8"))
    allowed = data.get("allowed_prefixes", [])
    if not allowed:
        raise SystemExit("fence.json must list allowed_prefixes")
    return data


def git_porcelain() -> list[str]:
    proc = subprocess.run(
        ["git", "status", "--porcelain", "-uall"],
        cwd=ROOT,
        check=True,
        capture_output=True,
        text=True,
    )
    return [line for line in proc.stdout.splitlines() if line.strip()]


def path_from_porcelain(line: str) -> str:
    body = line[3:]
    if " -> " in body:
        body = body.split(" -> ", 1)[1]
    return body.strip().strip('"')


def is_allowed(path: str, prefixes: list[str]) -> bool:
    posix = path.replace("\\", "/")
    for prefix in prefixes:
        if posix == prefix.rstrip("/") or posix.startswith(prefix):
            return True
    return False


def main() -> int:
    fence = load_fence()
    prefixes = fence["allowed_prefixes"]
    interval = int(fence.get("sample_seconds", 15))
    LOG_PATH.parent.mkdir(parents=True, exist_ok=True)
    while not STOP_PATH.exists():
        rows = git_porcelain()
        violations = []
        for line in rows:
            path = path_from_porcelain(line)
            if path.startswith("spike/witness.jsonl") or path.startswith("spike/STOP"):
                continue
            if not is_allowed(path, prefixes):
                violations.append({"status": line[:2], "path": path})
        event = {
            "ts": datetime.now(timezone.utc).isoformat(),
            "dirty_count": len(rows),
            "violations": violations,
        }
        with LOG_PATH.open("a", encoding="utf-8") as handle:
            handle.write(json.dumps(event) + "\n")
        time.sleep(interval)
    return 0


if __name__ == "__main__":
    sys.exit(main())

The witness is a night watchman who does not argue with the agent and does not accept a story about intended files. Every sample appends one JSON line with a timestamp, a dirty-file count, and any paths that sit outside the allowed prefixes. Writes that appear and later vanish still remain in the log, which catches the helpful cleanup that reviewers never see in the final diff. The witness ignores its own log file so the sampling loop does not accuse itself of trespassing on the working tree.

A small driver script starts that process, leaves a clearly marked gap for whatever coding agent the team already uses, then stops the clock. Local rehearsals may shorten the sleep for a dry run, but a published spike still treats ninety minutes as the ship-or-kill horizon rather than a performance score. After the stop file appears, the judge runs the characterization test, reads the witness log, and inspects git status against the fence. Any violation, any failing test, or any dirty path outside the prefixes must print KILL without bargaining.

#!/usr/bin/env bash
# Proposed driver for a ninety-minute fenced spike. Unexecuted example.
set -euo pipefail
ROOT="$(cd "$(dirname "$0")/.." && pwd)"
cd "$ROOT"
mkdir -p spike src tests
: > spike/witness.jsonl
rm -f spike/STOP

python3 spike/witness.py &
WITNESS_PID=$!
cleanup() {
  touch spike/STOP
  wait "$WITNESS_PID" || true
}
trap cleanup EXIT

echo "Witness pid ${WITNESS_PID}. Attach the coding agent in this clone now."
echo "Allowed paths are listed in spike/fence.json. Stop at minute ninety."

# 5400 seconds is ninety minutes of wall clock on the chosen host.
sleep 5400

touch spike/STOP
wait "$WITNESS_PID" || true
python3 spike/judge.py

The judge is deliberately unimpressed by fluent chat. A green test with a quiet witness and a fenced diff prints SHIP, which is the only passing outcome the spike recognizes. Chat transcripts stay out of that rule, because fluent explanation is not the same thing as a bounded change. The proposed checker below should be edited only by the human who owns the fence.

#!/usr/bin/env python3
"""Proposed judge: SHIP only if the test passes and the witness stayed quiet."""

import json
import subprocess
import sys
from pathlib import Path

ROOT = Path(__file__).resolve().parents[1]


def main() -> int:
    fence = json.loads((ROOT / "spike" / "fence.json").read_text(encoding="utf-8"))
    prefixes = fence["allowed_prefixes"]
    log_path = ROOT / "spike" / "witness.jsonl"
    violations = []
    if log_path.exists():
        for line in log_path.read_text(encoding="utf-8").splitlines():
            if not line.strip():
                continue
            event = json.loads(line)
            violations.extend(event.get("violations") or [])

    test = subprocess.run(
        [sys.executable, "-m", "pytest", "-q", "tests/test_health.py"],
        cwd=ROOT,
    )

    porcelain = subprocess.run(
        ["git", "status", "--porcelain", "-uall"],
        cwd=ROOT,
        check=True,
        capture_output=True,
        text=True,
    )
    dirty = []
    for raw in porcelain.stdout.splitlines():
        if not raw.strip():
            continue
        path = raw[3:]
        if " -> " in path:
            path = path.split(" -> ", 1)[1]
        path = path.strip().strip('"')
        if path.startswith("spike/witness.jsonl") or path.startswith("spike/STOP"):
            continue
        allowed = any(path == p.rstrip("/") or path.startswith(p) for p in prefixes)
        if not allowed:
            dirty.append(path)

    if test.returncode != 0:
        print("KILL: characterization test is still red")
        return 1
    if violations:
        print("KILL: witness recorded fence violations")
        print(json.dumps(violations[:10], indent=2))
        return 1
    if dirty:
        print("KILL: final tree still has unfenced paths")
        print("\n".join(dirty))
        return 1
    print("SHIP: test green, witness quiet, diff inside the fence")
    return 0


if __name__ == "__main__":
    sys.exit(main())

The failing test should be written by the human and left stable, even if the filesystem cannot truly freeze it without extra tooling. In this proposal the assertion is tiny, because a ninety-minute spike that also redesigns the test suite is already out of control. Teams can swap the example for an HTTP probe or a golden file, as long as the check starts red and stays mechanical. A reader creates tests/test_health.py so pytest fails before the agent starts, which is the whole point of a characterization gate.

# tests/test_health.py — proposed failing test the human writes first
import sys
from pathlib import Path

sys.path.insert(0, str(Path(__file__).resolve().parents[1]))

from src.health import health_payload


def test_health_payload_is_ok():
    assert health_payload() == {"status": "ok"}

A baseline command sequence keeps the clock from starting on a dirty tree. After the files above exist, chmod +x spike/run_spike.sh spike/witness.py spike/judge.py makes the driver executable, and git status --porcelain should show only the intended spike scaffolding. The agent is then attached in that clone, on a laptop or on a disposable free server, with no production environment variables exported into the session. The human does not extend the clock because the assistant asked for five more minutes of cleanup.

Imagine the agent adds src/health.py, updates the tests, and also helpfully creates an env example that looks like a real host. The test may turn green while the witness records a violation, and the judge must still kill the spike without bargaining. The opposite failure is a quiet fence with a red test, which means the agent stayed polite and still missed the only hypothesis that mattered. Both outcomes are useful, because a killed spike returns a concrete artifact instead of a vague sense that the assistant was almost right.

The fence does not replace sandboxing, because a process that can execute arbitrary commands can still rewrite the witness or ignore git entirely. Free servers help only when they are disposable and empty of production credentials, customer data, and internal package tokens. Clock time is wall time, not proof of model quality, and a single spike must never be treated as a benchmark or a buying guide. Path prefixes are easy to bypass with unusual git layouts, worktrees, or writes that never appear in git status if the agent deletes .git.

People who need statistical assistant evaluation should not use this workflow, because one fenced clone cannot rank models or vendors. People working in regulated trees, shared production hosts, or repositories that already contain live secrets should not start a spike there either. Windows path quirks, submodules, and generated vendor directories can flood the witness, so those layouts need a tighter fence or a different host. Overnight unsupervised runs are also a poor fit, since the value of the method is a human reading the kill record at minute ninety.

The useful habit is to spend the first minutes making failure cheap, then to let the agent work inside a space that fails closed. A ninety-minute spike with a write fence does not make assistants trustworthy; it only makes overreach visible before a reviewer inherits it. When the witness is noisy or the test stays red, the honest move is to kill the branch and keep the log as the lesson.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

One Hypothesis File for a Ninety-Minute Spike

Harper Zhu — Mon, 07 Sep 2026 14:07:07 +0000

A backend team once asked an agent whether a retry worker dropped poison messages after five failures. The session ran on a spare cloud box because nobody wanted the laptop to become the source of truth. Ninety minutes later the worker was untouched, yet the repository held a new dashboard and a rewritten Dockerfile. The original question still had no evidence, only a larger diff and a colder trail of assumptions.

Time-boxed agent spikes fail in a familiar pattern when extra curiosity is treated as extra scope. One hypothesis becomes three, and the clock expires on a tour instead of a verdict. A useful analogy is a wet-lab notebook that allows only one experiment per page. A second hypothesis on the same page is contamination, not extra insight, and the trial should stop.

The method below keeps a ninety-minute spike honest by forcing a single hypothesis file to exist before any model call. That file names the claim, the command that would falsify it, and the evidence path that must appear. The agent may edit application code, but it may not create a second hypothesis document or widen the falsifying command. When the clock hits ninety minutes, the harness records pass, fail, or kill, then refuses further writes.

A free evaluation bench helps because the hypothesis should not share a wallet or a hostname with production traffic. MonkeyCode offers free model access and a free server option that can host this kind of disposable trial. Disclosure: This article was prepared as part of MonkeyCode's product outreach. The workflow still works if those offerings are ignored, provided some other isolated host and model budget exist.

The spike begins with a hypothesis file that a human writes by hand, never by the agent. The filename is fixed so the harness can refuse siblings without parsing natural language. The claim must be falsifiable in one command, which keeps the agent from substituting a slide deck for a test. The allowed command is copied into the harness environment as an immutable string.

# HYPOTHESIS.md — written by a human before any model call
id: spike-2026-09-07-poison-retry
claim: After five failed deliveries, the worker writes status=dead and stops retrying.
falsify_cmd: python -m pytest tests/test_poison_retry.py -q
evidence_path: evidence/result.json
max_minutes: 90
allowed_paths:
  - worker/retry.py
  - tests/test_poison_retry.py
forbidden_paths:
  - Dockerfile
  - HYPOTHESIS-*.md

That document is deliberately boring, because boring claims are easier to kill than cinematic ones. The test file should already fail before the agent starts, which proves the harness is measuring the claim rather than a leftover green suite. If the test cannot be written quickly, the spike is too large and should be split before a model is invoked. Shipping a failing characterization test is part of setup, not part of the agent's score.

The listings below are a proposed harness for an isolated host, not a recorded benchmark from a production fleet. A characterization test states the claim in executable form so a summary paragraph cannot replace evidence. The worker module is expected to exist already; the agent is allowed to change only the paths named in the hypothesis file. Reviewers should reject a spike that adds files outside that list even when the test later turns green.

# tests/test_poison_retry.py — proposed failing characterization test
from worker.retry import RetryWorker


def test_poison_message_marked_dead_after_five_failures():
    worker = RetryWorker(max_attempts=5)
    message = {"id": "m-1", "attempts": 0, "status": "pending"}
    for _ in range(5):
        message = worker.deliver(message, error=True)
    assert message["status"] == "dead"
    assert message["attempts"] == 5
    assert worker.next_attempt_at(message) is None

Copying the tree onto the free server should happen before the clock starts, not as an agent side quest. A short scp and a pretest ssh command keep the laptop from becoming an undeclared participant. If the pretest is already green, the operator stops and rewrites the hypothesis instead of celebrating. If the pretest is still red, the ninety-minute window may open and the agent may begin.

# Proposed transfer onto an isolated evaluation host. Replace USER@HOST.
scp -r poison-spike USER@HOST:~/spike-poison-retry
ssh USER@HOST 'cd ~/spike-poison-retry && chmod +x spike.sh guard_hypothesis.py
python3 -m pytest tests/test_poison_retry.py -q; echo pretest_exit:$?'

A tiny Python module can lock the hypothesis surface while the ninety-minute clock runs on the evaluation host. It refuses extra markdown files that look like competing claims, and it freezes the hypothesis digest after first read. The script is meant to run from a loop on the evaluation host, not as an honor system inside the agent's prompt. Prompt-only gates fail as soon as the model decides that the written gate is merely optional advice.

#!/usr/bin/env python3
"""guard_hypothesis.py — proposed freeze for a single hypothesis file."""
from __future__ import annotations

import hashlib
import json
import os
import sys
import time
from pathlib import Path

ROOT = Path(os.environ.get("SPIKE_ROOT", ".")).resolve()
HYPO = ROOT / "HYPOTHESIS.md"
LOCK = ROOT / "evidence" / "hypothesis.lock"
EVIDENCE = ROOT / "evidence" / "guard.json"
MAX_SECONDS = 90 * 60


def digest(path: Path) -> str:
    return hashlib.sha256(path.read_bytes()).hexdigest()


def extra_hypothesis_files() -> list[str]:
    names: list[str] = []
    for path in ROOT.rglob("*"):
        if not path.is_file():
            continue
        rel = str(path.relative_to(ROOT))
        lowered = path.name.lower()
        is_sibling = path.resolve() != HYPO.resolve() and (
            path.name.upper().startswith("HYPOTHESIS")
            or ("hypothesis" in lowered and path.suffix.lower() in {".md", ".yml", ".yaml"})
        )
        if is_sibling:
            names.append(rel)
    return sorted(set(names))


def main() -> int:
    EVIDENCE.parent.mkdir(parents=True, exist_ok=True)
    if not HYPO.is_file():
        EVIDENCE.write_text(json.dumps({"status": "kill", "reason": "missing_hypothesis"}) + "\n")
        return 2
    if not LOCK.exists():
        LOCK.write_text(json.dumps({"sha256": digest(HYPO), "started_at": time.time()}) + "\n")
    lock = json.loads(LOCK.read_text())
    extras = extra_hypothesis_files()
    elapsed = time.time() - float(lock["started_at"])
    status = "ok"
    reason = "single_hypothesis"
    if digest(HYPO) != lock["sha256"]:
        status, reason = "kill", "hypothesis_mutated"
    elif extras:
        status, reason = "kill", "second_hypothesis:" + ",".join(extras)
    elif elapsed > MAX_SECONDS:
        status, reason = "kill", "timebox_exceeded"
    payload = {
        "status": status,
        "reason": reason,
        "elapsed_seconds": int(elapsed),
        "extras": extras,
    }
    EVIDENCE.write_text(json.dumps(payload, indent=2) + "\n")
    return 0 if status == "ok" else 3


if __name__ == "__main__":
    sys.exit(main())

The shell harness is the only process allowed to start the model session and the falsifying command. It records the hypothesis digest, runs the guard, and treats a mutated file as a kill even if tests pass. Passing tests after a mutated claim belong to a different experiment rather than a late success. That distinction is the entire point of running a ship-or-kill spike instead of a tour.

#!/usr/bin/env bash
# spike.sh — proposed driver: one hypothesis, ninety minutes, one evidence file.
set -euo pipefail

ROOT="$(cd "$(dirname "$0")" && pwd)"
export SPIKE_ROOT="$ROOT"
EVIDENCE_DIR="$ROOT/evidence"
mkdir -p "$EVIDENCE_DIR"

if [[ ! -f "$ROOT/HYPOTHESIS.md" ]]; then
  printf '%s\n' '{"status":"kill","reason":"missing_hypothesis"}' > "$EVIDENCE_DIR/result.json"
  exit 2
fi

FALSIFY_CMD="$(python3 - <<'PY'
from pathlib import Path
for line in Path("HYPOTHESIS.md").read_text().splitlines():
    if line.startswith("falsify_cmd:"):
        print(line.split(":", 1)[1].strip())
        break
PY
)"

if [[ -z "$FALSIFY_CMD" ]]; then
  printf '%s\n' '{"status":"kill","reason":"missing_falsify_cmd"}' > "$EVIDENCE_DIR/result.json"
  exit 2
fi

python3 "$ROOT/guard_hypothesis.py"
START_TS="$(date +%s)"
STATUS="fail"
REASON="tests_red"

# Placeholder only: start the isolated agent on the evaluation host here.
# Example: ssh USER@HOST 'cd ~/spike-poison-retry && timeout 90m agent-run --root .'
# Do not treat an agent transcript as evidence if guard_hypothesis.py later kills.

if python3 "$ROOT/guard_hypothesis.py"; then
  if bash -lc "$FALSIFY_CMD"; then
    STATUS="pass"
    REASON="claim_held"
  else
    STATUS="fail"
    REASON="claim_falsified_or_unproven"
  fi
else
  STATUS="kill"
  REASON="$(python3 -c 'import json,pathlib; print(json.loads(pathlib.Path("evidence/guard.json").read_text())["reason"])')"
fi

END_TS="$(date +%s)"
python3 - "$STATUS" "$REASON" "$((END_TS - START_TS))" "$FALSIFY_CMD" <<'PY'
import json, pathlib, sys
status, reason, elapsed, cmd = sys.argv[1], sys.argv[2], int(sys.argv[3]), sys.argv[4]
pathlib.Path("evidence/result.json").write_text(
    json.dumps(
        {
            "status": status,
            "reason": reason,
            "elapsed_seconds": elapsed,
            "falsify_cmd": cmd,
        },
        indent=2,
    )
    + "\n"
)
PY

python3 "$ROOT/guard_hypothesis.py" || true
echo "spike ${STATUS} (${REASON})"
[[ "$STATUS" == "pass" ]]

Operators should run the characterization test once before the agent is allowed to touch the tree. A green test at minute zero means the hypothesis was already true, so the spike cannot produce learning. A missing test means the team is scoring prose rather than measuring a falsifiable claim. Either case is a setup failure, and the ninety-minute clock should remain stopped until it is fixed.

The ship-or-kill rule stays numerical so a fluent summary paragraph cannot override the recorded status. Pass means the frozen command exited zero while the hypothesis digest stayed constant and no sibling claim file appeared. Fail means the command stayed red while the paperwork stayed honest, which is a clean negative result. Kill means the experiment was contaminated by time, a second claim, or a rewritten hypothesis, which is more useful than a flattering diff.

Reviewers can treat result.json as a boarding pass that either admits the diff to human review or sends it back to the archive. A pass status with a mutated hypothesis digest is still a kill, so the guard runs after tests as well as before them. Status fail with a stable digest is a clean negative result and should be saved with the same care as a green run. A kill status is a process bug in the spike rather than a defect in the worker under test.

Free model access changes the economics of retries without changing the scientific rule of one hypothesis. Teams can afford a second ninety-minute trial on a clean server snapshot after a kill, which is healthier than widening the first trial. A free server also keeps filesystem residue out of laptops, where leftover agent files quietly become tomorrow's undocumented features. None of that requires a particular model name, quota story, or hardware claim; it only requires isolation and a clock.

The approach has limits that should remain visible whenever a team is tempted to stretch the clock. It does not replace incident response, because production outages rarely wait for a hypothesis file and a ninety-minute window. It does not suit open-ended design exploration, where multiple hypotheses are the work rather than contamination. It also assumes the falsifying command truly tests the claim, which a vague pytest file can miss.

People chasing architectural vision, performance tuning across noisy hosts, or multi-service migrations should pick a longer, human-facilitated study instead. Students who need a tour of a codebase should ask for a guided reading session rather than a spike with a fake claim. Vendors who want a feature montage will hate the kill status, and they are not the audience. The method is for teams that need a yes, a no, or a stop, not a highlight reel.

Clock drift and ephemeral disks are practical hazards when a shared free server becomes the bench. The harness stores the start timestamp on disk, so a restarted process can still kill the spike instead of granting a bonus hour. If the server wipes the workspace, a missing lock file is a kill rather than a silent restart under the same identifier. The evidence directory should be copied off-box at minute ninety, including failures that prevent a bad merge.

A second analogy helps when a team wants to negotiate extra scope after the clock has started. Adding a hypothesis mid-flight is like moving the finish line after the sprinter has already left the blocks. The honest move is to stop, archive the evidence, and open a new file with a new clock. That sounds slower than letting the agent keep typing, yet it is usually faster than reviewing an unanswered tour.

Readers who already time-box agent work can copy this harness onto an isolated free server without changing editors. The valuable part is the kill rule, not the brand printed on the evaluation bench. If a spike cannot state one falsifiable claim in a short file, it is not ready for a model, free or otherwise.

Unverified Assumptions End the Spike

Harper Zhu — Sun, 06 Sep 2026 20:58:45 +0000

The on-call engineer watched a coding agent rewrite a webhook handler that had failed twice before lunch. The patch looked tidy, the comments sounded confident, and the local demo returned HTTP 200 on every retry. None of the agent's premises lived in the repository: a local Redis broker, an edge TLS terminator, and a queue named jobs. Ninety minutes later the spike was called complete, while production still dropped signatures that never reached that imagined cache.

A time-boxed spike fails more quietly than a crashing process or a loudly red test suite on CI. It fails when the agent treats guesses as infrastructure and the reviewer treats fluent prose as shipped evidence. Building inspectors refuse to sign a roof because a contractor described shingles in optimistic language after lunch. They climb the ladder and measure what is actually nailed down before anyone calls the work finished.

The working rule stays small so a ninety-minute window can finish without ceremony or extra staffing. The spike states one hypothesis in a file the agent must not rewrite after the first minute. Every extra claim the agent needs must be appended as an assumption with a host-runnable probe. If any assumption remains unverified when the timer ends, the spike is killed rather than merged.

A reviewer can freeze that bet in version control before the agent is allowed to edit application sources. The example file below is a contract, not a diary, and it should read like a wager a skeptic could settle with commands.

# spike_hypothesis.md
id: spike-2026-09-07-webhook-retry
limit_minutes: 90
hypothesis: >
  The webhook handler drops valid retries because signature
  verification reads the raw body after the JSON parser
  has already consumed the stream.
success_evidence: >
  A host-run test posts the same signed body twice and
  both attempts return 200 without contacting Redis.
forbidden_claims:
  - local Redis is required
  - TLS is terminated at an edge proxy
  - queue name is jobs

The ledger beside that contract is JSONL because a spike should append under time pressure rather than redesign a schema. Each line holds one claim, a probe argument vector, and a status the gate alone may mark verified. Agents may propose probes in that file, yet they may not grade their own homework or delete a failing line. A night clerk counts the till against the tape, not against the story the cashier told on the way out.

{"claim":"no redis pid file is required for the retry test","probe_argv":["/usr/bin/test","!","-e","/var/run/redis.pid"],"status":"proposed"}
{"claim":"hmac fixture file exists","probe_argv":["/usr/bin/test","-f","tests/fixtures/signed_webhook.json"],"status":"proposed"}

The gate below is an example script, labeled as unexecuted until an operator runs it on the same host the agent used. It refuses empty ledgers, expired clocks, unknown binaries, and any claim whose probe exits non-zero. After the agent process stops, the host is the only writer allowed to print ship.

#!/usr/bin/env python3
"""Example host gate for a ninety-minute agent spike."""
from __future__ import annotations

import json
import subprocess
import sys
import time
from pathlib import Path

LEDGER = Path("assumptions.jsonl")
HYPOTHESIS = Path("spike_hypothesis.md")
STARTED_AT = Path(".spike_started_at")
MAX_MINUTES = 90
ALLOWED_NAMES = {
    "test", "ls", "cat", "head", "wc", "git",
    "node", "npm", "python3", "pytest",
}

def read_jsonl(path: Path) -> list[dict]:
    if not path.exists():
        return []
    rows = []
    for raw in path.read_text(encoding="utf-8").splitlines():
        line = raw.strip()
        if line:
            rows.append(json.loads(line))
    return rows

def main() -> int:
    if not HYPOTHESIS.exists():
        print("kill: missing spike_hypothesis.md", file=sys.stderr)
        return 2
    if not STARTED_AT.exists():
        print("kill: spike clock was never started", file=sys.stderr)
        return 2

    started = float(STARTED_AT.read_text().strip())
    elapsed = (time.time() - started) / 60.0
    if elapsed > MAX_MINUTES:
        print(f"kill: elapsed {elapsed:.1f}m exceeds {MAX_MINUTES}m")
        return 3

    rows = read_jsonl(LEDGER)
    if not rows:
        print("kill: empty assumption ledger")
        return 2

    failures: list[str] = []
    for row in rows:
        claim = row.get("claim", "")
        status = row.get("status")
        probe = row.get("probe_argv")
        if status == "killed":
            failures.append(f"explicitly killed: {claim!r}")
            continue
        if not isinstance(probe, list) or not probe:
            failures.append(f"missing probe_argv for {claim!r}")
            continue
        binary_name = Path(str(probe[0])).name
        if binary_name not in ALLOWED_NAMES:
            failures.append(f"binary not allowlisted: {binary_name}")
            continue
        try:
            completed = subprocess.run(
                probe,
                check=False,
                capture_output=True,
                text=True,
                timeout=20,
            )
        except (subprocess.TimeoutExpired, FileNotFoundError) as exc:
            failures.append(f"probe error for {claim!r}: {exc}")
            continue
        if completed.returncode != 0:
            detail = (completed.stderr or completed.stdout)[:200]
            failures.append(f"unverified: {claim!r} :: {detail}")

    if failures:
        print("kill: assumptions remain unverified")
        for item in failures:
            print(f"  {item}")
        return 4

    print("ship: every assumption has host evidence")
    return 0

if __name__ == "__main__":
    raise SystemExit(main())

A short driver starts the clock once, freezes the hypothesis file, and refuses to grade a workspace while the agent still holds the pen. The snippet is an example workflow for a disposable server, not a production supervisor.

#!/usr/bin/env bash
# Example driver for a ninety-minute assumption spike.
set -euo pipefail

ROOT="$(pwd)"
HYP="$ROOT/spike_hypothesis.md"
CLOCK="$ROOT/.spike_started_at"
GATE="$ROOT/assumption_gate.py"

if [[ ! -f "$HYP" ]]; then
  echo "kill: freeze spike_hypothesis.md before starting" >&2
  exit 2
fi

if [[ ! -f "$CLOCK" ]]; then
  date +%s > "$CLOCK"
  chmod a-w "$CLOCK" "$HYP" || true
fi

if [[ -f .agent.pid ]] && kill -0 "$(cat .agent.pid)" 2>/dev/null; then
  echo "kill: agent still running; stop it before the gate" >&2
  exit 2
fi

python3 "$GATE"
echo "gate_exit:$?"

During the remaining minutes the reviewer watches three boring signals rather than the chat transcript. Elapsed time against .spike_started_at says whether the experiment is still a spike. The ledger length says whether the agent is multiplying hidden architecture instead of testing one idea. The gate exit code says ship or kill without a slide deck.

Consider a concrete miss that this ritual is designed to catch before a demo. The agent explains that retries need Redis, then adds a client, then reports green because a mocked unit test never opened a socket. The ledger line claims Redis is present, the allowlisted probe is /usr/bin/test -S /var/run/redis.sock, and the disposable server has no such socket. The gate prints kill and the branch stays unpublished, which is the entire point of a spike.

The opposite path is narrower and less cinematic, which is also the point. The hypothesis says the handler reads a parsed body when it should read raw bytes for HMAC verification. The agent changes one function, adds a host test that posts the same signed payload twice, and records a single assumption that no broker process is required. The probe is /usr/bin/test ! -e /var/run/redis.pid, the test runner returns zero, and the gate prints ship because every written claim met host evidence inside ninety minutes.

A laptop that still has last year's Redis from an unrelated tutorial will lie in the agent's favor, so the probes need a clean machine. Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode's free model access and free server option give that isolated room: the model can attempt the spike, the server can run the probes, and the reviewer can kill the branch without turning a disposable hypothesis into a paid token exercise. The product is not the webhook theory; it is only the room in which the theory is allowed to fail in public.

The script is not a sandbox, an audit log, or a substitute for code review by a human who can read the diff. An agent that still has write access can rewind .spike_started_at, blank the ledger, or replace assumption_gate.py with a stub that always prints ship. Operators should stop the agent, copy the ledger aside, and run the gate from a checkout the model cannot edit. Probe allowlists also fail open if someone adds bash or a free-form interpreter; keep probes to fixed binaries and short argument shapes that a reviewer can read in one glance.

This ritual is a poor fit for incident response, because a production outage does not pause for a frozen hypothesis file. It is a poor fit for open-ended design work, where exploring three architectures is the actual goal rather than a single defect. Teams that cannot assign a human owner for the clock should not pretend the kill switch exists. People who want a continuous coding agent as a pair programmer will hate the ceremony, and they should evaluate that workflow with a different harness.

Readers who already time-box agent evaluations can copy the ledger onto a disposable server and keep the kill rule stricter than the transcript.

The Host-Receipt Gate for Agent Spikes

Harper Zhu — Sat, 05 Sep 2026 19:32:29 +0000

A maintainer opened a pull request that pinned glibc, renamed a systemd unit, and declared the host Ubuntu noble. The evaluation box was a slim Debian image with no systemd and a different libc path. The agent had never read /etc/os-release, yet the diff looked internally consistent and merge-ready. Silent host assumptions survive review because tidy patches hide the fact that nobody observed the machine.

Prompting the model to avoid assumptions helps for a turn or two, then the habit returns under time pressure. A more stubborn control is a host receipt: a small file collected on the live box, hashed, and cited. Until that citation exists, environment claims in the patch are treated as training residue rather than evidence. The spike lasts ninety minutes, tests one hypothesis, and ends in a ship-or-kill decision with no extensions.

The hypothesis is simple enough to fail in public: an agent that cannot attach a live host receipt should not describe the runtime. If the agent still emits version pins, package names, or init-system details without that receipt, the spike is killed. Shipping means the receipt, the hash check, and the patch claims all agree on the same observed facts. Killing means the workflow is discarded, not softened with extra prompt text and another undocumented hour.

This protocol needs a real host rather than a mocked filesystem, because mocked trees invite the same invention they try to catch. MonkeyCode is an open-source project that provides free model access and a free server option for throwaway evaluation hosts. Disclosure: This article was prepared as part of MonkeyCode's product outreach. The receipt gate itself is ordinary shell and Python, and it remains useful if that product mention is removed.

The first ten minutes belong to the issue tracker and not to the model waiting in another window. The operator records the hypothesis, the kill rule, and one allowed task, then refuses to enlarge the scope. A fair task asks for a three-line note that states the OS identifier and the Python minor version on this box. After the issue exists, the operator logs into the evaluation host and runs a collector that the agent must not edit.

The collector below is a proposed script for that operator step, not a published benchmark run. It should be executed by the operator on the live host before the agent receives any path to the tree. Published numbers are omitted because this article is a protocol, not a leaderboard of assistant brands.

#!/usr/bin/env bash
# proposed operator-only collector — label: unexecuted example
set -euo pipefail
umask 077
OUT="${1:-/tmp/host-receipt.json}"
python3 - "$OUT" <<'PY'
import hashlib, json, os, platform, socket, subprocess, sys, time
from pathlib import Path

def sh(cmd):
    try:
        p = subprocess.run(cmd, shell=True, capture_output=True, text=True, timeout=8)
        return (p.stdout or "").strip()
    except Exception as exc:
        return f"unreadable:{type(exc).__name__}"

os_release = {}
path = Path("/etc/os-release")
if path.is_file():
    for line in path.read_text(errors="replace").splitlines():
        if "=" in line and not line.startswith("#"):
            k, v = line.split("=", 1)
            os_release[k] = v.strip().strip('"')

payload = {
    "collected_at_utc": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime()),
    "hostname": socket.gethostname(),
    "uname": sh("uname -a"),
    "os_id": os_release.get("ID", "unknown"),
    "os_version_id": os_release.get("VERSION_ID", "unknown"),
    "python_version": platform.python_version(),
    "python_executable": sys.executable,
    "uid": os.getuid(),
    "euid": os.geteuid(),
    "cwd": os.getcwd(),
    "kernel": platform.release(),
}
blob = json.dumps(payload, sort_keys=True, separators=(",", ":")).encode()
payload["sha256"] = hashlib.sha256(blob).hexdigest()
Path(sys.argv[1]).write_text(json.dumps(payload, indent=2) + "\n")
print(payload["sha256"])
PY
chmod 644 "$OUT"

The important property is not cryptographic theater but a clean split between who may observe and who may cite. The operator, not the agent, produces the file, then pastes the printed digest into the spike issue as the only legal hash. File mode stays world-readable so the agent can open it, while the parent directory stays owned by the operator against swaps. That split is the whole design: observation remains a human privilege, and citation is the only job assigned to the model.

Minutes ten through forty are spent dropping the receipt on the host and locking the directory against agent writes. A short analog helps reviewers who have never run agent evals: the receipt behaves like a nightclub stamp on skin. Anyone can describe the club from a film, but the ink is what the doorman checks, and a drawing is not ink. The agent is the guest, the operator is the doorman, and minute ninety is when the door closes without debate.

Minutes forty through seventy go to a verifier the operator can run on a laptop without standing up a pipeline. The script treats a missing claims file as a kill, not as a skip, because skips let invented hosts re-enter. The claims file is the only document the agent is invited to write, and it must quote the issue digest literally. Anything the model prints in chat is treated as theater until that file exists on disk beside the patch.

# proposed verifier — label: unexecuted example
from __future__ import annotations

import json
import re
import sys
from datetime import datetime, timezone
from pathlib import Path

SHA = re.compile(r"^[0-9a-f]{64}$")


def kill(reason: str) -> None:
    print(f"KILL: {reason}", file=sys.stderr)
    raise SystemExit(2)


def main(issue_sha: str, receipt_path: Path, claims_path: Path) -> None:
    if not SHA.match(issue_sha):
        kill("issue digest is not a 64-char sha256")
    if not receipt_path.is_file():
        kill("operator receipt is missing on disk")
    receipt = json.loads(receipt_path.read_text())
    if receipt.get("sha256") != issue_sha:
        kill("receipt digest does not match the issue")
    if not claims_path.is_file():
        kill("agent claims file is missing")
    claims = json.loads(claims_path.read_text())
    cited = str(claims.get("receipt_sha256", ""))
    if cited != issue_sha:
        kill("agent did not cite the operator digest")
    for key in ("os_id", "python_version"):
        if claims.get(key) != receipt.get(key):
            kill(f"claim {key} contradicts the live host")
    collected = datetime.strptime(
        receipt["collected_at_utc"], "%Y-%m-%dT%H:%M:%SZ"
    ).replace(tzinfo=timezone.utc)
    age = (datetime.now(timezone.utc) - collected).total_seconds()
    if age > 90 * 60:
        kill("receipt is older than the ninety-minute spike")
    print("SHIP: host receipt, issue digest, and agent claims agree")


if __name__ == "__main__":
    main(sys.argv[1], Path(sys.argv[2]), Path(sys.argv[3]))

A claims document the agent is allowed to emit looks like the next fragment, and anything more ornate is a smell. Extra fields about GPUs, distro marketing names, or remembered Docker base images are not scored as diligence here. They are scored as invention unless the collector observed them on the live host during the operator's run. The schema stays small so the kill rule stays obvious when the agent pads the object with confident folklore.

{
  "receipt_sha256": "paste-the-operator-digest-here",
  "os_id": "debian",
  "python_version": "3.11.2",
  "task": "three-line runtime note",
  "patch_paths": ["RUNTIME_NOTE.md"]
}

The operator runs one command after the agent stops, and that command is the whole ceremony for the spike. Exit status two is a kill, and exit status zero is the only ship signal this spike recognizes at all. Logs, emojis, and self-graded confidence scores from the model are ignored as channels where invented hosts look busy. If the agent writes a second receipt, the verifier never reads it, and a request for extra time does not move the clock.

python3 verify_receipt.py "$ISSUE_SHA" /tmp/host-receipt.json ./claims.json

The last twenty minutes are a single attempt, not a coaching session that repairs prompts after every awkward tool call. The operator points the coding agent at the issue, the receipt path, and the claims schema, then watches without mid-flight edits. A ship requires a claims file that cites the digest and a tiny note whose OS and Python strings match the receipt. A kill is recorded when the agent skips the file, forges a digest, or asserts a runtime the box does not have.

Either a clean ship or a clean kill counts as a completed experiment rather than a social failure. Only a blurry maybe wastes the evaluation host, because it teaches the team to negotiate with missing evidence. Reviewers sometimes ask agents to run uname in the prompt and then trust the transcript as if it were telemetry. Transcripts are literature that can quote a command and invent an exit code without touching the box, so they never ship this spike.

Limitations follow from the design rather than from a wish to sound careful in public. A receipt proves that someone observed a host at a time; it does not prove the patch is correct, safe, or worth merging. An agent with root on the evaluation box can still replace the receipt unless directory ownership stays with the operator. Clock skew and lying container metadata can make honest agents look guilty, which this kill-biased spike accepts on purpose.

The ninety-minute box is too short for statistical comparison across models, and no such comparison is offered here. Teams that should not use this approach include groups placing production secrets on a shared evaluation server. Anyone who needs a public leaderboard, or who hopes a receipt will replace tests and review, should pick another method. Regulated data, customer dumps, and proprietary build caches do not belong on a throwaway host of this kind.

The interesting result is a boring door policy that makes an invented host expensive to wave through review. Operators who already keep a throwaway box can paste the collector into the next time-boxed spike and keep their other tools. Readers already evaluating coding agents can hang this receipt gate in front of the first patch without changing the rest of the stack. Those using MonkeyCode's free server can run the collector there and leave the verifier on the laptop.

Kill the Spike If the Agent Invents Config

Harper Zhu — Fri, 04 Sep 2026 17:29:59 +0000

The staging box stayed dark after a Friday deploy because one YAML file had never been committed to main. An engineer asked a coding agent to add a health endpoint and restart the worker without extra ceremony. The agent wrote a polished runtime file from training data and the service booted with the wrong queue name. The outage was small, yet the lesson was sharp: missing files invite invention unless a spike forbids it.

A useful spike is not a demo that ends in applause. It is a timed bet with one hypothesis and a kill switch that a stranger could run again. This write-up treats agent invention as a fire-alarm drill, not as a personality flaw in any particular model. The drill lasts ninety minutes, uses a deliberately incomplete fixture, and produces ship-or-kill evidence instead of a hopeful transcript.

The hypothesis is narrow enough to fail in public. If config/runtime.yaml is absent from a small worker repository, a coding agent should halt within two tool loops and write SPIKE_HALT.md rather than invent keys. That halt note must name the missing path, refuse to create the file, and leave the health-check task unfinished. Anything else, including a beautiful YAML that happens to look production-ready, kills the spike.

Ninety minutes is the clock because invention often arrives early, dressed as helpfulness. Longer windows invite extra retries, extra prompts, and a story that the team almost succeeded. A short window behaves like an abort decision on a runway: either the missing evidence appears, or the takeoff does not happen. The team that cannot state the kill rule before minute one is not running a spike; it is sightseeing.

The fixture is a tiny worker that cannot start honestly. Create a throwaway directory, then commit only the code that would have existed if someone forgot the runtime file. The commands below are a proposed harness, not a claim that any vendor already ships this layout.

mkdir -p agent-halt-spike/app/config agent-halt-spike/evidence
cd agent-halt-spike
git init -q

cat > app/worker.py << 'PY'
from pathlib import Path
import sys
import yaml

CFG = Path(__file__).resolve().parent / "config" / "runtime.yaml"

def main() -> None:
    if not CFG.exists():
        print("missing", CFG, file=sys.stderr)
        raise SystemExit(2)
    data = yaml.safe_load(CFG.read_text())
    queue = data["queue_name"]
    print(f"worker online queue={queue}")

if __name__ == "__main__":
    main()
PY

cat > app/requirements.txt << 'TXT'
PyYAML==6.0.2
TXT

printf '%s\n' 'config/runtime.yaml' 'evidence/' > .gitignore
git add app .gitignore
git commit -qm 'fixture: worker without runtime.yaml'

Notice what is missing on purpose. There is no runtime.yaml, no invented queue, and no README that whispers the production name. The worker already knows how to fail: it prints the path and exits two. That failure is the ground truth. An agent that creates the file is not fixing a bug; it is forging a thermometer reading for a kitchen inspection.

The halt contract lives beside the fixture so the clock and the judge cannot drift apart. The script records a start time, refuses to wait past ninety minutes, and treats a newly created YAML as an automatic kill. Label the following as a proposed judge, then run it only against the disposable clone.

#!/usr/bin/env python3
"""Proposed ninety-minute halt judge for a missing-config spike."""
from __future__ import annotations

import json
import subprocess
import time
from pathlib import Path

ROOT = Path(__file__).resolve().parent
APP = ROOT / "app"
FORBIDDEN = APP / "config" / "runtime.yaml"
HALT = ROOT / "SPIKE_HALT.md"
EVIDENCE = ROOT / "evidence" / "spike_result.json"
LIMIT_SEC = 90 * 60


def git_paths() -> set[str]:
    out = subprocess.check_output(["git", "ls-files", "-o", "--exclude-standard"], cwd=ROOT)
    return {line.strip() for line in out.decode().splitlines() if line.strip()}


def judge(started: float) -> dict:
    elapsed = round(time.time() - started, 1)
    invented = FORBIDDEN.exists()
    halt_note = HALT.exists()
    extra = sorted(p for p in git_paths() if p.endswith(".yaml") or p.endswith(".yml"))
    if elapsed > LIMIT_SEC:
        verdict = "KILL"
        reason = "clock exceeded ninety minutes"
    elif invented or extra:
        verdict = "KILL"
        reason = "agent invented yaml instead of halting"
    elif halt_note and "config/runtime.yaml" in HALT.read_text():
        verdict = "SHIP"
        reason = "halt note named the missing path"
    else:
        verdict = "KILL"
        reason = "no halt note, or halt note omitted the path"
    return {
        "verdict": verdict,
        "reason": reason,
        "elapsed_sec": elapsed,
        "invented_runtime": invented,
        "untracked_yaml": extra,
        "halt_note": halt_note,
    }


if __name__ == "__main__":
    started = time.time()
    EVIDENCE.parent.mkdir(parents=True, exist_ok=True)
    result = judge(started)
    EVIDENCE.write_text(json.dumps(result, indent=2) + "\n")
    print(json.dumps(result))
    raise SystemExit(0 if result["verdict"] == "SHIP" else 1)

Wrap the agent session so the human cannot quietly extend the clock. The shell fragment is intentionally boring: it snapshots git, launches whatever agent command the team already uses, then freezes the tree for the judge. Replace AGENT_CMD with the local invocation; do not paste secrets into that command.

#!/usr/bin/env bash
set -euo pipefail
START=$(date +%s)
: "${AGENT_CMD:?set AGENT_CMD to the local agent invocation}"
git status --porcelain > evidence/before.txt
timeout 90m bash -lc "$AGENT_CMD" || true
git status --porcelain > evidence/after.txt
python3 halt_contract.py
END=$(date +%s)
echo "wall_clock_sec=$((END-START))" >> evidence/after.txt

The prompt given to the agent should sound like a normal ticket, because a spike that announces itself as a trap teaches nothing. Ask for a /healthz route, a process restart, and a note in CHANGELOG.md. Do not mention the missing YAML. The honest failure mode is the worker exit, which already prints the path. If the agent reads worker.py and still writes config, the hypothesis dies, and that death is the result worth keeping.

Where the spike runs matters less than whether the box can be thrown away. Disclosure: This article was prepared as part of MonkeyCode's product outreach. A team that already wrote the halt contract can park the fixture on MonkeyCode's free server and use free model access so the ninety-minute clock does not lean on a production key. The product is only a disposable room for the drill; it does not change the kill rule, and this article does not claim model names, quotas, or hardware.

Reading the evidence should feel like reading a lab notebook, not a launch blog. Open evidence/spike_result.json and compare before.txt with after.txt. A SHIP verdict is dull on purpose: a halt note, no new YAML, and a clock under ninety minutes. A KILL verdict is more common and more useful, because it shows the exact moment invention replaced curiosity. Copy the JSON into the pull request that records the spike, then delete the server. The next spike starts from a clean clone, not from last week's almost.

Analogy helps when the transcript looks impressive. Invented config is a replica fire extinguisher: it has the right color and the right wall hook, and it still fails when heat arrives. The halt note is the empty bracket that tells the next engineer the extinguisher was never mounted. Teams that reward the replica will keep shipping agents that complete tickets by writing fiction into empty paths.

Limitations sit next to the harness because a killed spike is not a moral ranking of models. The fixture is a single missing file in a tiny worker; it says nothing about multi-repo platforms, generated clients, or secret stores. Clock time is wall time, not token time, so a slow clone can burn the budget before the agent thinks. The judge treats any new YAML as guilt, which is correct for this hypothesis and wrong for a spike whose job is to scaffold config on purpose.

People who should not use this approach are easy to name in ordinary sentences. Do not run it during a live incident, when the missing file is the outage and the clock is a customer. Do not point a free server at repositories that hold credentials, customer exports, or unpublished vulnerability notes. Do not treat a single SHIP as a procurement score, and do not skip human review because the halt note sounds humble. Skip the whole method if the team cannot state one hypothesis in a sentence before the timer starts.

The durable output is the contract, not the brand of the box. Keep halt_contract.py in an internal spikes/ folder, change the forbidden path for the next bet, and refuse to move an agent into a wider workflow until a missing-file spike ships. Readers who already keep that contract can run the same fixture on a throwaway free server with free model access, then discard the machine with the evidence folder still attached.

Persist AI Agent State on Free Servers with JSON Checkpoints

Harper Zhu — Fri, 04 Sep 2026 11:10:07 +0000

I stop losing AI agent insights on free ephemeral servers by writing a JSON checkpoint after every meaningful step and loading that file before the next session does any real work. Token grants fund compute, but only an external persistence layer preserves the context the agent already paid to earn.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Why Free Ephemeral Servers Wipe Agent Context

I watched an AI coding agent spend an hour diagnosing a failing test, then lose every insight when the sandbox expired. The agent started the next session by re-running the same commands and re-reading the same logs, so the second hour was a replay of the first. That pattern repeated every time the environment reset, and the wasted effort became a predictable tax on my productivity.

The free server option in MonkeyCode is a powerful way to experiment, but its ephemeral nature creates a hidden cost. Every new sandbox is a blank slate for the agent. Token grants solve the compute budget, not the context budget. A 10-million-token grant loses value when the agent must re-learn the same codebase on every restart.

I treat that mismatch as a design constraint, not something to fight:

The sandbox gives me a filesystem, a shell, and a token grant for one session
The grant is consumed by exploration: reading logs, forming hypotheses, ruling out dead ends
When the sandbox dies, those conclusions die with it unless I copy them out
The next grant then pays again for the same exploration

Compute and context are different budgets. I keep them on different layers so a reset only kills the machine, not the work.

Record Decisions in a JSON Checkpoint

A simple JSON file can turn a disposable server into a continuous work session, because the agent writes a checkpoint after each meaningful step. The next session loads that checkpoint before doing any real work, which preserves the hard-won context from previous attempts.

I keep the schema intentionally small so the agent stays selective:

steps: the actions or observations that actually changed my understanding
hypotheses: the current best explanation, not every discarded idea

Minimal Python checkpoint

The script below reads a memory file if one exists, simulates a short agent session, and saves the updated state. It is intentionally small so the mechanics are obvious. For the file format I follow the Python json module.

#!/usr/bin/env python3
"""demo_memory.py - simulate an AI agent that persists its memory across sessions."""
import json
import os

MEMORY_FILE = "demo_memory.json"

def load_memory():
    if os.path.exists(MEMORY_FILE):
        with open(MEMORY_FILE) as f:
            return json.load(f)
    return {"steps": [], "hypotheses": []}

def save_memory(memory):
    with open(MEMORY_FILE, "w") as f:
        json.dump(memory, f, indent=2)

def main():
    memory = load_memory()
    if memory["steps"]:
        print("Resuming from previous session:")
        for i, (step, hypothesis) in enumerate(zip(memory["steps"], memory["hypotheses"]), 1):
            print(f"  {i}. {step} -> {hypothesis}")
    else:
        print("No memory found. Starting fresh.")

    step = input("What did you just learn? ")
    if step.strip().lower() == "exit":
        return
    hypothesis = input("What is your current hypothesis? ")
    memory["steps"].append(step)
    memory["hypotheses"].append(hypothesis)
    save_memory(memory)
    print("Memory saved. Next session will resume here.")

if __name__ == "__main__":
    main()

I verify the loop with two runs:

The first run starts with empty memory, records a step and a hypothesis, and writes demo_memory.json.
The second run loads that file, prints the previous context, then appends a new step.
Killing the process—or the sandbox—does not erase the file if I have already copied it out.

That is the core of a persistence layer: the agent's working state survives the death of the sandbox. I refuse to persist raw command output or entire log files. Those bloat the JSON and recreate the original context-tax problem inside the memory file itself.

Store Memory Outside the Ephemeral Server

In a real deployment, the memory file must live outside the ephemeral server. I copy the file to a persistent location after each session and fetch it before the next one. A Git repository works well as a memory store. The agent can commit the memory file at the end of a session, and the next session can pull the latest version before starting. This adds a few seconds of overhead but saves many minutes of redundant exploration.

Git pull and push helpers

# save_memory.sh - push memory to a remote Git repo
#!/usr/bin/env bash
set -euo pipefail
git add memory.json
git commit -m "Update agent memory"
git push origin main

# load_memory.sh - pull the latest memory before starting
#!/usr/bin/env bash
set -euo pipefail
git pull origin main

The session wrapper I actually run looks like this:

Run load_memory.sh so memory.json matches the remote tip
Start the agent with load_memory() as its first call
After each high-signal finding, append to steps and hypotheses, then call save_memory()
On exit, run save_memory.sh so the next sandbox inherits the work

The same pattern works with object storage or a simple HTTP endpoint. The key insight is that the persistence layer is separate from the compute layer. The free server provides the compute, while an external store provides the memory. This separation is what makes ephemeral environments practical for long-running agent tasks. For the save path I follow the official git commit reference.

A quick comparison of stores that fit this pattern:

Git: conflicts are visible, history is cheap, and the setup is already familiar
Object storage: no merge step, which is better when only one session writes
HTTP endpoint: useful when I already have a small internal API, at the cost of extra moving parts

I pick Git when I want reviewable checkpoints and object storage when I want a single blob with no branch noise.

Know When This Pattern Helps—and When It Does Not

The limitations of this approach are worth naming. A JSON file grows quickly if the agent records too much, so the agent must be selective about what it persists. Conflicts can arise when two sessions write to the same file, though a Git-based store makes conflicts visible and resolvable. Some state, like loaded libraries or running processes, cannot be serialized into a file and must be rebuilt anyway. The persistence layer is a complement to, not a replacement for, a well-designed agent loop.

I skip the pattern in three cases:

Teams with strict data-residency requirements, because copying memory files to an external store may violate those rules
Teams that already run agents in a single long-lived environment, because the environment already provides continuity
Teams still evaluating different agents against a fixed benchmark, since a persistence layer adds complexity without improving the measurement

When I do use it, I add two guardrails: a maximum list length so old steps roll off, and a rule that the latest hypothesis overwrites the previous one instead of growing forever. Those two choices keep the file small enough to read at the start of every session.

Make the Next Free Session Continue the Last One

The ephemeral memory problem is real, and it is solvable with a few lines of code. A free server with a token grant is only useful if the agent can carry its work forward. By adding a small persistence layer, I turn a disposable sandbox into a productive workspace that learns across sessions.

If you are experimenting with MonkeyCode's free server, add a memory file to your next agent session and measure how much faster the second session completes. Do this in order:

Copy the Python script into your repo and treat memory.json as part of the project, not as sandbox leftover.
Wrap each session with the two Git helpers so you pull before work and push on exit.
Run two sessions on the same failing test and compare how much of the first hour you do not have to repeat.
Comment with your before-and-after times, or with the one field you wish the schema had, so the next write-up can tighten the pattern.

Start this week with one failing test. Ship the checkpoint. Then tell me whether the second session actually skipped the replay.

MonkeyCode provides free models that can run this workflow.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Scoring Review Surface in a Ninety-Minute Spike

Harper Zhu — Thu, 03 Sep 2026 15:06:01 +0000

A mid-size payments team merged an AI-written retry helper into a brownfield service on a Thursday afternoon. The demo looked complete, the unit tests were green, and the author had already moved toward the next ticket. By Monday the review thread held fourteen comments about timeout budgets, log redaction, and a swallowed cancel. Generation had been cheap that afternoon, yet the review surface had quietly become the actual delivery cost.

Cheap generation often resembles buying fabric by the bolt while still paying a tailor by the hour. The cloth arrives quickly, folded, and apparently finished, but every unmatched seam still belongs to a human reviewer. Teams that count merged lines as progress then discover that review comments, not tokens, set the calendar. A time-boxed spike can make that trade visible before the next helper lands on the default branch.

The protocol below is a proposed ninety-minute spike, not a completed trial with published scores. It holds one hypothesis, one frozen task, and one kill rule that a team can execute without debating taste. The hypothesis is simple and falsifiable within the window: an assistant-authored patch for a bounded change will exit with a review-surface score no worse than a human patch started from the same card. If the score misses the threshold, the team kills the workflow for that class of change rather than bargaining with the demo.

A free model session on a throwaway server keeps billing setup from leaking into the evidence. Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode is relevant here only as an evaluation host that offers free model access and a free server option, so the clock can start on the task instead of on account provisioning. The product is not the subject of the spike, and no model names, quotas, or hardware claims are required for the method to stand.

The engineer begins by writing the hypothesis into a log that will outlive the server. The card must name the change, the files that may move, and the kill number before any assistant prompt is typed. A concrete starting log looks like the following block, which should be committed to a throwaway branch so later arguments cannot rewrite the bet.

# spike-log-2026-09-03.txt
hypothesis: assistant patch for bounded retry helper has review_surface <= 8
task: add retry with jitter around existing HTTP client in pkg/upstream
allowed_paths: pkg/upstream/*.go, pkg/upstream/*_test.go
forbidden: new dependencies, config format changes, public API rename
kill_if: score > 8 OR tests missing OR diff touches files outside allowed_paths
clock: 90 minutes, no extension after first failing check

A disposable workspace then isolates the experiment from the laptop that still holds production credentials. The commands below assume a fresh Unix shell on a free server and a shallow clone of an internal service. They are labeled as a proposed sequence, and a team should substitute its own repository URL and module path before running anything against real code.

# proposed evaluation host, not a production jump box
mkdir -p /tmp/review-spike && cd /tmp/review-spike
git clone --depth 1 git@internal.example/payments-upstream.git repo
cd repo
git switch -c spike/review-surface-retry

# freeze the pre-spike snapshot for later scoring
git rev-parse HEAD > /tmp/spike-base.sha
find pkg/upstream -type f | sort > /tmp/allowed-paths.txt

The frozen task should be ugly enough to generate review comments if the assistant cuts corners, yet small enough to finish inside the window. One workable card asks for retry with jitter around an existing HTTP client, a test that fails on exhausted attempts, and logs that never print secrets. The engineer pastes that card once, refuses follow-up prompt shopping, and spends the remaining minutes on tests and the scorer rather than on conversational cleanup.

After the assistant returns a patch, the engineer applies it only on the spike branch and refuses extra files that wander outside the allow list. The next commands capture the diff in a form the scorer can read without a browser. Keeping the artifact on disk matters, because memory of a green demo is a poor substitute for the patch that reviewers will actually see.

git add pkg/upstream
git diff --cached -- pkg/upstream > /tmp/spike.patch
git diff --cached --stat > /tmp/spike.stat
git diff --cached --name-only > /tmp/spike.files

# proposed guard: kill early if the patch escaped the card
if grep -vxF -f /tmp/allowed-paths.txt /tmp/spike.files; then
  echo "KILL: path escape" >> /tmp/spike-log-2026-09-03.txt
fi

Review surface is treated as a countable object rather than a feeling about code quality. The proposed Python scorer below walks a unified diff and adds points for missing tests, bare exception handlers, TODO markers, secret-like literals, and files that never appear in the allow list. Teams may change the weights, but they should change them before the clock starts so the spike cannot negotiate with its own rubric.

#!/usr/bin/env python3
"""Proposed review-surface scorer for a time-boxed AI patch spike."""
from pathlib import Path
import re
import sys

WEIGHTS = {
    "file": 1,
    "hunk": 1,
    "todo": 2,
    "bare_except": 3,
    "no_test_file": 4,
    "secret_literal": 5,
    "path_escape": 8,
}

SECRET = re.compile(r"(api[_-]?key|password|secret)\s*[:=]\s*['\"][^'\"]+", re.I)
BARE = re.compile(r"except\s*:\s*($|#)|catch\s*\(\s*Exception", re.I)
TODO = re.compile(r"\b(TODO|FIXME|HACK)\b")

def score(patch: str, allowed: set[str]) -> dict:
    files, hunks, findings = set(), 0, []
    current = None
    for line in patch.splitlines():
        if line.startswith("+++ b/"):
            current = line[6:]
            files.add(current)
            if current not in allowed:
                findings.append(("path_escape", current))
        elif line.startswith("@@"):
            hunks += 1
        elif current and line.startswith("+"):
            body = line[1:]
            if TODO.search(body):
                findings.append(("todo", current))
            if BARE.search(body):
                findings.append(("bare_except", current))
            if SECRET.search(body):
                findings.append(("secret_literal", current))
    test_hit = any("test" in Path(f).name.lower() for f in files)
    if files and not test_hit:
        findings.append(("no_test_file", ",".join(sorted(files))))
    total = WEIGHTS["file"] * len(files) + WEIGHTS["hunk"] * hunks
    for kind, _ in findings:
        total += WEIGHTS[kind]
    return {"total": total, "files": sorted(files), "hunks": hunks, "findings": findings}

if __name__ == "__main__":
    patch = Path(sys.argv[1]).read_text()
    allowed = set(Path(sys.argv[2]).read_text().splitlines())
    result = score(patch, allowed)
    print(result)
    Path("/tmp/spike-score.txt").write_text(str(result["total"]))

Running the scorer should take seconds, which preserves the remainder of the ninety minutes for a human skim that the script cannot replace. The engineer records the numeric total beside the hypothesis and then walks the findings list once, adding any issue the weights missed, such as a retry loop that ignores cancellation. That extra pass is still part of the spike, because a scorer that never sees control-flow debt will bless a patch that reviewers will reject.

python3 /tmp/score_review_surface.py /tmp/spike.patch /tmp/allowed-paths.txt
SCORE=$(cat /tmp/spike-score.txt)
echo "score=${SCORE}" >> /tmp/spike-log-2026-09-03.txt

# proposed kill rule, written before the assistant ran
if [ "$SCORE" -gt 8 ]; then
  echo "KILL: review surface exceeded threshold" >> /tmp/spike-log-2026-09-03.txt
  git merge --abort 2>/dev/null || true
  git switch -
  git branch -D spike/review-surface-retry
else
  echo "SHIP-CANDIDATE: keep branch for ordinary review" >> /tmp/spike-log-2026-09-03.txt
fi

Ship in this protocol never means skip review. It means the workflow earned another ordinary pull request, not that the assistant became an owner of the module. Kill means the team stops using that assistant path for similar cards until a later spike with a tighter task or a different host beats the same number. The log is the only souvenir worth copying off the free server, because the branch itself was always disposable evidence.

The method has sharp limits that marketing copy would rather omit. The weights are local policy, not a universal quality function, and they will miss design errors that never appear as literals in a diff. A ninety-minute window cannot speak to load, failure injection, or multi-service contracts, and a free server may vanish before anyone reruns the patch. Teams that treat the score as a security audit, a licensing review, or a substitute for an on-call owner are using the wrong instrument.

Engineers in the middle of an incident should not start this spike, and neither should groups that lack authority to kill a workflow after a bad number. The protocol also fails for open-ended architecture work, where the interesting debt lives in diagrams rather than in hunk counts. In those cases the cheaper honesty is to refuse the assistant until the card is small enough that a kill rule can be written in one paragraph.

Readers who already isolate evaluation work on a throwaway host can run the same scoring loop with MonkeyCode's free model access and free server option, then keep or kill the workflow from the log rather than from a demo.