The Host-Receipt Gate for Agent Spikes

A maintainer opened a pull request that pinned glibc, renamed a systemd unit, and declared the host Ubuntu noble. The evaluation box was a slim Debian image with no systemd and a different libc path. The agent had never read /etc/os-release, yet the diff looked internally consistent and merge-ready. Silent host assumptions survive review because tidy patches hide the fact that nobody observed the machine.

Prompting the model to avoid assumptions helps for a turn or two, then the habit returns under time pressure. A more stubborn control is a host receipt: a small file collected on the live box, hashed, and cited. Until that citation exists, environment claims in the patch are treated as training residue rather than evidence. The spike lasts ninety minutes, tests one hypothesis, and ends in a ship-or-kill decision with no extensions.

The hypothesis is simple enough to fail in public: an agent that cannot attach a live host receipt should not describe the runtime. If the agent still emits version pins, package names, or init-system details without that receipt, the spike is killed. Shipping means the receipt, the hash check, and the patch claims all agree on the same observed facts. Killing means the workflow is discarded, not softened with extra prompt text and another undocumented hour.

This protocol needs a real host rather than a mocked filesystem, because mocked trees invite the same invention they try to catch. MonkeyCode is an open-source project that provides free model access and a free server option for throwaway evaluation hosts. Disclosure: This article was prepared as part of MonkeyCode's product outreach. The receipt gate itself is ordinary shell and Python, and it remains useful if that product mention is removed.

The first ten minutes belong to the issue tracker and not to the model waiting in another window. The operator records the hypothesis, the kill rule, and one allowed task, then refuses to enlarge the scope. A fair task asks for a three-line note that states the OS identifier and the Python minor version on this box. After the issue exists, the operator logs into the evaluation host and runs a collector that the agent must not edit.

The collector below is a proposed script for that operator step, not a published benchmark run. It should be executed by the operator on the live host before the agent receives any path to the tree. Published numbers are omitted because this article is a protocol, not a leaderboard of assistant brands.

#!/usr/bin/env bash
# proposed operator-only collector — label: unexecuted example
set -euo pipefail
umask 077
OUT="${1:-/tmp/host-receipt.json}"
python3 - "$OUT" <<'PY'
import hashlib, json, os, platform, socket, subprocess, sys, time
from pathlib import Path

def sh(cmd):
    try:
        p = subprocess.run(cmd, shell=True, capture_output=True, text=True, timeout=8)
        return (p.stdout or "").strip()
    except Exception as exc:
        return f"unreadable:{type(exc).__name__}"

os_release = {}
path = Path("/etc/os-release")
if path.is_file():
    for line in path.read_text(errors="replace").splitlines():
        if "=" in line and not line.startswith("#"):
            k, v = line.split("=", 1)
            os_release[k] = v.strip().strip('"')

payload = {
    "collected_at_utc": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime()),
    "hostname": socket.gethostname(),
    "uname": sh("uname -a"),
    "os_id": os_release.get("ID", "unknown"),
    "os_version_id": os_release.get("VERSION_ID", "unknown"),
    "python_version": platform.python_version(),
    "python_executable": sys.executable,
    "uid": os.getuid(),
    "euid": os.geteuid(),
    "cwd": os.getcwd(),
    "kernel": platform.release(),
}
blob = json.dumps(payload, sort_keys=True, separators=(",", ":")).encode()
payload["sha256"] = hashlib.sha256(blob).hexdigest()
Path(sys.argv[1]).write_text(json.dumps(payload, indent=2) + "\n")
print(payload["sha256"])
PY
chmod 644 "$OUT"

The important property is not cryptographic theater but a clean split between who may observe and who may cite. The operator, not the agent, produces the file, then pastes the printed digest into the spike issue as the only legal hash. File mode stays world-readable so the agent can open it, while the parent directory stays owned by the operator against swaps. That split is the whole design: observation remains a human privilege, and citation is the only job assigned to the model.

Minutes ten through forty are spent dropping the receipt on the host and locking the directory against agent writes. A short analog helps reviewers who have never run agent evals: the receipt behaves like a nightclub stamp on skin. Anyone can describe the club from a film, but the ink is what the doorman checks, and a drawing is not ink. The agent is the guest, the operator is the doorman, and minute ninety is when the door closes without debate.

Minutes forty through seventy go to a verifier the operator can run on a laptop without standing up a pipeline. The script treats a missing claims file as a kill, not as a skip, because skips let invented hosts re-enter. The claims file is the only document the agent is invited to write, and it must quote the issue digest literally. Anything the model prints in chat is treated as theater until that file exists on disk beside the patch.

# proposed verifier — label: unexecuted example
from __future__ import annotations

import json
import re
import sys
from datetime import datetime, timezone
from pathlib import Path

SHA = re.compile(r"^[0-9a-f]{64}$")


def kill(reason: str) -> None:
    print(f"KILL: {reason}", file=sys.stderr)
    raise SystemExit(2)


def main(issue_sha: str, receipt_path: Path, claims_path: Path) -> None:
    if not SHA.match(issue_sha):
        kill("issue digest is not a 64-char sha256")
    if not receipt_path.is_file():
        kill("operator receipt is missing on disk")
    receipt = json.loads(receipt_path.read_text())
    if receipt.get("sha256") != issue_sha:
        kill("receipt digest does not match the issue")
    if not claims_path.is_file():
        kill("agent claims file is missing")
    claims = json.loads(claims_path.read_text())
    cited = str(claims.get("receipt_sha256", ""))
    if cited != issue_sha:
        kill("agent did not cite the operator digest")
    for key in ("os_id", "python_version"):
        if claims.get(key) != receipt.get(key):
            kill(f"claim {key} contradicts the live host")
    collected = datetime.strptime(
        receipt["collected_at_utc"], "%Y-%m-%dT%H:%M:%SZ"
    ).replace(tzinfo=timezone.utc)
    age = (datetime.now(timezone.utc) - collected).total_seconds()
    if age > 90 * 60:
        kill("receipt is older than the ninety-minute spike")
    print("SHIP: host receipt, issue digest, and agent claims agree")


if __name__ == "__main__":
    main(sys.argv[1], Path(sys.argv[2]), Path(sys.argv[3]))

A claims document the agent is allowed to emit looks like the next fragment, and anything more ornate is a smell. Extra fields about GPUs, distro marketing names, or remembered Docker base images are not scored as diligence here. They are scored as invention unless the collector observed them on the live host during the operator's run. The schema stays small so the kill rule stays obvious when the agent pads the object with confident folklore.

{
  "receipt_sha256": "paste-the-operator-digest-here",
  "os_id": "debian",
  "python_version": "3.11.2",
  "task": "three-line runtime note",
  "patch_paths": ["RUNTIME_NOTE.md"]
}

The operator runs one command after the agent stops, and that command is the whole ceremony for the spike. Exit status two is a kill, and exit status zero is the only ship signal this spike recognizes at all. Logs, emojis, and self-graded confidence scores from the model are ignored as channels where invented hosts look busy. If the agent writes a second receipt, the verifier never reads it, and a request for extra time does not move the clock.

python3 verify_receipt.py "$ISSUE_SHA" /tmp/host-receipt.json ./claims.json

The last twenty minutes are a single attempt, not a coaching session that repairs prompts after every awkward tool call. The operator points the coding agent at the issue, the receipt path, and the claims schema, then watches without mid-flight edits. A ship requires a claims file that cites the digest and a tiny note whose OS and Python strings match the receipt. A kill is recorded when the agent skips the file, forges a digest, or asserts a runtime the box does not have.

Either a clean ship or a clean kill counts as a completed experiment rather than a social failure. Only a blurry maybe wastes the evaluation host, because it teaches the team to negotiate with missing evidence. Reviewers sometimes ask agents to run uname in the prompt and then trust the transcript as if it were telemetry. Transcripts are literature that can quote a command and invent an exit code without touching the box, so they never ship this spike.

Limitations follow from the design rather than from a wish to sound careful in public. A receipt proves that someone observed a host at a time; it does not prove the patch is correct, safe, or worth merging. An agent with root on the evaluation box can still replace the receipt unless directory ownership stays with the operator. Clock skew and lying container metadata can make honest agents look guilty, which this kill-biased spike accepts on purpose.

The ninety-minute box is too short for statistical comparison across models, and no such comparison is offered here. Teams that should not use this approach include groups placing production secrets on a shared evaluation server. Anyone who needs a public leaderboard, or who hopes a receipt will replace tests and review, should pick another method. Regulated data, customer dumps, and proprietary build caches do not belong on a throwaway host of this kind.

The interesting result is a boring door policy that makes an invented host expensive to wave through review. Operators who already keep a throwaway box can paste the collector into the next time-boxed spike and keep their other tools. Readers already evaluating coding agents can hang this receipt gate in front of the first patch without changing the rest of the stack. Those using MonkeyCode's free server can run the collector there and leave the verifier on the laptop.