Name a Replay Owner: A One-Page Wiki SOP When Chat History Is Not a Build

You walk into Monday standup and someone claims the overnight helper finally unstuck the flaky checkout test. The pull request looks green, the diff is short, and the description simply says the model figured it out. You ask them to replay the same change on a clean branch, and they paste a screenshot of a vanished chat. That gap is not a tooling failure so much as a missing human role on the team wiki.

Community threads this week keep arguing whether generated code already outruns most developers, or whether calling that work engineering is the real mistake. You do not need to settle that debate before your next merge window. You need a named person who can re-run the work from files in git, without hunting a private transcript. This article gives you that role, a paste-ready wiki page, and a small receipt script you can keep beside the patch.

Chat history is not a build artifact

When a change only lives in a vendor transcript, you cannot hand the work to the next on-call engineer. You also cannot prove the command, the prompt file, and the repository SHA belonged together. Teams then argue about whether the model was brilliant or whether someone quietly edited the output. Engineering still needs a replay path that starts from ordinary files you already store in git.

You have seen the failure mode even when tests are green. A teammate copies a suggested patch, tweaks one import, and later cannot remember which instruction produced which hunk. Another teammate reruns “the same prompt” on a newer checkout and gets a different helper function with the same name. A third person is out sick, and the chat product has already scrolled the session out of the shared browser profile.

Name a Replay Owner for the current week

You do not need another committee, and you do not need a new job title on LinkedIn. You need one named person who refuses to merge AI-assisted work that cannot be re-run from the repo. Call that person the Replay Owner for the current week, and write the name on a one-page wiki SOP. Rotate the role like you rotate on-call, because stale ownership is how chat screenshots become the record.

The Replay Owner is not the author of every patch, and is not a substitute for code review. The owner is the person who checks that a later engineer can reconstruct the run from a receipt file. If that reconstruction fails, the owner blocks the merge and files a short note instead of arguing about model quality in Slack.

What a replayable change actually contains

A replayable change has four pieces sitting next to the patch, not in someone's private browser. First, a frozen command line that does not depend on interactive chat or a hidden desktop session. Second, a prompt or instruction file with a content hash, so silent edits cannot hide after review. Third, the git commit that the run believed it was editing, recorded before anyone restated the problem. Fourth, hashes of stdout, stderr, and any generated files, so a later run can fail closed.

If any of those four pieces is missing, you do not have a replay. You have a story about a replay. Treat the story as a draft until the receipt file exists on the branch.

One-page wiki SOP you can paste

Copy the block below into your team wiki. Replace the names, paths, and runner notes with whatever your group already uses. Keep the page in the same repository as the code whenever you can, so the next owner does not need a second hunt through docs sites.

# Replay Owner — one-page run

**Week of:** YYYY-MM-DD
**Replay Owner (human):** @name
**Backup:** @name
**Handoff window:** weekdays 16:00 local; owner posts in #eng-ai

## Goal
No AI-assisted change merges unless a later engineer can re-run it from git
without the original chat transcript.

## In scope
- Prompt or instruction files committed with the patch
- Exact command recorded in `replay/receipt.json`
- Input git SHA, output hashes, and a pass/fail verify step

## Out of scope
- Model beauty contests and unofficial benchmark screenshots
- Secret material, production credentials, or customer data
- Replacing code review, security review, or release management

## Numbered run (author, then Replay Owner)
1. Author freezes work on a dedicated branch; no more prompt edits in the chat UI.
2. Author saves the instruction text to `replay/instruction.md` and stops editing it.
3. Author records the exact command in `replay/command.sh` (no interactive flags).
4. Author runs `python3 replay/write_receipt.py` on a clean checkout of that SHA.
5. Author opens a PR that includes the patch, instruction, command, and receipt.
6. Replay Owner checks out the PR, runs `python3 replay/verify_receipt.py`, and pastes the result.
7. If verify fails, Replay Owner blocks merge and writes a three-line incident note.
8. If verify passes, Replay Owner comments `REPLAY-OK @name <utc timestamp>` and reviews as usual.
9. Friday 15:00: owner names next week's Replay Owner in this page and in the channel.

## Handoff rule
If the owner is offline, the backup may verify but may not skip the receipt.
If both are offline, merge waits. Chat screenshots are not a substitute.

## Incident note template
- SHA attempted:
- Command:
- Expected hash vs actual hash:
- Decision: reject / new receipt required / escalate to reviewer

That page is the entire operating procedure. You can add links, but you should not add a second philosophy section that nobody will read during a merge freeze.

Artifact: write and verify a receipt from a checkout

The following Python is a template you can adapt; it is not a hosted service and it does not call a model. Put both files under replay/ and keep them boring. Boring receipts are easier to read at 5 p.m. than clever ones.

#!/usr/bin/env python3
"""replay/write_receipt.py — template for a local or shared runner."""
from __future__ import annotations

import hashlib
import json
import os
import subprocess
import sys
from datetime import datetime, timezone
from pathlib import Path

ROOT = Path(__file__).resolve().parents[1]
REPLAY = ROOT / "replay"
INSTRUCTION = REPLAY / "instruction.md"
COMMAND = REPLAY / "command.sh"
RECEIPT = REPLAY / "receipt.json"
LOG_DIR = REPLAY / "logs"


def sha256_file(path: Path) -> str:
    digest = hashlib.sha256()
    with path.open("rb") as handle:
        for chunk in iter(lambda: handle.read(65536), b""):
            digest.update(chunk)
    return digest.hexdigest()


def git_sha() -> str:
    result = subprocess.run(
        ["git", "rev-parse", "HEAD"],
        cwd=ROOT,
        check=True,
        capture_output=True,
        text=True,
    )
    return result.stdout.strip()


def run_command() -> dict:
    LOG_DIR.mkdir(parents=True, exist_ok=True)
    stdout_path = LOG_DIR / "stdout.txt"
    stderr_path = LOG_DIR / "stderr.txt"
    completed = subprocess.run(
        ["bash", str(COMMAND)],
        cwd=ROOT,
        capture_output=True,
        text=True,
    )
    stdout_path.write_text(completed.stdout)
    stderr_path.write_text(completed.stderr)
    return {
        "exit_code": completed.returncode,
        "stdout_sha256": sha256_file(stdout_path),
        "stderr_sha256": sha256_file(stderr_path),
        "stdout_path": str(stdout_path.relative_to(ROOT)),
        "stderr_path": str(stderr_path.relative_to(ROOT)),
    }


def main() -> int:
    for required in (INSTRUCTION, COMMAND):
        if not required.is_file():
            print(f"missing {required}", file=sys.stderr)
            return 2
    if os.getenv("REPLAY_ALLOW_DIRTY") != "1":
        dirty = subprocess.run(
            ["git", "status", "--porcelain"],
            cwd=ROOT,
            check=True,
            capture_output=True,
            text=True,
        )
        if dirty.stdout.strip():
            print("working tree is dirty; refuse to write a receipt", file=sys.stderr)
            return 3
    run_info = run_command()
    receipt = {
        "recorded_at_utc": datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ"),
        "git_sha": git_sha(),
        "instruction_sha256": sha256_file(INSTRUCTION),
        "command_sha256": sha256_file(COMMAND),
        "command_path": "replay/command.sh",
        "instruction_path": "replay/instruction.md",
        **run_info,
    }
    RECEIPT.write_text(json.dumps(receipt, indent=2) + "\n")
    print(RECEIPT)
    return 0 if run_info["exit_code"] == 0 else 4


if __name__ == "__main__":
    raise SystemExit(main())

#!/usr/bin/env python3
"""replay/verify_receipt.py — fail closed if the frozen run cannot be repeated."""
from __future__ import annotations

import json
import subprocess
import sys
from pathlib import Path

ROOT = Path(__file__).resolve().parents[1]
REPLAY = ROOT / "replay"
RECEIPT = REPLAY / "receipt.json"
INSTRUCTION = REPLAY / "instruction.md"
COMMAND = REPLAY / "command.sh"


def sha256_file(path: Path) -> str:
    import hashlib

    digest = hashlib.sha256()
    with path.open("rb") as handle:
        for chunk in iter(lambda: handle.read(65536), b""):
            digest.update(chunk)
    return digest.hexdigest()


def main() -> int:
    if not RECEIPT.is_file():
        print("missing replay/receipt.json", file=sys.stderr)
        return 2
    data = json.loads(RECEIPT.read_text())
    head = subprocess.run(
        ["git", "rev-parse", "HEAD"],
        cwd=ROOT,
        check=True,
        capture_output=True,
        text=True,
    ).stdout.strip()
    checks = [
        ("git_sha", data.get("git_sha"), head),
        ("instruction", data.get("instruction_sha256"), sha256_file(INSTRUCTION)),
        ("command", data.get("command_sha256"), sha256_file(COMMAND)),
    ]
    failed = False
    for label, expected, actual in checks:
        ok = expected == actual
        print(f"{label}: {'OK' if ok else 'MISMATCH'}")
        if not ok:
            print(f"  expected {expected}")
            print(f"  actual   {actual}")
            failed = True
    if failed:
        return 1
    print("receipt files match this checkout; re-run write_receipt.py to compare logs")
    return 0


if __name__ == "__main__":
    raise SystemExit(main())

A minimal replay/command.sh can be as small as a unit test target. Keep it non-interactive, and keep it free of secrets. The Replay Owner should reject a command that curls hidden environment variables or that only works inside one person's laptop session.

#!/usr/bin/env bash
set -euo pipefail
cd "$(dirname "$0")/.."
python3 -m pytest tests/test_checkout_retry.py -q

Numbered handoff you run on Friday

1. Open the wiki page and confirm the current Replay Owner is still the person on the calendar.
2. List open AI-assisted pull requests that lack replay/receipt.json and move them back to draft.
3. Re-run python3 replay/verify_receipt.py on any PR that claims REPLAY-OK from an earlier SHA.
4. Post a three-line summary: owner name, PRs blocked, PRs verified, and the next owner.
5. If the next owner declines, the current owner keeps the role; an empty name is not a handoff.

That Friday list is the entire ceremony. You should not add a slide deck, and you should not require a meeting unless two receipts disagree about the same SHA.

Where a shared free runner fits, and where it does not

Some teams already park draft agent runs on a shared box so laptops can close. Disclosure: This article was prepared as part of MonkeyCode's product outreach. If you need a shared place to execute write_receipt.py beside those drafts, MonkeyCode's free model access and free server option is one runner you can point the wiki page at. The SOP still works if you use a spare workstation or a self-hosted job instead; the receipt files are the contract, not the brand of the machine.

Do not put production secrets on a shared free runner. Do not treat a free shared host as an evidence locker for regulated data. The Replay Owner records commands and hashes, then leaves credential handling to whatever boundary process your team already maintains.

A small decision table for the owner

What you see	What you do	What you do not do
No replay/instruction.md	Block merge; ask for a frozen file	Accept a chat screenshot
Instruction hash drifted after review	Treat it as new work; demand a new receipt	Rubber-stamp the old REPLAY-OK
Git SHA in the receipt is not HEAD	Ask for a rebase and a fresh run	Verify against memory of an older branch
Tests pass but stdout hash changed	File the incident note; inspect the log diff	Argue about whether the model improved
Command requires a local GUI session	Reject the command file	SSH into the author's laptop
Owner and backup are both offline	Leave the PR open	Merge because standup sounded confident

Limitations you should write on the wiki page

This receipt format does not prove the generated idea was correct, only that the same files and command were used. Hashing stdout will fail on timestamps, random ports, and unordered logs, so you must keep the recorded command deterministic. The scripts above do not sandbox the command, do not measure model quality, and do not replace a reviewer who understands the domain. A shared runner can disappear, so you still commit the receipt into git instead of trusting disk on that host.

You should also expect social failure modes. Authors will ask you to skip the receipt because the diff is “obvious.” Reviewers will treat REPLAY-OK as a quality score. Neither request is in scope. The Replay Owner only answers whether a later human can re-run the work without the original chat.

Who should not use this SOP

Skip this process if you are a solo hobbyist with no handoff and no shared branch. Skip it if your company already requires experiment tracking with sealed artifacts and a compliance officer. Skip it for emergency production hotfixes that already have a named incident commander and a recorded rollback. Do not use it as cover for merging unreviewed patches, and do not use it to shame teammates who still think in a scratch buffer.

If you cannot name a human owner for the coming week, you do not have a replay practice. You have a Markdown file. Put a real handle on the wiki page first, then run the scripts on one small flaky test before you roll the SOP across every AI-assisted pull request.