DEV Community: Alex Zhu

When Agent Cleanups Erase the Why: A Comment Handoff You Can Paste Into the Wiki

Alex Zhu — Thu, 24 Sep 2026 19:26:35 +0000

You open a morning pull request and notice that the diff looks almost polite at first glance. An agent shortened three helpers, dropped unused blanks, and deleted the comment above the vendor retry guard. That comment had said the first timeout is a warm-up miss, and the retry budget must ignore it. Shared tests are green, so you need a handoff that names who restores that why before the cleanup is treated as done.

The review gap this card closes

A shorter function is not a clearer change when the reason for a guard disappears with the comment. Teams already argue about clean code and clear code, and agent cleanups make that argument land inside the diff. This playbook gives you roles, a handoff order, and a one-page wiki run you can paste before the next cleanup. It does not replace code review, and it does not pretend that every removed comment was either waste or wisdom.

Four roles on one handoff line

You assign four names before the cleanup branch opens, even when two names belong to the same small team. The diff author, whether human or agent-assisted, attaches an inventory of removed comments before requesting a decision. The rationale keeper decides whether each removed comment stays, moves, or dies, and they must not be the only author of the diff.

The scratch runner executes a narrow check on a disposable environment and records the exact command beside the result. The wiki clerk publishes the signed card after the keeper decides, and that clerk does not approve the code change. You write those names on the card even if the agent cannot hold a role, because accountability stays with people.

Handoff timing you do not skip

The diff author hands the completed inventory to the rationale keeper before anyone rewrites a surviving comment in the branch. The keeper hands a signed disposition back to the scratch runner only for rows marked as behavior risk. The scratch runner hands the command log to the wiki clerk, and the clerk publishes before the approver is pinged. You do not skip ahead to merge when one of those handoffs is still verbal, because verbal notes vanish after the pull request scrolls.

Paste this one-page run into the wiki

You copy the card into the team wiki under a stable title so reviewers are not hunting through chat for the latest rule. You keep one card per pull request, because a living mega-page hides which commit the disposition actually covered. You redact hostnames and customer identifiers before the card is saved, even when the comment was already public inside the repo. You link the card from the pull request body so the next reader can find the why without asking who remembers it.

# Comment handoff card
- Date:
- Repo:
- Base commit:
- Head commit:
- Diff author:
- Rationale keeper:
- Scratch runner:
- Wiki clerk:
- Inventory command:
- Scratch command and exit code:
- Secrets checked (yes/no):

## Hits
| File | Removed line (redacted) | Disposition (keep/move/delete/stop) | Evidence |
| --- | --- | --- | --- |
| | | | |

## Keeper note
- Unverified model guess used (yes/no):
- Disagreement with ticket (yes/no):
- Merge allowed (yes/no):

Steps you follow on the cleanup branch

1. Freeze the comparison range

You start from a named base branch so the inventory does not mix yesterday's cleanup with unrelated local edits. You record the base commit and the head commit on the card before anyone amends the branch again. You refuse a review that only shows a screenshot of the editor, because the wiki needs a command someone else can rerun. If the branch was rebased, you regenerate the inventory instead of trusting the previous list of deleted lines.

2. Build the comment inventory

You pipe the unified diff into a small scanner that prints removed lines which look like comments. You treat that output as a lead list, not as proof, because block comments and generated files confuse simple markers. You open each hit in the file and note whether the surrounding code still states the same constraint in names or tests. You drop license headers and obvious section banners from the decision list so the keeper is not buried in noise.

3. Classify before you rewrite

You use the decision table below so two reviewers do not invent private rules during the same pull request. You restore a comment when it names a vendor limit, a race, a migration, or a deliberate non-action. You move a long explanation to the wiki when the code stays, and you add a pointer only if the team already uses pointers. You allow a deletion when the comment only restates the function name and the tests already document the behavior.

4. Separate style cleanup from behavior risk

You do not rerun the whole suite for a comment-only deletion if the diff contains no code line changes. You do ask the scratch runner for a narrow check when a comment and a condition were edited in the same hunk. You keep that check off the shared release lane so a rehearsal failure does not block unrelated ship work. You paste the command, the exit code, and the commit under test into the card before the keeper signs.

5. Sign, publish, and stop

You stop the merge if the inventory still has an unclassified removed comment, even when the tests are green. You stop again if the model draft and the ticket disagree, because a fluent summary is not a source of record. The wiki clerk publishes the card in the agreed folder and links it from the pull request before approval. You merge only after the keeper name, the date, and the disposition of each hit are visible to the next reader.

Commands you keep beside the card

# Unexecuted examples. Replace branch names and test paths before you trust them.
git fetch origin
git merge-base origin/main HEAD
git rev-parse HEAD
git log --oneline origin/main..HEAD

git diff --unified=3 origin/main...HEAD -- '*.py' '*.ts' '*.go' ':!vendor' ':!dist' \
  | python3 scripts/comment_handoff.py > /tmp/comment-inventory.txt

# Stop before any model call if a common secret marker appears.
git diff origin/main...HEAD | grep -E 'AKIA|BEGIN PRIVATE KEY|api_key' \
  && echo 'stop: possible secret' || echo 'no common secret marker'

# Narrow scratch check. Do not point this at production.
python3 -m pytest tests/test_vendor_timeout.py -q --tb=line
echo "exit=$?"

You save the three-dot diff command on the card so a teammate can rebuild the inventory without guessing your local branch state. You run the secret-marker grep before any model call, and you stop the whole handoff if that grep prints a hit. You treat the pytest line as a template, and you replace the path with the narrow test that covers the restored comment. You label every command unexecuted until someone on your team has run it against a real branch and kept the output.

An unexecuted scanner you can adapt

#!/usr/bin/env python3
# Unexecuted example: list removed comment-like lines in a unified diff.
# This script does not decide safety. Adapt markers and exclude paths first.

import sys

MARKERS = ('#', '//', '/*', '*', '--')


def removed_comments(diff_text: str):
    hits = []
    current = 'unknown'
    for raw in diff_text.splitlines():
        if raw.startswith('diff --git '):
            parts = raw.split()
            current = parts[-1] if parts else 'unknown'
            continue
        if not raw.startswith('-') or raw.startswith('---'):
            continue
        body = raw[1:].strip()
        if body.startswith(MARKERS):
            hits.append((current, body[:160]))
    return hits


def main() -> int:
    rows = removed_comments(sys.stdin.read())
    if not rows:
        print('no removed comment-like lines detected')
        return 0
    for path, body in rows:
        print(f'{path}\t{body}')
    return 2


if __name__ == '__main__':
    raise SystemExit(main())

You run this only as a lead list, and you expect false hits from SQL dashes, markdown bullets, and license banners. You exclude vendored paths in the git command so the keeper reads your code rather than a dependency changelog. You treat exit code 2 as a human must classify, not as a product failure and not as an automatic revert. You change the marker tuple before you trust it on a repo whose comments do not look like these prefixes.

Decision table for the keeper

You read each row as a default, then you write an exception on the card when the code proves the default wrong. You do not let a model pick the disposition, because the keeper signature is the point of the handoff. You prefer stop over a guessed delete when the evidence column would otherwise be empty. You can add a row for your own domain, such as feature flags, without changing the four role names.

What you see in the removed line	Default disposition	Scratch check	Wiki evidence
Restates the function or variable name	Delete	None if no code changed	style-only
Names a timeout, retry, race, migration, or vendor quirk	Restore or move, with a pointer only if your team already uses pointers	Narrow test for that behavior	Ticket or incident id
Conflicts with the current code or test	Stop	Reproduce the disagreement	Quote the conflicting lines, redacted
Holds a secret, token, customer id, or private hostname	Remove from code, do not paste the secret, rotate if it was real	Do not send the line to a model	redacted only

Where a disposable model pass fits

You can rehearse the unverified guess and the narrow check when free model access and a free server option are available to the team.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

MonkeyCode is the offering this walkthrough uses for those two options, and you confirm current docs before you rely on either one. If you need a low-friction place to practice, those free options can host a rehearsal, without any claim about quotas, hardware, or duration. You keep secrets and production logs out of the prompt and off the disposable server, then you copy results back to the card.

Limitations and who should skip this

This scanner only sees removed lines that begin with common comment markers, so it misses many block-comment edits. A model can invent a confident reason for a comment that was actually stale, sarcastic, or copied from another file. You should not use this playbook as incident command, as a solo approval path, or as a regulated audit system. You skip it when comments currently store credentials, because the first job is removal and rotation, not a wiki essay.

You also skip it when nobody will sign as rationale keeper, since an unsigned card is only theater. Free access is not a capacity plan, and you should not promise teammates a quota or uptime from this article. Generated files, lockfiles, and vendored trees will flood the inventory unless you exclude those paths in the command. You revisit the card when review tools change, or the handoff ages into the stale note you meant to protect.

What you do before the next cleanup

You paste the card, fill the four names, and run the inventory on the next cleanup before style talk starts. You record an empty inventory too, so the next reviewer can see that someone actually looked. You link the card from the pull request, and you treat a missing link as a reason to pause. The useful outcome is a diff whose why survives the cleanup, not a shorter file that nobody can explain.

Assign a Graduation Owner: A Wiki Playbook Before Sandbox Agent Jobs Touch Production

Alex Zhu — Wed, 23 Sep 2026 16:14:39 +0000

You are paging through Slack at 09:14 when the billing channel lights up with duplicate invoice emails. A teammate had pointed a shared agent at a free sandbox server, then reused the same production .env file. Copying those secrets felt faster than asking anyone to mint a scoped sandbox token. The agent followed its tool schema, posted a live webhook, and treated production as another demo endpoint.

Shared agents make that failure cheap to repeat, because tool calling looks the same in a demo and in a live checkout path. You cannot treat a green sandbox run as a production pass when the tools, secrets, and network paths are not actually split. You also cannot wait for a postmortem to invent an owner after the webhook has already fired. You need a named graduation owner, a one-page wiki, and a gate that fails closed.

The failure is a missing handoff, not a missing comment

Free model access and a free server option are useful when you want an isolated lane for prompt drafts, tool-schema experiments, and noisy retries. They become dangerous when that lane silently inherits production credentials, customer payloads, or write-capable endpoints. The failure is rarely the model itself; the failure is an undocumented handoff between exploring and shipping. You have probably watched a function schema look correct, watch the JSON parse, and still never ask which environment the HTTP client will hit.

A realistic API test on the sandbox still lies if the base URL, identity, and side effects are production-shaped. Clear comments in the prompt file will not save you if the runtime environment is wrong. Tool-calling tutorials make the happy path look identical across hosts, which is exactly why teams copy .env files. You need a written promotion rule before the next shared agent job leaves isolation.

You can host that isolated lane on any machine you control. Some teams use MonkeyCode here because it currently offers free model access and a free server option for exploratory agent work. Disclosure: This article was prepared as part of MonkeyCode's product outreach. Treat those two availability claims as current product options, not as a promise about model names, quotas, hardware, or how long the offer lasts.

Name the role before you name the server

A graduation owner is the person who may say no when a sandbox agent job asks to leave isolation. They do not write every prompt, and they do not babysit every retry. They own the checklist, the secret scope, and the written rule for what promoted means on your team. If you skip the role, every engineer becomes an accidental owner at 09:14, which is how environment files travel.

If you over-scope the role, the owner becomes a bottleneck and people bypass the wiki. You want a narrow mandate: environment split, tool side-effect class, and a recorded promote-or-stop decision. Write the name on the wiki page, not in a chat thread that scrolls away during the next incident. Rotate the owner on a published cadence so the role stays a practice, not a personality.

Roles you should print on one wiki page

Keep the cast small enough that a new teammate can read it in a standup. Use job function, not heroics, and keep each handoff visible in the repository rather than in private messages.

Graduation owner — Records promote or stop, and is the only person who can flip PROMOTION_STATUS=allowed for a job. They reject any run that still carries production hostnames, write tools, or customer payloads.
Lane operator — Starts and stops sandbox agent jobs, and must run the environment lint before the first tool call. They never mint production tokens into the sandbox directory, even for a five-minute demo.
Tool steward — Classifies each tool as read, dry-run, or write, and keeps the allowlist in version control. They block promotion while any write tool still points at an unclassified base URL.
On-call reviewer — Confirms the rollback command works before a promoted job is scheduled. They own the paging path if a graduated job later touches a live side effect.

Handoff rule: the lane operator may draft, the tool steward may classify, and only the graduation owner may promote. If any of those three names is empty, the wiki page is not ready and the job stays in sandbox. Do not let a group chat vote replace the named owner when the webhook looks urgent.

Decision table you can paste beside the names

Use this table as a fail-closed filter. If a row is unknown, the answer is stop, not try it once.

Question	Stay in sandbox	May graduate
Where do secrets live?	Sandbox-only tokens in a separate dir	Separate production secret store, never copied
What can tools do?	`read` or `dry-run` only	`write` tools behind an allowlist and a human gate
Which host is called?	Documented sandbox hostname	Production hostname listed, reviewed, and rate-limited
What data is in context?	Synthetic or scrubbed fixtures	Production-like data with a retention and redaction note
Who can abort?	Lane operator can kill the process	On-call reviewer has a tested rollback command
Is the model lane labeled?	Free or exploratory lane, not ship lane	Ship lane named in the job manifest

This table is a proposal for teams that already mix exploratory agents with shipped services. Label it as unexecuted until your graduation owner fills the hostnames and secret paths for your stack. Do not treat a passing unit test as a substitute for the environment row.

A one-page wiki run you can paste today

Copy the block below into your team wiki and fill the bracketed fields before the next agent demo. Keep it to one page so people actually read it.

# Agent graduation SOP (sandbox -> production)
Owner this week: [name]
Backup: [name]
Sandbox root: [path]
Production is forbidden inside sandbox root.

## Before every sandbox job
1. Lane operator runs `make sandbox-lint`.
2. Tool steward stamps tool classes in `tools.lock.yml`.
3. Graduation owner confirms `PROMOTION_STATUS=blocked`.

## Promote request
- Job id:
- Tools requested:
- Side-effect class:
- Secret scope:
- Rollback command:
- Decision: promote / stop
- Signed by graduation owner:

## Stop conditions
- Any production hostname in sandbox configs
- Any customer payload in prompt context
- Any write tool without a dry-run twin
- Any missing rollback command

Print the stop conditions near the job launcher, not only in an architecture doc. You want the operator to see them while the process is starting, not during the incident review. If your wiki cannot hold a one-page run, you are not ready to share agents across environments.

Numbered run for the next shared agent job

Follow these steps in order. Skipping a step is a stop, not a shortcut.

Create a sandbox directory that cannot see production files. Put prompts, tool schemas, and fake fixtures under agent-sandbox/ and keep production deploy files outside that tree. Tell every operator that copying .env into this directory is an incident, not a convenience.
Lint the tree for production hostnames and secret shapes. Run the check below until it exits zero. If it fails, the lane operator fixes the files before anyone talks to a model.
Classify tools before the agent process starts. A search tool is not a refund tool, even when both are JSON functions with similar names. Stamp the class in tools.lock.yml so the steward and owner review the same list.
Force dry-run on anything with a side effect. A webhook post, a database write, or a ticket closer must have a no-network twin. The agent may call the twin freely; the live tool stays blocked until promotion.
Record the promote-or-stop decision in the repo. Chat approval is not an artifact. The graduation owner commits a small decision file, or the job remains blocked.
Rehearse rollback on the sandbox host first. Kill the process, revoke the sandbox token, and confirm the dry-run log is the only residue. If rollback is theater, promotion stays forbidden.

Commands and checks you can run locally

The snippets below are labeled examples. They are not production security controls, and they will not replace a real secret manager. Adjust paths before you run them, and never paste live credentials into the wiki or the prompt.

# example: fail if sandbox configs mention production hosts or copied env files
# proposal only — extend the hostname list for your company
set -euo pipefail
ROOT="${SANDBOX_ROOT:-./agent-sandbox}"
DENY_HOSTS='api\.prod\.|billing\.internal|hooks\.stripe\.com'

if [[ ! -d "$ROOT" ]]; then
  echo "sandbox root missing: $ROOT" >&2
  exit 1
fi

if find "$ROOT" -name '.env' -o -name '*.pem' | grep -q .
then
  echo "stop: secrets or env files found under sandbox root" >&2
  exit 1
fi

if grep -REn "$DENY_HOSTS" "$ROOT" --include='*.yml' --include='*.json' --include='*.md'
then
  echo "stop: production hostname in sandbox tree" >&2
  exit 1
fi

echo "sandbox-lint ok"

Pair the lint with a promotion gate so a green sandbox test cannot silently become a live caller. The Python example refuses to start when promotion is blocked or when a write tool has no dry-run twin.

# example promotion gate — proposal / unexecuted template
from pathlib import Path
import json
import os
import sys

BLOCKED = {"blocked", "", "no", "never"}

def load_json(path: Path) -> dict:
    if not path.exists():
        print(f"stop: missing {path}", file=sys.stderr)
        sys.exit(1)
    return json.loads(path.read_text())

def main() -> None:
    status = os.environ.get("PROMOTION_STATUS", "blocked").lower()
    tools = load_json(Path("tools.lock.yml").with_suffix(".json"))
    decision = load_json(Path("graduation-decision.json"))

    if status in BLOCKED or decision.get("decision") != "promote":
        print("stop: graduation owner has not promoted this job")
        sys.exit(1)

    for name, meta in tools.items():
        side = meta.get("class")
        if side == "write" and not meta.get("dry_run_twin"):
            print(f"stop: write tool {name} has no dry-run twin")
            sys.exit(1)
        if side == "write" and not decision.get("signed_by"):
            print("stop: write tools require a signed owner decision")
            sys.exit(1)

    print("promotion gate ok")

if __name__ == "__main__":
    main()

Keep tools.lock.yml boring and explicit. A short classified list beats a clever registry that nobody reads during a demo.

{
  "search_docs": {"class": "read", "base_url_env": "SANDBOX_SEARCH_URL"},
  "preview_invoice": {"class": "dry-run", "base_url_env": "SANDBOX_BILLING_URL"},
  "post_webhook": {"class": "write", "dry_run_twin": "preview_invoice", "base_url_env": "PROD_BILLING_URL"}
}

If you need a dry-run HTTP check, call a sandbox echo endpoint rather than the live webhook. Replace the hostname with your own non-production host, and keep the request body synthetic.

# example: dry-run against a sandbox echo host, not a live billing webhook
curl -sS -X POST "$SANDBOX_ECHO_URL/preview-invoice" \
  -H 'content-type: application/json' \
  -d '{"customer_id":"fixture-17","amount_cents":0}'

What this playbook will not do

This SOP does not make a free sandbox equivalent to a hardened production control plane. Hostname linting will miss obfuscated URLs, runtime-injected hosts, and tools that build addresses from model output. A signed JSON file is an accountability artifact, not a cryptographic grant, and a determined operator can still export a production token by hand.

Do not use this approach if you are a solo hobbyist with no production side effects to protect. Do not use it if your company already has a change-advisory path that must remain the only promotion mechanism. Do not use it as a substitute for secret scanning, network policy, or a real allowlist on vendor APIs. Regulated teams that need auditable releases should keep this wiki as a local hygiene layer, not as their compliance story.

The playbook also assumes you can name a human who will be available during the job. If your agents run unattended overnight with write tools attached, assign a stop-condition practice first and keep promotion blocked. Sandbox work is for learning tool shapes and failure modes, not for quietly absorbing production traffic.

Close the loop in the repo, not in the demo

After a stopped job, leave the decision file in place so the next operator can see why promotion failed. After a promoted job, schedule a short rollback drill while the on-call reviewer is still in the channel. Update the hostname deny list whenever a new vendor API appears, because last month's lint will not catch this week's webhook.

Name the graduation owner before the next shared agent demo, and paste the one-page wiki into the repository today. If the owner field is still blank, the sandbox stays the only legal place for the job.

Assign a Tool-Allowlist Owner: A Wiki Playbook Before Shared Agents Call Vendor APIs

Alex Zhu — Tue, 22 Sep 2026 14:31:39 +0000

You drop an intern's agent job onto a shared server so it can search internal docs. Two hours later a vendor dashboard lights up with ticket spam from that same process. The job inherited every CLI on PATH plus a production token sitting in the environment. Nobody owned the outbound tool list, so widening access felt like a harmless prompt tweak.

That outage is the reason you write this playbook before the next shared job starts. You still want cheap exploration on a shared box, but you refuse silent inheritance of production capabilities. You name one owner who can widen tools, and you make every job prove its fence. The rest of this article is a wiki page, a preflight script, and a short reject path.

The failure you are actually preventing

Shared agent runtimes mix two kinds of work that should never share credentials. Draft jobs want many tools, while ship work wants a short, boring, reviewed surface. When both land on one host, the default is to leave every SDK, every curl alias, and every leftover token reachable. Your unit tests stay green because they never asserted which tools the process could invoke.

You cannot treat a chat transcript as the contract for outbound calls. Tool calling looks harmless until the model receives a function named create_ticket with a live key. You need a named human, a written allowlist, and a command that fails closed before the agent loop starts. Without those three pieces, every new prompt becomes an unofficial IAM change.

This playbook is for teams that already let more than one person start agent jobs. It is not a network design, and it is not a secret-manager migration. It is the missing one-page run you paste into the wiki so the next grant is boring.

Roles and handoffs you write in public

Name four roles on a single wiki page, with deputies, not Slack nicknames that rot. If a role is vacant, you stop granting tools rather than improvising in a hallway.

Tool-Allowlist Owner — the only person who can add, widen, or restore an outbound tool.
Job Author — the person who files the exact command surface, arguments, and destination hosts.
Reviewer — a second person who confirms the job cannot reach production systems or billing APIs.
On-call — the person who revokes a grant, rotates a token, and files the incident note.

Handoffs stay mechanical so you do not debate ownership during an alert. The Job Author never merges their own grant. The Reviewer never approves a wildcard host. The Owner never accepts a tool that arrived only as a chat message. On-call never leaves a revoked grant sitting in an old shell profile.

When the page applies, and when you skip it

Use this page when agent jobs can reach HTTP APIs, ticket systems, cloud CLIs, or package registries. Use it when two people can start jobs on the same runtime without pairing. Skip it for a local notebook that cannot leave localhost and cannot read team secrets.

Cheap shared boxes make the gap worse because drafts and ship work collide on one PATH. Disclosure: This article was prepared as part of MonkeyCode's product outreach. If you already park exploratory jobs on a host with MonkeyCode's free model access and free server option, run that work only after this allowlist page exists, not as a shortcut around it.

One-page wiki run you can paste today

Copy the block below into your team wiki and fill the bracketed fields in one sitting. Do not publish the page until every role has a living person and a deputy.

# Tool-Allowlist Runbook

Owner: [name] (deputy: [name])
Reviewer rotation: [names]
On-call: [rotation calendar]
Last review date: [YYYY-MM-DD]
Runtime hosts in scope: [hostnames]
Default deny: yes

## Allowed tools (exact argv prefixes)
- docs.search --collection=public-kb --read-only
- repo.grep --path=apps/docs --max-hits=20

## Denied by default
- any HTTP client without a listed host
- any ticket, mail, or billing CLI
- any cloud CLI with write verbs
- shell wildcards, eval, and unquoted curl

## Grant request (Job Author fills this)
- Job id:
- Purpose (one sentence):
- Tools requested (exact prefixes):
- Destination hosts:
- Secrets required (names only, never values):
- Production reach: yes/no (must be no for draft jobs)
- Rollback: command to revoke and who runs it

## Reject if
- the tool arrived only in chat history
- the argv prefix is a wildcard
- production reach is unknown
- Owner and Reviewer are the same person

Keep the allowed list short enough to read aloud in a standup. If you cannot read it aloud, you do not understand the blast radius. Re-review the page on a fixed weekday so stale grants cannot hide behind a busy sprint.

Numbered preflight you run before every shared job

Treat the wiki page as policy and the script as the gate. Humans forget; a failing exit code does not. Label the script as a local proposal until you have executed it against your own fixture files.

Freeze the job's declared tools into a YAML file in the same change as the prompt.
Diff that file against the wiki allowlist, not against yesterday's shell history.
Refuse to start when any prefix is missing, wildcarded, or pointing at production hosts.
Export a redacted receipt that names Owner, Reviewer, job id, and the exact argv prefixes.
Start the agent only after the receipt is committed or attached to the tracker ticket.
If the job needs a new tool, stop and file a grant instead of editing PATH live.

Proposed fixture and checker you can save as tool_allowlist_preflight.py:

#!/usr/bin/env python3
"""Fail closed when a job's tools are not on the team allowlist.

Proposal: run locally against fixtures. It does not call vendor APIs.
"""
from __future__ import annotations

import sys
from pathlib import Path

import yaml  # PyYAML

DENY_MARKERS = ("*", "eval", "curl ", "kubectl ", "aws ", "gcloud ")


def load_yaml(path: Path) -> dict:
    data = yaml.safe_load(path.read_text()) or {}
    if not isinstance(data, dict):
        raise ValueError(f"{path} must be a mapping")
    return data


def prefixes_ok(declared: list[str], allowed: list[str]) -> list[str]:
    errors = []
    allowed_set = tuple(allowed)
    for item in declared:
        if any(marker in item for marker in DENY_MARKERS):
            errors.append(f"denied marker in {item!r}")
            continue
        if not any(item == prefix or item.startswith(prefix + " ") for prefix in allowed_set):
            errors.append(f"not allowlisted: {item!r}")
    return errors


def main(argv: list[str]) -> int:
    if len(argv) != 3:
        print("usage: tool_allowlist_preflight.py allowlist.yaml job.yaml", file=sys.stderr)
        return 2
    allowlist = load_yaml(Path(argv[1]))
    job = load_yaml(Path(argv[2]))
    if job.get("production_reach") is not False:
        print("reject: production_reach must be false for shared draft jobs", file=sys.stderr)
        return 1
    errors = prefixes_ok(job.get("tools") or [], allowlist.get("allowed_tools") or [])
    if job.get("owner") == job.get("reviewer"):
        errors.append("owner and reviewer must be different people")
    if errors:
        print("reject:", file=sys.stderr)
        for line in errors:
            print(f"- {line}", file=sys.stderr)
        return 1
    print(
        "pass:"
        f" job={job.get('job_id')}"
        f" owner={job.get('owner')}"
        f" reviewer={job.get('reviewer')}"
        f" tools={len(job.get('tools') or [])}"
    )
    return 0


if __name__ == "__main__":
    raise SystemExit(main(sys.argv))

Fixture files keep the check reviewable in the same pull request as the prompt. Proposed allowlist.yaml and job.yaml look like this.

# allowlist.yaml
allowed_tools:
  - docs.search --collection=public-kb --read-only
  - repo.grep --path=apps/docs --max-hits=20

# job.yaml
job_id: docs-draft-2026-09-22
owner: sam
reviewer: renee
production_reach: false
tools:
  - docs.search --collection=public-kb --read-only

Run the gate from a clean shell so leftover aliases cannot hide extra clients:

python3 -m pip install --user pyyaml
python3 tool_allowlist_preflight.py allowlist.yaml job.yaml
echo "exit=$?"

A passing run prints a one-line receipt you can paste under the ticket. A failing run must block the job start, not become a warning people scroll past. If your orchestrator cannot fail closed, you do not have a fence yet.

Decision table for grant requests

Use this table in review comments so approvals stay consistent across time zones. If a request does not map to a row, the answer is reject and rewrite.

Request shape	Owner action	Reviewer action	Job start
Exact argv prefix, read-only, non-prod host	May grant	Confirm host and verb	Allowed after receipt
Same tool, new collection or path	Treat as a new grant	Re-check data class	Block until re-approved
Wildcard host or unquoted shell	Reject	Reject	Never
Write verb on tickets, mail, or billing	Reject for draft jobs	Escalate only with a ship ticket	Never on the shared draft host
Tool appeared only in chat	Reject	Ask for YAML	Never
Owner equals Reviewer	Invalid	Invalid	Never

The table is the artifact reviewers can apply without rereading the whole philosophy. You want disagreements to happen on a row, not on vibes about how smart the model felt yesterday.

What you log, and what you never log

Write job id, people, argv prefixes, destination hosts, and the git SHA of the allowlist. Never write secret values, raw tokens, or full prompt bodies that contain customer data. If a grant is revoked, append a line with time, on-call name, and the reason, then rotate the named secret even when you think the job failed early.

Keep receipts next to the job definition so a later incident has a file, not a memory. Chat search is not an audit log, and neither is a screenshot of a green terminal. If you cannot reconstruct which tools were legal last Tuesday, you cannot honestly say the fence worked.

Limitations, and who should not use this

This playbook does not replace network policy, IAM roles, or a real secret manager. A Python prefix check cannot see a binary that shells out under another name. It also cannot stop a tool that is allowlisted and then used with surprising arguments that still match the prefix. You still need host firewalls, scoped tokens, and production credentials that never land on the draft runtime.

Do not use this approach if you are a solo hobbyist on localhost with no outbound tools. Do not use it as cover for putting production write keys on a shared exploratory server. Do not use it when your orchestrator ignores exit codes, because a documented reject path that cannot fire is theater. Regulated workloads that forbid third-party runtimes need your legal and security path, not a wiki page from a blog post.

You should also refuse the pattern if nobody will do the weekday review. An allowlist that only grows is a backlog of unowned privileges. In that case, delete tools until the page fits in one screen, or stop sharing the runtime.

Close the loop after the first blocked job

The first useful moment is the first reject, not the first clever agent demo. When the script blocks a job, you file the grant, argue on the decision table, and either widen the list in public or tell the author to shrink the tools. You then rerun the preflight and attach the receipt so the next person can copy the motion without asking you in chat.

Paste the wiki page today, then refuse the next tool grant that arrives as a screenshot. Shared agents can draft against a narrow, named surface, but they should not discover your vendor APIs by accident.

Name a Lane Owner: A One-Page Wiki SOP Before Free-Model Drafts Mix With Ship Work

Alex Zhu — Mon, 21 Sep 2026 12:41:57 +0000

You open the shared board on Monday and find three tickets marked done by an overnight agent run. The comments look confident, the diffs are large, and nobody can say which model or machine produced them. A teammate almost merged one change because the summary read like careful human work.

Later that afternoon, staging failed because the run had used fixture data and a looser tool allowlist. You did not lack another model, another prompt, or another late-night retry. You lacked a named owner for the boundary between sandbox lanes and ship lanes.

This playbook gives you that owner, a handoff map, and a one-page wiki run. You can paste the SOP into an internal wiki and enforce it with a small check script. Treat the script as an example you adapt, not as a production control plane.

Why mixed lanes fail quietly

Shared agent work usually starts on a cheap or free path because tickets are messy and humans are tired. That is reasonable for exploration, reproduction, and first drafts that nobody should trust yet. Trouble starts when the same queue, labels, and bots treat sandbox output as ship-ready evidence.

You then get three failure modes that look like productivity from a distance. First, reviewers skim agent prose and skip the command that would prove the environment. Second, retry storms from the sandbox lane steal capacity from jobs that already passed a human gate. Third, fixture credentials and production secrets drift onto the same runtime because someone said it was only a draft.

A lane owner does not pick the smartest model for the team. The lane owner keeps two roads visible, named, and boring enough that midnight operators can follow them. If you cannot point at a person who may recategorize a run, you do not have a lane. You only have a hope that chat history will remember the difference.

Roles and handoffs

Name people, not chat threads. If a role is vacant, the SOP is not in force, and you should stop launching shared agent jobs.

Lane Owner — decides sandbox versus ship for each agent job, and can recategorize a run after the fact.
Ticket Driver — the human who asked for the work and must attach a receipt before asking for review.
Runtime Steward — keeps sandbox machines, free-model endpoints, and ship CI from sharing disks, secrets, and queues.
Reviewer of Record — a human who may reject "done" if the lane label is missing, stale, or contradicted by the receipt.

Handoff rule: sandbox output never assigns the Reviewer of Record by itself. The Ticket Driver promotes a run by adding evidence, not by pasting a longer summary into the ticket. If the Lane Owner is away, the backup owner must already be named on the wiki page, or the queue stays closed.

Write the names in the same meeting where you adopt the page. Do not wait for a perfect org chart, because mixed-lane failures happen on the first unsupervised night. A vacant backup is a hidden single point of failure, and you should treat it as an incident waiting for a quiet weekend.

Decide the lane before the first token

Use this decision table as a proposal. If two columns conflict, choose ship controls or do not run the job.

Question	Sandbox lane	Ship lane
May the model or server change under you?	Yes	No, pin and record the stack
Are production secrets in scope?	No	Only through approved CI brokers
Is the output allowed to close a ticket?	No	Only after a human receipt
May tools write to main or prod?	No	Behind merge and deploy gates
Is retry-on-failure automatic?	Bounded, then stop	Bounded, then page a human
Can the job share a scratch disk with drafts?	Yes, if wiped	No

You should print this table above the wiki SOP so operators do not invent a third lane at 1 a.m. A third lane is how fixture data, unpaid experiments, and merge candidates collapse into one comment thread. If a job needs one ship property and one sandbox property, split it into two jobs instead of bargaining.

One-page wiki SOP (paste this)

Copy the block below into your team wiki. Fill names on the same day you adopt it, then keep the page to a single screen.

# Lane SOP (sandbox vs ship)

Lane Owner: [name]
Runtime Steward: [name]
Backup Lane Owner: [name]
Reviewers of Record: [names]

## Before any agent job
1. Label the ticket `lane:sandbox` or `lane:ship`. No label means no run.
2. Sandbox jobs use the free-model / free-server path only.
3. Ship jobs use pinned CI, recorded tool allowlists, and no shared scratch disks.
4. Paste the lane, runtime, and exact command into the ticket before launch.

## During the run
1. If the job needs production data, stop and recategorize; do not "just this once."
2. If the sandbox path is saturated, queue or drop sandbox jobs. Never steal ship capacity.
3. If output will be used as a spec, test, or runbook, it has already left sandbox.

## After the run
1. Ticket Driver attaches a receipt: command, exit code, artifact path, and lane label.
2. Lane Owner may demote a ship run back to sandbox if the receipt is incomplete.
3. Reviewer of Record ignores unlabeled "done" comments, including confident ones.

## Promotion (sandbox -> ship)
Promotion is a new job, not a label edit. Re-run with ship controls, then review.

That page is the entire operating agreement for mixed agent work. If your wiki grows past one screen, you are adding folklore again, and folklore is how sandbox drafts become canon. Keep links to longer runbooks off this page, or people will stop reading the only section that matters during an incident.

What a receipt must contain

A receipt is a small text artifact, not a paragraph of model self-praise. You want a future teammate to replay the claim without opening a chat transcript. Store it next to the ticket identifier so the lane check can find it without crawling message history.

ticket=PAY-4418
lane=ship
runtime=ci-pinned
allowlist=tools.allow
cmd=pytest tests/test_refund_paths.py -q
exit=0
artifact=artifacts/PAY-4418.xml
closed_ticket=false

If closed_ticket is true on a sandbox receipt, the Lane Owner should demote the run in public on the ticket. Do not debate tone, because the failure is categorical. You are teaching the queue that labels beat eloquence, and you only get that lesson if demotion is visible.

A reproducible lane check you can run today

The following example is unexecuted here. You should run it in a throwaway directory and read the exit codes before wiring it to chatbots or webhooks.

Create lane_check.py:

#!/usr/bin/env python3
"""Example lane gate. Adapt labels to your tracker; do not treat this as policy."""

from __future__ import annotations

import json
import os
import sys
from pathlib import Path

ALLOWED_LANES = {"sandbox", "ship"}
SHIP_REQUIRED = ("LANE_RUNTIME", "LANE_ALLOWLIST", "LANE_RECEIPT_DIR")


def load_ticket(path: Path) -> dict:
    data = json.loads(path.read_text(encoding="utf-8"))
    if not isinstance(data, dict):
        raise ValueError("ticket JSON must be an object")
    return data


def main() -> int:
    if len(sys.argv) != 2:
        print("usage: lane_check.py TICKET.json", file=sys.stderr)
        return 2
    ticket = load_ticket(Path(sys.argv[1]))
    lane = str(ticket.get("lane", "")).strip().lower()
    if lane not in ALLOWED_LANES:
        print("reject: missing or unknown lane", file=sys.stderr)
        return 1
    if lane == "sandbox" and ticket.get("closes_ticket"):
        print("reject: sandbox output cannot close a ticket", file=sys.stderr)
        return 1
    if lane == "ship":
        missing = [name for name in SHIP_REQUIRED if not os.environ.get(name)]
        if missing:
            print(f"reject: ship lane missing {missing}", file=sys.stderr)
            return 1
        receipt = Path(os.environ["LANE_RECEIPT_DIR"]) / f"{ticket.get('id', 'unknown')}.txt"
        if not receipt.is_file():
            print(f"reject: no receipt at {receipt}", file=sys.stderr)
            return 1
    print(f"ok: lane={lane} ticket={ticket.get('id')}")
    return 0


if __name__ == "__main__":
    raise SystemExit(main())

Keep a tiny ticket fixture so newcomers can fail the check on purpose:

{
  "id": "PAY-4418",
  "lane": "sandbox",
  "closes_ticket": true
}

Commands you can run locally:

python3 lane_check.py ticket.json
echo $?

mkdir -p /tmp/lane-receipts
export LANE_RUNTIME=ci-pinned
export LANE_ALLOWLIST=tools.allow
export LANE_RECEIPT_DIR=/tmp/lane-receipts
printf 'cmd=pytest\nexit=0\nlane=ship\n' > /tmp/lane-receipts/PAY-4418.txt

python3 - <<'PY'
from pathlib import Path
Path("ticket-ship.json").write_text(
    '{"id":"PAY-4418","lane":"ship","closes_ticket":true}\n',
    encoding="utf-8",
)
PY
python3 lane_check.py ticket-ship.json

If the first command does not exit 1, your copy of the script is not enforcing the SOP. You should fix the gate before you teach people to trust it. A green check that ignores closes_ticket is worse than no check, because it launders sandbox work into the ship column.

Optional wrapper for humans who live in the shell:

#!/usr/bin/env bash
set -euo pipefail
ticket="${1:?ticket json}"
lane="$(python3 -c 'import json,sys; print(json.load(open(sys.argv[1]))["lane"])' "$ticket")"
if [[ "$lane" == "sandbox" ]]; then
  echo "sandbox: free-model path only, no production secrets, no ticket close"
fi
if [[ "$lane" == "ship" ]]; then
  : "${LANE_RUNTIME:?}" "${LANE_ALLOWLIST:?}" "${LANE_RECEIPT_DIR:?}"
  echo "ship: pinned runtime only"
fi
python3 lane_check.py "$ticket"

Wire this wrapper to your tracker only after five manual failures have been recorded on real tickets. Automation without those scars will encode the wrong social rule, and you will spend a month arguing with a bot instead of naming an owner.

Where a free model path belongs

You need a sandbox lane that is allowed to be slower, cheaper, and occasionally wrong. MonkeyCode's free model access and free server option can sit in that sandbox lane as a place for drafts, reproductions, and throwaway tool trials.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Do not point that path at production secrets, customer exports, or merge queues. The value is isolation, not a promise about models, quotas, uptime, or hardware. If the free path disappears tomorrow, the SOP should still name a sandbox lane and a ship lane, because the failure mode is mixed evidence, not missing tokens.

The Runtime Steward should treat the free server as disposable scratch, with a wipe rule after each job. That is the opposite of using it as quiet overflow for ship work when CI is slow. Overflow is how lanes collapse, and collapsed lanes recreate the Monday board you started with.

Limitations, and who should skip this

This SOP does not measure model quality, and it does not replace security review, license review, or incident command. The example script only reads a JSON file and a few environment variables. It cannot see your chat tools, your Kubernetes queue, or a model that changed its behavior overnight.

You should not adopt this if you are a solo hobbyist with one machine and no shared tickets. You should also not adopt it if your rules already forbid third-party model endpoints, or if you were hoping a free server would become silent production capacity. Mixed-lane failures are social failures. A wiki page without a named owner will not save you, and a named owner without receipts is only a title.

Skip the approach when the work is a legal hold, a production incident with customer data, or a change that needs a formal change-advisory record. Those jobs start in the ship lane or they do not start. Sandbox theater during an incident wastes the only people who can still read the system.

A one-week rollout that stays small

Put names on the wiki page during the next standup, including a backup Lane Owner who can recategorize runs.
Add lane:sandbox and lane:ship to the tracker, and reject unlabeled agent comments for seven days.
Run lane_check.py on five real tickets; keep the failures in the ticket, not in a slide deck.
Only after those five receipts exist, connect a sandbox path such as a free-model server to lane:sandbox.

If you already have a free-model sandbox, take one labeled ticket this week and force it through the receipt step before anyone calls the work done. That single boring pass is the whole lesson: the lane is real only when a missing file can stop a confident agent.

Name an Oracle Owner: A One-Page Wiki SOP When Green Tests Stop Meaning Pass

Alex Zhu — Sun, 20 Sep 2026 10:57:13 +0000

Your teammate merged an agent patch that painted the entire suite green in twelve quiet minutes. Nobody on the call could explain why staging still rejected VAT-inclusive carts during the afternoon checkout drill. The same agent had rewritten the failing tests, so assertions now described the patch instead of the contract. You watched a green pipeline become a story the team tells itself after every shared run.

You do not have a coverage gap in that moment; you have an oracle problem sitting in git. An oracle is the set of checks that must not change in the same job as the implementation. If the model edits tests and production code together, a pass is a tautology rather than evidence. Name one owner before the next agent queue opens, or that tautology will keep shipping into main.

Give the role a boring, visible name

Call this person the Oracle Owner, and put the name on the team wiki above the runbook. They do not write every test, and they do not approve every product change by themselves. They decide which files are frozen evidence, who may thaw them, and how dual-runs get recorded. Without a published name, people argue in pull requests after the agent has already moved the goalposts.

Hand the role to someone who already reads production incidents, not to whoever launched the latest agent thread. Product managers can nominate customer behaviors; the Oracle Owner translates those behaviors into files that CI refuses to rewrite. Rotate the assignment on a published calendar if the review burden stays uneven across sprints. A blank ownership cell is exactly how last Tuesday's VAT checkout bug reached main without debate.

What you freeze, and what you allow the agent to touch

You should treat generated tests as notes the model wrote to itself, not as proof of the behavior you ship. Proof lives in the locked suite, which you run twice: once against the patch, and once against main. If both runs pass while staging still fails, your oracle is too thin, and the owner owes a frozen case. If the patch run fails, you do not repair the oracle inside the same agent job that produced the diff.

Freeze customer-visible contracts such as HTTP status maps, price rounding, tax inclusion, auth redirects, and migration rollbacks.
Allow the agent to add characterization tests under tests/generated/, which must never gate merge by themselves.
Keep a short allowlist of oracle paths in oracle/LOCKFILE, checked into git beside the frozen suite.
Require an ORACLE-CHANGE: git trailer plus the Oracle Owner as reviewer before those frozen paths may differ.

Why a shared free runtime makes the job sharper

Local green still lies, but at least the filesystem, locale, and interpreter match the developer laptop. Shared free servers drift in small ways that your laptop will not reproduce during a normal afternoon. Missing system libraries, a different default timezone, or a patched interpreter can flip a single assertion. You want the locked suite executed in that environment before you trust an agent diff produced there.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. If your team already routes shared agent work through MonkeyCode, the free model access and the free server option can host the dual-run. Keep oracle files read-only in that remote job, and let the model propose implementation patches only. Bring the pytest report back to the ticket, rather than a screenshot of a chat transcript.

You should not send customer records, production dumps, or live secrets to any shared free server. Stub those dependencies inside the oracle suite so the remote run exercises contracts rather than vendor accounts. If your compliance team forbids off-laptop execution, run the same SOP on a self-hosted runner instead. Leave the free server path unused rather than arguing with a policy you cannot change this quarter.

Numbered run you can execute this afternoon

Create oracle/ and move three checks that failed in production last quarter, not vanity unit tests from a tutorial.
Add oracle/LOCKFILE listing those paths, then protect the directory with CODEOWNERS and a required human reviewer.
Install a trailer check so CI fails when oracle files change without ORACLE-CHANGE: on the merge commit.
Add make oracle that runs only the locked suite, with timezone UTC and a pinned locale in the recipe.
On every agent job, run make oracle against main, run it again against the branch, and attach both reports.
If those reports diverge, the Oracle Owner files a ticket; the agent author does not edit oracle/ to restore green.
Once a week, replay the locked suite on the shared free server and diff the report hash against the laptop run.

That seventh step is the one most teams skip, and it is where remote and local skew quietly hide. You are not chasing performance numbers in this ritual; you are chasing identical pass and fail sets. A single extra failure on the free server signals that the agent used a laptop-only tool or hidden file. Treat that mismatch as a handoff into the wiki thread, not as noise you can dismiss after standup.

Artifact: a guard, a pinned recipe, and a hash diff

The following script is a starting point you should adapt, not a scored benchmark or a vendor integration. It fails the build when oracle paths change without the required trailer, then prints a list for the wiki. Label it as unexecuted until you have run it against a throwaway branch on your own repository.

#!/usr/bin/env python3
"""oracle_guard.py — proposal for CI. Adapt before you rely on it."""
from __future__ import annotations

import pathlib
import subprocess
import sys

ROOT = pathlib.Path(__file__).resolve().parents[1]
LOCKFILE = ROOT / "oracle" / "LOCKFILE"
TRAILER = "ORACLE-CHANGE:"


def git(*args: str) -> str:
    result = subprocess.run(
        ["git", *args],
        cwd=ROOT,
        check=True,
        text=True,
        capture_output=True,
    )
    return result.stdout


def locked_paths() -> set[str]:
    lines = LOCKFILE.read_text(encoding="utf-8").splitlines()
    return {
        line.strip()
        for line in lines
        if line.strip() and not line.startswith("#")
    }


def changed_files(against: str) -> set[str]:
    out = git("diff", "--name-only", against)
    return {line.strip() for line in out.splitlines() if line.strip()}


def head_message() -> str:
    return git("log", "-1", "--pretty=%B")


def main() -> int:
    if not LOCKFILE.exists():
        print("oracle/LOCKFILE missing; refusing to guess the frozen set", file=sys.stderr)
        return 2
    base = sys.argv[1] if len(sys.argv) > 1 else "origin/main"
    overlap = sorted(changed_files(base) & locked_paths())
    if not overlap:
        print("oracle paths unchanged")
        return 0
    if TRAILER not in head_message():
        print("oracle files changed without ORACLE-CHANGE: trailer:")
        for path in overlap:
            print(f"  - {path}")
        return 1
    print("oracle change explicitly labeled:")
    for path in overlap:
        print(f"  - {path}")
    return 0


if __name__ == "__main__":
    raise SystemExit(main())

Keep the lockfile boring and short enough that a reviewer can read it in one sitting.

# oracle/LOCKFILE
oracle/test_tax_inclusion.py
oracle/test_checkout_status_map.py
oracle/test_auth_redirect.py

Pair the guard with a Makefile target that pins environment values the agent is not allowed to reinterpret.

.PHONY: oracle oracle-against
export TZ := UTC
export LC_ALL := C.UTF-8
export PYTHONHASHSEED := 0

oracle:
    mkdir -p artifacts
    python -m pytest oracle -q --junitxml=artifacts/oracle-local.xml

oracle-against:
    mkdir -p artifacts
    python -m pytest oracle -q --junitxml=artifacts/oracle-remote.xml

On the laptop and later on the shared free server, you can compare report identity without turning the ritual into a benchmark harness.

make oracle
cp artifacts/oracle-local.xml artifacts/oracle-laptop.xml
# after the same commit is checked out on the shared free server:
make oracle-against
sha256sum artifacts/oracle-laptop.xml artifacts/oracle-remote.xml

A CI sketch belongs beside those commands so the trailer rule is not a local honor system. The workflow below is a proposal you should edit for your branches and Python setup.

# .github/workflows/oracle.yml — proposal, unexecuted here
name: oracle
on:
  pull_request:
jobs:
  locked-suite:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0
      - uses: actions/setup-python@v5
        with:
          python-version: "3.12"
      - run: pip install pytest
      - run: python scripts/oracle_guard.py origin/${{ github.base_ref }}
      - run: make oracle

Finish the handoff with a CODEOWNERS line that makes refusal visible in the pull request UI.

/oracle/ @your-oracle-owner

Replace the owner handle with the person named on the wiki, not with a rotating group nobody reads. You want a person who can refuse a thaw at four in the afternoon, not a silent team alias. Put the CODEOWNERS change in the same pull request as LOCKFILE so the constraint arrives with the suite. Reviewers will see the required owner on the files changed tab before they debate the implementation.

Decision table for the afternoon handoff

Print the table under the owner name so the next failure already has a row instead of a thread. You will argue less in Slack because the handoff is already written as an action, not a vibe. Keep the table boring; cleverness here usually means someone will skip the row under time pressure. If a situation is not on the table, the Oracle Owner adds a row the same day, not next quarter.

Observation	Owner action	Agent author action	Merge
Locked suite fails on the branch	Keep oracle frozen; open a product ticket if the contract itself is wrong	Change implementation or abandon the job	No
Locked suite passes, generated tests fail	Ignore generated tests for merge	Optional cleanup only	Yes, if review is otherwise clean
Locked suite passes locally, fails on the free server	Capture both JUnit files; inspect timezone, locale, and missing binaries	Do not rewrite oracle to match the server	No until hashes match or the skew is documented
Oracle files appear in the agent diff	Reject the job	Revert oracle paths; request an `ORACLE-CHANGE:` follow-up	No
Both environments pass, staging still fails	Oracle is too thin; owner adds one frozen case from the incident	Wait	No

Limitations, and who should skip this

This SOP will not save a team that has no written product contract outside the test folder. If your only specification is the code, freezing tests simply freezes today's bugs and names them evidence. Do not use a shared free server when the oracle needs production data, licensed corpora, or customer fixtures. Do not appoint a junior on-call as Oracle Owner during their first incident rotation under pager noise.

The guard script does not prove correctness, and it does not replace contract tests against real providers. It only stops one cheap lie: the agent grading its own homework inside the same diff as the patch. If your models only draft comments or documentation, you do not need this role on the wiki yet. If two human reviewers already refuse test-only green, you may already be covered without new ceremony.

One-page wiki paste

Copy the block below into your team wiki and fill the brackets the same day you name the owner. Fill those brackets in front of the team, not inside a private document that nobody can find later. The value is the social constraint on who may move the goalposts, not the formatting of the YAML. When the next agent queue opens, you will already know who is allowed to thaw the frozen suite.

# Oracle Owner runbook

Owner this sprint: [name]
Deputy: [name]
Frozen paths: oracle/ plus oracle/LOCKFILE
Thaw rule: ORACLE-CHANGE: trailer AND owner review
Local command: make oracle
Remote command: make oracle-against on the shared free server
Handoff artifact: both JUnit files attached to the ticket
Never: agent job edits oracle/ in the same branch as implementation
Weekly: replay locked suite on the free server; diff report hashes
Escalation: if staging fails while oracle is green, owner adds one frozen case within one business day

If you already send shared agent jobs through a free model runtime, run this SOP against one service this week. Keep the JUnit pair beside the diff so the next reviewer can see whether green still means pass. That experiment is small enough to finish before standup, and honest enough to change how you merge. Do not expand the frozen set until that first service has survived a week of agent jobs without silent thaws.

Name a Merge-Readiness Owner: A One-Page Wiki SOP Before Agent Diffs Hit Main

Alex Zhu — Sat, 19 Sep 2026 09:12:10 +0000

Last Tuesday your teammate opened a twelve-file pull request that an overnight coding agent had drafted alone. Continuous integration reported green, the description quoted the chat transcript, and nobody on the rotation had actually read the diff. By Wednesday morning a silent retry loop was saturating a shared staging queue, and the original session logs had already scrolled away. You do not need another model comparison; you need a named Merge-Readiness Owner before that pattern repeats.

Why merge readiness is a job, not a feeling

Agent sessions collapse planning, coding, and self-review into one transcript that looks complete from the outside. Green tests often measure the agent's own scaffolding rather than the contract your production callers actually require. A large generated diff can still look correct in pieces while remaining unreviewable, which is how pretend-engineering reaches main. Naming an owner turns that fuzzy discomfort into a stoppable checklist that still requires a human signature.

You already have code owners for directories and on-call owners for incidents, so this role should feel familiar. The Merge-Readiness Owner does not write every patch, and they should not rubber-stamp comments the agent generated. They decide whether a change is small enough, evidenced enough, and reversible enough to enter the shared integration branch. If they are away, a documented backup must hold the same veto, because an unsigned agent pull request is just a draft.

Role, backup, and the handoff window

Write the names in your team wiki before the next shared agent run, not after a bad merge. Keep the rotation weekly so the job does not stick to the person who first complained about unreadable diffs. Treat handoff as a five-minute ritual at the start of the owner window, with the previous owner pasting unfinished evidence packs.

Primary owner: the engineer who may mark merge-readiness for agent-assisted pull requests during this week.
Backup owner: the engineer who inherits the same veto if the primary stays offline more than two working hours.
Producer: whoever launched the agent session, which may be the primary owner or a different teammate.
Integrator: the person who presses merge only after the owner posts a complete evidence pack.

The producer may be the owner, but you should still fill both fields so the audit trail stays readable. When the producer and owner are the same person, the backup must glance at the evidence pack before merge. That extra pair of eyes is the whole point of the ritual, and it should not be treated as optional paperwork.

The one-page wiki SOP you can paste

Copy the block below into your team wiki and replace the placeholders with this week's names. Do not expand the page into a novel, because a wall of text will not stop a Friday merge. Keep the stop conditions boring and numerical so an exhausted backup can apply them after dinner.

# Merge-Readiness SOP (agent-assisted PRs)

## Owners this week
- Primary: @name
- Backup: @name
- Escalation (engineering manager): @name
- Owner window: Monday 10:00 to Friday 16:00 local

## Applies when
- Any pull request where an agent produced more than one source file
- Any pull request whose description is mainly a chat export
- Any change that touches shared CI, migrations, auth, or queue consumers

## Ready-to-merge means all of the following
1. Diff budget: <= 400 changed lines OR an explicit owner waiver in the PR
2. File budget: <= 8 touched production files, excluding generated lockfiles
3. Human review note: a HUMAN_REVIEW section in the PR body, written by a person
4. Repro: one command the backup can run without the original chat
5. Rollback: one revert plan that does not require re-opening the agent
6. Secrets scan: no new .env, token, or host files in the diff
7. Signature: primary or backup comment `READINESS: PASS` on the PR

## Hard stops
- Agent-written tests are the only new tests
- The session cannot be replayed from committed prompts or scripts
- The producer cannot explain the diff in five minutes
- CI was greened by skipping a required check

## Handoff
- Unfinished packs move to the backup with a link and a deadline
- After Friday 16:00, agent-assisted merges wait until the next window

That page is the working contract for the week, and every later command exists only to make it executable. Store a link to the page in the pull request template so producers cannot claim they never saw the budgets. If your template already asks for screenshots, add the readiness marker beside that field instead of creating a second form.

A numbered run for the next agent pull request

Walk this sequence in order, and keep the pull request in draft when any step fails. Green CI does not skip a step, and a complete chat export does not skip a step either. You are measuring whether a human can still own the change after the session disappears.

Freeze the producer session and commit the prompts or command files that actually produced the branch.
Ask the producer for a five-minute walkthrough of the behavioral change, not a tour of the transcript.
Run your usual test target plus one command that exercises the production path the diff claims to fix.
Count changed lines and production files, then compare those counts against the wiki budgets before style talk.
Reject the pack when the only new tests sit beside agent-generated helpers that merely mirror the implementation.
Require a rollback note that names the revert command, the feature flag, or the forward-fix owner.
Scan the diff for secrets, credentials, and machine-specific paths before anyone debates naming or formatting.
Post READINESS: PASS or READINESS: BLOCK with failing step numbers, then stop adding optional commentary.

You should record the step that failed, because a vague cleanup comment returns as another huge agent rewrite. Specific step numbers keep the producer from restarting the whole session as a stalling tactic. The owner may waive the line budget, but the waiver must quote a reason the backup can reread later.

Artifact: a merge-readiness contract and a local checker

Keep a committed YAML contract next to the wiki so the budgets cannot drift inside chat threads. The file below is a starting point you can tighten after two weeks of real pull requests. Tune the numbers using your own history rather than copying another team's thresholds from a talk.

# merge_readiness.yml
version: 1
diff_line_budget: 400
production_file_budget: 8
require_human_review_heading: true
forbid_only_agent_tests: true
required_pr_marker: "READINESS: PASS"
hard_stop_paths:
  - ".env"
  - "token"
  - "credentials"
waiver_label: "readiness-waiver"

The checker below is a local helper, not a security scanner and not a proof of correctness. Run it from the repository root against your merge base so the owner and the producer see the same numbers. Treat binary files as budget exceptions in the waiver, not as silent skips inside the script.

#!/usr/bin/env python3
"""Merge-readiness helper for agent-assisted pull requests.

This script measures reviewability budgets. It does not approve merges.
"""
from __future__ import annotations

import argparse
import subprocess
import sys
from pathlib import Path

try:
    import yaml
except ImportError:
    yaml = None

DEFAULTS = {
    "diff_line_budget": 400,
    "production_file_budget": 8,
    "hard_stop_paths": [".env", "token", "credentials"],
}


def git(*args: str) -> str:
    result = subprocess.run(
        ["git", *args],
        check=True,
        capture_output=True,
        text=True,
    )
    return result.stdout


def load_contract(path: Path) -> dict:
    if not path.exists() or yaml is None:
        return DEFAULTS
    data = yaml.safe_load(path.read_text()) or {}
    merged = dict(DEFAULTS)
    merged.update(data)
    return merged


def changed_files(base: str) -> list[str]:
    output = git("diff", "--name-only", base)
    return [line for line in output.splitlines() if line]


def line_stats(base: str) -> tuple[int, int]:
    output = git("diff", "--numstat", base)
    added = deleted = 0
    for line in output.splitlines():
        parts = line.split("\t")
        if len(parts) < 3 or parts[0] == "-" or parts[1] == "-":
            continue
        added += int(parts[0])
        deleted += int(parts[1])
    return added, deleted


def looks_like_secret(path: str, needles: list[str]) -> bool:
    lowered = path.lower()
    return any(needle.lower() in lowered for needle in needles)


def main() -> int:
    parser = argparse.ArgumentParser()
    parser.add_argument("--base", default="origin/main")
    parser.add_argument("--contract", default="merge_readiness.yml")
    args = parser.parse_args()

    contract = load_contract(Path(args.contract))
    files = changed_files(args.base)
    added, deleted = line_stats(args.base)
    changed_lines = added + deleted
    production = [
        path for path in files
        if not path.endswith((".lock", ".sum")) and "/generated/" not in path
    ]

    failures: list[str] = []
    if changed_lines > int(contract["diff_line_budget"]):
        failures.append(
            f"line budget {changed_lines} > {contract['diff_line_budget']}"
        )
    if len(production) > int(contract["production_file_budget"]):
        failures.append(
            f"file budget {len(production)} > {contract['production_file_budget']}"
        )
    for path in files:
        if looks_like_secret(path, contract["hard_stop_paths"]):
            failures.append(f"hard-stop path in diff: {path}")

    print(f"files={len(files)} production={len(production)} lines={changed_lines}")
    if failures:
        print("READINESS: BLOCK")
        for item in failures:
            print(f"- {item}")
        return 1
    print("READINESS: CHECKS_OK (human signature still required)")
    return 0


if __name__ == "__main__":
    sys.exit(main())

Install PyYAML only if you want the YAML contract to load; otherwise the script uses the built-in defaults. Example commands for the owner window are ordinary git and GitHub CLI calls you already trust.

python3 -m pip install --user pyyaml
git fetch origin
python3 merge_readiness_check.py --base origin/main
gh pr comment "$PR" --body "READINESS: BLOCK
- step 4 file budget
- step 6 rollback note missing"

Label the output honestly when you paste it, because a green checker is not a merge approval. CHECKS_OK means the budgets passed, not that the change is correct, and not that the owner has signed. The human comment remains mandatory even when the script exits zero on a clean worktree.

Where a shared free runtime fits

Some teams generate the evidence pack on a laptop that locks, which leaves the backup unable to rerun the checker overnight. If you already use a shared agent workspace for housekeeping, keep the contract and the script in git. Run the same commands in that workspace so the backup can repeat the numbers without the producer's laptop.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode's free model access and free server option can host that rerun when you need a room that is not tied to one sleeping laptop. Use it to regenerate the five-minute summary and the rollback note from committed files, then paste results into the pull request. Do not let the shared runtime post READINESS: PASS for you, because that signature is still the owner's job.

Limitations, and who should not use this SOP

This playbook measures reviewability, not product quality, latency, or security beyond a naive path check. The line budget will be wrong for generated protobufs, snapshot tests, and lockfile bumps, so waive those classes in writing. A wiki page cannot replace CODEOWNERS, required reviewers, or a change-management ticket when your organization already mandates them. Treat the checker as a ruler for diff size, not as a secret scanner and not as a proof of behavior.

You should skip this SOP if you are a solo developer merging to a private branch with no shared runtime. You should also skip it when a staffed review rotation already blocks unreadable diffs without extra ceremony. A second owner on top of a working review SLA will mostly add latency and duplicate comments. Regulated environments that need signed artifacts and retained session recordings need a stronger control plane than a wiki. Treat agent-assisted infrastructure changes, including IAM edits and production data backfills, as out of scope until a human writes the plan.

The useful part of the vibe-coding debate is that a green pipeline is being treated as complete engineering evidence. A named Merge-Readiness Owner is a small way to keep the human veto visible when the transcript looks finished. Paste the wiki page, commit the contract, and refuse to merge unsigned agent diffs even when the dashboard is green.

Name a Stop-Condition Owner: A One-Page Wiki SOP for Runaway Shared Agent Jobs

Alex Zhu — Fri, 18 Sep 2026 07:08:49 +0000

You kick a Friday agent job onto a shared box because the prompt already feels close enough to merge. The first loop looks harmless until files keep changing after dinner and nobody can name who may kill the process. Monday morning the queue is jammed, two teammates have duplicated the same repair, and the originating thread is gone. Shared free runtimes usually fail as operations problems long before they fail as model-quality or prompt-quality problems.

What a runaway job actually is

A runaway job is not only an infinite loop in generated code or a retry storm against a flaky tool. It is also a social failure because the submitter went offline and the runtime still has no named killer. Every bystander then hesitates to send SIGTERM, so you lose wall-clock, disk, and the next person's slot. If your team treats vibe sessions as engineering output, you still need a halt policy a stranger can run at 01:00.

Cheap retries make that social failure sharper, because starting another loop costs almost nothing until the shared queue collapses. A free server option then concentrates those loops onto one machine that several people believe they kind of own. MonkeyCode is one place this pattern shows up, through free model access and a free server option for shared draft runs.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

You do not need a new vendor workflow to repair the gap when Monday arrives. You need a stop-condition owner, a written halt contract, and a watchdog that dies louder than the agent process. The rest of this page is the run you paste into the wiki before anyone launches again.

Roles you name before anyone launches

Stop-Condition Owner. This person is the only teammate who may raise max wall-clock, max steps, or retry depth during the current week. They also reject jobs whose done-sentence cannot be checked without rereading a private chat. If the owner is unclear, you do not have a shared engineering runtime yet.

Job Submitter. This is whoever starts the process, and they must paste a halt contract into the wiki before launch. They do not get to argue that the agent will probably finish after one more tool call. Launch without a contract is treated as an unowned process, not as enthusiasm.

On-call Killer. This backup may send SIGTERM without waiting for the submitter's phone, including overnight. Their job is to stop an unowned process first and reconstruct intent later from the receipt file. Politeness is not a substitute for a kill path on a shared box.

Reviewer. This person checks that the halt contract matches the branch, the allowed write roots, and the intended test command. They are not there to admire the prompt; they are there to confirm the job can end. A review that starts after three hours of writes is a postmortem, not a review.

Handoffs stay explicit on purpose, because informal ownership is how Friday jobs survive into Monday. The submitter cannot also be the only killer overnight when the box is shared. The owner cannot approve an unbounded run with an emoji reaction in chat. The reviewer cannot look later after the job has already been writing files for three hours.

Seven steps you can run this afternoon

Freeze the done-sentence. Write one sentence that a teammate could mark complete without opening the original thread. If you cannot write that sentence, you do not have a stop condition, and the work stays on a laptop. Shared hardware is for jobs with an observable end, not for open-ended browsing.
Fill the halt contract before launch. Require four numbers: max wall-clock seconds, max agent steps, max write paths, and a heartbeat interval. Keep that contract beside the command so the killer does not hunt through chat history. Numbers that live only in someone's head are not numbers.
Name the killer in the same block. Record a person, a backup, and a timezone rather than a group mention nobody reads. If the owner is offline, the backup kills first and files the receipt second. A missing name means the job is already unowned.
Launch under a wrapper, never a raw long-lived shell. The wrapper must exit non-zero when any halt number trips, and it must leave a receipt JSON file. Receipts are the only honest memory Monday-you will have of Friday-you. A tmux session with no receipt is not an operations story you can replay.
Broadcast PID, receipt path, and kill command. Post them in the team channel at launch, not after the first anomaly. Missing broadcast means the on-call killer treats the job as unowned and stops it. Silence is not consent to keep the CPU.
Snapshot, then stop. On halt, copy the receipt, git status --short, and the last 200 log lines into the ticket. Do not restart until the owner edits a number in the contract and initials the change. Almost done is not a halt condition you can hand to a stranger.
Hold a ten-minute post-halt. The only agenda is which number was wrong and whether the done-sentence was fake. A fake done-sentence sends the next attempt back to a personal machine, not the shared server. Repeat failures against the same unbounded prompt are a process bug, not bad luck.

Artifact: halt contract, watchdog, and kill decision table

Treat the following as a proposed local wrapper, not as a platform API and not as a measured benchmark. You can keep it next to any agent CLI you already run on a shared box. Label the script unexecuted until you run it against a dummy sleep job on your own machine.

Halt contract template

# Halt contract — paste above every shared run

- job_id: 2026-09-18-fix-retry-storm
- submitter: riley
- stop_condition_owner: mei
- oncall_killer: sam (UTC+8)
- reviewer: jordan
- done_sentence: Receipt JSON exists and pytest tests/test_retry.py passes.
- max_wall_seconds: 1200
- max_steps: 40
- max_write_paths: 12
- heartbeat_seconds: 30
- allowed_write_roots:
  - /tmp/agent-jobs/2026-09-18-fix-retry-storm
  - /home/shared/work/retry-storm
- launch_command: python3 watchdog.py --config halt.env -- python3 run_agent.py
- kill_command: kill -TERM $(cat /tmp/agent-jobs/2026-09-18-fix-retry-storm/job.pid)

`halt.env` for the wrapper

# halt.env
JOB_ID=2026-09-18-fix-retry-storm
JOB_DIR=/tmp/agent-jobs/2026-09-18-fix-retry-storm
MAX_WALL_SECONDS=1200
MAX_STEPS=40
HEARTBEAT_SECONDS=30

Proposed `watchdog.py` (unexecuted example)

#!/usr/bin/env python3
"""Proposed watchdog for shared agent jobs. Label: unexecuted example."""
from __future__ import annotations

import json
import os
import signal
import subprocess
import sys
import time
from pathlib import Path


def load_env(path: str) -> None:
    for raw in Path(path).read_text().splitlines():
        line = raw.strip()
        if not line or line.startswith("#") or "=" not in line:
            continue
        key, value = line.split("=", 1)
        os.environ.setdefault(key.strip(), value.strip())


def receipt_path(job_dir: Path) -> Path:
    return job_dir / "receipt.json"


def write_receipt(job_dir: Path, payload: dict) -> None:
    job_dir.mkdir(parents=True, exist_ok=True)
    receipt_path(job_dir).write_text(json.dumps(payload, indent=2) + "\n")


def main(argv: list[str]) -> int:
    if "--config" not in argv or "--" not in argv:
        print("usage: watchdog.py --config halt.env -- <command>", file=sys.stderr)
        return 2
    cfg = argv[argv.index("--config") + 1]
    cmd = argv[argv.index("--") + 1 :]
    load_env(cfg)

    job_id = os.environ["JOB_ID"]
    job_dir = Path(os.environ["JOB_DIR"])
    max_wall = int(os.environ["MAX_WALL_SECONDS"])
    max_steps = int(os.environ["MAX_STEPS"])
    heartbeat = int(os.environ["HEARTBEAT_SECONDS"])
    job_dir.mkdir(parents=True, exist_ok=True)

    proc = subprocess.Popen(cmd, cwd=job_dir)
    (job_dir / "job.pid").write_text(str(proc.pid))
    started = time.time()
    steps = 0
    halt_reason = "completed"

    try:
        while True:
            code = proc.poll()
            if code is not None:
                halt_reason = "child_exited"
                break
            elapsed = time.time() - started
            if elapsed > max_wall:
                halt_reason = "max_wall_seconds"
                proc.send_signal(signal.SIGTERM)
                try:
                    proc.wait(timeout=15)
                except subprocess.TimeoutExpired:
                    proc.kill()
                    halt_reason = "max_wall_seconds_sigkill"
                break
            step_file = job_dir / "steps.count"
            if step_file.exists():
                steps = int(step_file.read_text().strip() or "0")
            if steps > max_steps:
                halt_reason = "max_steps"
                proc.send_signal(signal.SIGTERM)
                break
            write_receipt(
                job_dir,
                {
                    "job_id": job_id,
                    "pid": proc.pid,
                    "elapsed_seconds": int(elapsed),
                    "steps": steps,
                    "status": "running",
                },
            )
            time.sleep(heartbeat)
    finally:
        elapsed = int(time.time() - started)
        final_code = proc.poll()
        write_receipt(
            job_dir,
            {
                "job_id": job_id,
                "pid": proc.pid,
                "elapsed_seconds": elapsed,
                "steps": steps,
                "status": halt_reason,
                "exit_code": final_code,
            },
        )
        print(f"halt_reason={halt_reason} receipt={receipt_path(job_dir)}")
    return 1 if halt_reason != "child_exited" else int(final_code or 0)


if __name__ == "__main__":
    raise SystemExit(main(sys.argv[1:]))

Your agent loop should increment steps.count after every tool call so the wrapper can halt on policy, not vibes. That single file is the difference between a killable job and a mystery CPU spike. Dry-run the wrapper with python3 watchdog.py --config halt.env -- sleep 9999 and confirm receipt.json appears before you wrap a real agent.

Commands the on-call killer actually runs

JOB_DIR=/tmp/agent-jobs/2026-09-18-fix-retry-storm
cat "$JOB_DIR/receipt.json"
ps -p "$(cat "$JOB_DIR/job.pid")" -o pid,etime,cmd
kill -TERM "$(cat "$JOB_DIR/job.pid")"
sleep 5
ps -p "$(cat "$JOB_DIR/job.pid")" >/dev/null && kill -KILL "$(cat "$JOB_DIR/job.pid")"
git -C /home/shared/work/retry-storm status --short
tail -n 200 "$JOB_DIR/agent.log"

Decision table for wait, kill, or refuse restart

Observation	Owner action	Killer action	Restart allowed?
Heartbeat older than two intervals	Read receipt only	SIGTERM if the PID is still live	No, until a contract edit
Steps exceed `max_steps`	Narrow scope, or raise the cap with initials	Stop immediately	No
Writes outside allowed roots	Treat the run as an incident	SIGKILL, then snapshot	No
Done-sentence already true	Close the job in the wiki	Do not wait for more agent talk	Not applicable
Submitter offline, no broadcast	Mark the job unowned	Stop, then file the ticket	No

The table is the handoff. Chat debate after a halt is how unbounded jobs get a second life they did not earn.

One-page wiki block to paste

# Stop-condition run (shared agent server)

Purpose: every shared job has a done-sentence, a halt number, and a named killer.

This week
- Stop-condition owner:
- On-call killer:
- Reviewer:

Before launch
1. Paste halt contract with job_id, done-sentence, four numbers, write roots.
2. Run watchdog.py; never start a raw long-lived agent shell.
3. Post PID, receipt path, and kill_command in #agent-jobs.

During run
1. Killer checks receipt.json every heartbeat * 2.
2. Any missing broadcast is an unowned job and gets SIGTERM.
3. Nobody raises limits in chat; only the owner edits the contract.

After halt
1. Attach receipt, git status, and tail -n 200 of the log.
2. Ten-minute post-halt: which number failed, was the done-sentence fake?
3. Fake done-sentence => next attempt stays off the shared server.

Do not
- Leave Friday jobs running without a killer timezone.
- Restart because it was almost done.
- Store secrets in the job directory that the watchdog snapshots.

Fill the three names before the next launch, then refuse jobs that arrive with blank lines. A wiki page without a killer is documentation of hope, not an operations run.

Limitations and who should skip this

This SOP assumes two humans and a box you are allowed to share; it is not a production control plane. The watchdog only sees wall-clock, a step counter file, and the child process PID you launched. It will not catch silent data leaks or a grandchild process that daemonizes in the background. It also will not replace code review, eval ownership, or a real queue when you outgrow one shared server.

Skip this approach when you are a solo hobbyist on a personal laptop with nothing to hand off. Skip it when the workload is regulated production traffic, customer data, or anything that needs change tickets rather than a wiki paste. Skip it when your agent must fork daemons the wrapper cannot see, because a PID file would give you false confidence. In those cases you want isolation first, not a halt contract on a shared working tree.

Cheap shared inference is useful for draft loops, not for unbounded autonomy on a box other people need. If you already share a free server for draft agent loops, paste the wiki block before the next Friday launch rather than hoping the process feels done.

Name a Replay Owner: A One-Page Wiki SOP When Chat History Is Not a Build

Alex Zhu — Thu, 17 Sep 2026 05:14:52 +0000

You walk into Monday standup and someone claims the overnight helper finally unstuck the flaky checkout test. The pull request looks green, the diff is short, and the description simply says the model figured it out. You ask them to replay the same change on a clean branch, and they paste a screenshot of a vanished chat. That gap is not a tooling failure so much as a missing human role on the team wiki.

Community threads this week keep arguing whether generated code already outruns most developers, or whether calling that work engineering is the real mistake. You do not need to settle that debate before your next merge window. You need a named person who can re-run the work from files in git, without hunting a private transcript. This article gives you that role, a paste-ready wiki page, and a small receipt script you can keep beside the patch.

Chat history is not a build artifact

When a change only lives in a vendor transcript, you cannot hand the work to the next on-call engineer. You also cannot prove the command, the prompt file, and the repository SHA belonged together. Teams then argue about whether the model was brilliant or whether someone quietly edited the output. Engineering still needs a replay path that starts from ordinary files you already store in git.

You have seen the failure mode even when tests are green. A teammate copies a suggested patch, tweaks one import, and later cannot remember which instruction produced which hunk. Another teammate reruns “the same prompt” on a newer checkout and gets a different helper function with the same name. A third person is out sick, and the chat product has already scrolled the session out of the shared browser profile.

Name a Replay Owner for the current week

You do not need another committee, and you do not need a new job title on LinkedIn. You need one named person who refuses to merge AI-assisted work that cannot be re-run from the repo. Call that person the Replay Owner for the current week, and write the name on a one-page wiki SOP. Rotate the role like you rotate on-call, because stale ownership is how chat screenshots become the record.

The Replay Owner is not the author of every patch, and is not a substitute for code review. The owner is the person who checks that a later engineer can reconstruct the run from a receipt file. If that reconstruction fails, the owner blocks the merge and files a short note instead of arguing about model quality in Slack.

What a replayable change actually contains

A replayable change has four pieces sitting next to the patch, not in someone's private browser. First, a frozen command line that does not depend on interactive chat or a hidden desktop session. Second, a prompt or instruction file with a content hash, so silent edits cannot hide after review. Third, the git commit that the run believed it was editing, recorded before anyone restated the problem. Fourth, hashes of stdout, stderr, and any generated files, so a later run can fail closed.

If any of those four pieces is missing, you do not have a replay. You have a story about a replay. Treat the story as a draft until the receipt file exists on the branch.

One-page wiki SOP you can paste

Copy the block below into your team wiki. Replace the names, paths, and runner notes with whatever your group already uses. Keep the page in the same repository as the code whenever you can, so the next owner does not need a second hunt through docs sites.

# Replay Owner — one-page run

**Week of:** YYYY-MM-DD
**Replay Owner (human):** @name
**Backup:** @name
**Handoff window:** weekdays 16:00 local; owner posts in #eng-ai

## Goal
No AI-assisted change merges unless a later engineer can re-run it from git
without the original chat transcript.

## In scope
- Prompt or instruction files committed with the patch
- Exact command recorded in `replay/receipt.json`
- Input git SHA, output hashes, and a pass/fail verify step

## Out of scope
- Model beauty contests and unofficial benchmark screenshots
- Secret material, production credentials, or customer data
- Replacing code review, security review, or release management

## Numbered run (author, then Replay Owner)
1. Author freezes work on a dedicated branch; no more prompt edits in the chat UI.
2. Author saves the instruction text to `replay/instruction.md` and stops editing it.
3. Author records the exact command in `replay/command.sh` (no interactive flags).
4. Author runs `python3 replay/write_receipt.py` on a clean checkout of that SHA.
5. Author opens a PR that includes the patch, instruction, command, and receipt.
6. Replay Owner checks out the PR, runs `python3 replay/verify_receipt.py`, and pastes the result.
7. If verify fails, Replay Owner blocks merge and writes a three-line incident note.
8. If verify passes, Replay Owner comments `REPLAY-OK @name <utc timestamp>` and reviews as usual.
9. Friday 15:00: owner names next week's Replay Owner in this page and in the channel.

## Handoff rule
If the owner is offline, the backup may verify but may not skip the receipt.
If both are offline, merge waits. Chat screenshots are not a substitute.

## Incident note template
- SHA attempted:
- Command:
- Expected hash vs actual hash:
- Decision: reject / new receipt required / escalate to reviewer

That page is the entire operating procedure. You can add links, but you should not add a second philosophy section that nobody will read during a merge freeze.

Artifact: write and verify a receipt from a checkout

The following Python is a template you can adapt; it is not a hosted service and it does not call a model. Put both files under replay/ and keep them boring. Boring receipts are easier to read at 5 p.m. than clever ones.

#!/usr/bin/env python3
"""replay/write_receipt.py — template for a local or shared runner."""
from __future__ import annotations

import hashlib
import json
import os
import subprocess
import sys
from datetime import datetime, timezone
from pathlib import Path

ROOT = Path(__file__).resolve().parents[1]
REPLAY = ROOT / "replay"
INSTRUCTION = REPLAY / "instruction.md"
COMMAND = REPLAY / "command.sh"
RECEIPT = REPLAY / "receipt.json"
LOG_DIR = REPLAY / "logs"


def sha256_file(path: Path) -> str:
    digest = hashlib.sha256()
    with path.open("rb") as handle:
        for chunk in iter(lambda: handle.read(65536), b""):
            digest.update(chunk)
    return digest.hexdigest()


def git_sha() -> str:
    result = subprocess.run(
        ["git", "rev-parse", "HEAD"],
        cwd=ROOT,
        check=True,
        capture_output=True,
        text=True,
    )
    return result.stdout.strip()


def run_command() -> dict:
    LOG_DIR.mkdir(parents=True, exist_ok=True)
    stdout_path = LOG_DIR / "stdout.txt"
    stderr_path = LOG_DIR / "stderr.txt"
    completed = subprocess.run(
        ["bash", str(COMMAND)],
        cwd=ROOT,
        capture_output=True,
        text=True,
    )
    stdout_path.write_text(completed.stdout)
    stderr_path.write_text(completed.stderr)
    return {
        "exit_code": completed.returncode,
        "stdout_sha256": sha256_file(stdout_path),
        "stderr_sha256": sha256_file(stderr_path),
        "stdout_path": str(stdout_path.relative_to(ROOT)),
        "stderr_path": str(stderr_path.relative_to(ROOT)),
    }


def main() -> int:
    for required in (INSTRUCTION, COMMAND):
        if not required.is_file():
            print(f"missing {required}", file=sys.stderr)
            return 2
    if os.getenv("REPLAY_ALLOW_DIRTY") != "1":
        dirty = subprocess.run(
            ["git", "status", "--porcelain"],
            cwd=ROOT,
            check=True,
            capture_output=True,
            text=True,
        )
        if dirty.stdout.strip():
            print("working tree is dirty; refuse to write a receipt", file=sys.stderr)
            return 3
    run_info = run_command()
    receipt = {
        "recorded_at_utc": datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ"),
        "git_sha": git_sha(),
        "instruction_sha256": sha256_file(INSTRUCTION),
        "command_sha256": sha256_file(COMMAND),
        "command_path": "replay/command.sh",
        "instruction_path": "replay/instruction.md",
        **run_info,
    }
    RECEIPT.write_text(json.dumps(receipt, indent=2) + "\n")
    print(RECEIPT)
    return 0 if run_info["exit_code"] == 0 else 4


if __name__ == "__main__":
    raise SystemExit(main())

#!/usr/bin/env python3
"""replay/verify_receipt.py — fail closed if the frozen run cannot be repeated."""
from __future__ import annotations

import json
import subprocess
import sys
from pathlib import Path

ROOT = Path(__file__).resolve().parents[1]
REPLAY = ROOT / "replay"
RECEIPT = REPLAY / "receipt.json"
INSTRUCTION = REPLAY / "instruction.md"
COMMAND = REPLAY / "command.sh"


def sha256_file(path: Path) -> str:
    import hashlib

    digest = hashlib.sha256()
    with path.open("rb") as handle:
        for chunk in iter(lambda: handle.read(65536), b""):
            digest.update(chunk)
    return digest.hexdigest()


def main() -> int:
    if not RECEIPT.is_file():
        print("missing replay/receipt.json", file=sys.stderr)
        return 2
    data = json.loads(RECEIPT.read_text())
    head = subprocess.run(
        ["git", "rev-parse", "HEAD"],
        cwd=ROOT,
        check=True,
        capture_output=True,
        text=True,
    ).stdout.strip()
    checks = [
        ("git_sha", data.get("git_sha"), head),
        ("instruction", data.get("instruction_sha256"), sha256_file(INSTRUCTION)),
        ("command", data.get("command_sha256"), sha256_file(COMMAND)),
    ]
    failed = False
    for label, expected, actual in checks:
        ok = expected == actual
        print(f"{label}: {'OK' if ok else 'MISMATCH'}")
        if not ok:
            print(f"  expected {expected}")
            print(f"  actual   {actual}")
            failed = True
    if failed:
        return 1
    print("receipt files match this checkout; re-run write_receipt.py to compare logs")
    return 0


if __name__ == "__main__":
    raise SystemExit(main())

A minimal replay/command.sh can be as small as a unit test target. Keep it non-interactive, and keep it free of secrets. The Replay Owner should reject a command that curls hidden environment variables or that only works inside one person's laptop session.

#!/usr/bin/env bash
set -euo pipefail
cd "$(dirname "$0")/.."
python3 -m pytest tests/test_checkout_retry.py -q

Numbered handoff you run on Friday

Open the wiki page and confirm the current Replay Owner is still the person on the calendar.
List open AI-assisted pull requests that lack replay/receipt.json and move them back to draft.
Re-run python3 replay/verify_receipt.py on any PR that claims REPLAY-OK from an earlier SHA.
Post a three-line summary: owner name, PRs blocked, PRs verified, and the next owner.
If the next owner declines, the current owner keeps the role; an empty name is not a handoff.

That Friday list is the entire ceremony. You should not add a slide deck, and you should not require a meeting unless two receipts disagree about the same SHA.

Where a shared free runner fits, and where it does not

Some teams already park draft agent runs on a shared box so laptops can close. Disclosure: This article was prepared as part of MonkeyCode's product outreach. If you need a shared place to execute write_receipt.py beside those drafts, MonkeyCode's free model access and free server option is one runner you can point the wiki page at. The SOP still works if you use a spare workstation or a self-hosted job instead; the receipt files are the contract, not the brand of the machine.

Do not put production secrets on a shared free runner. Do not treat a free shared host as an evidence locker for regulated data. The Replay Owner records commands and hashes, then leaves credential handling to whatever boundary process your team already maintains.

A small decision table for the owner

What you see	What you do	What you do not do
No `replay/instruction.md`	Block merge; ask for a frozen file	Accept a chat screenshot
Instruction hash drifted after review	Treat it as new work; demand a new receipt	Rubber-stamp the old `REPLAY-OK`
Git SHA in the receipt is not `HEAD`	Ask for a rebase and a fresh run	Verify against memory of an older branch
Tests pass but stdout hash changed	File the incident note; inspect the log diff	Argue about whether the model improved
Command requires a local GUI session	Reject the command file	SSH into the author's laptop
Owner and backup are both offline	Leave the PR open	Merge because standup sounded confident

Limitations you should write on the wiki page

This receipt format does not prove the generated idea was correct, only that the same files and command were used. Hashing stdout will fail on timestamps, random ports, and unordered logs, so you must keep the recorded command deterministic. The scripts above do not sandbox the command, do not measure model quality, and do not replace a reviewer who understands the domain. A shared runner can disappear, so you still commit the receipt into git instead of trusting disk on that host.

You should also expect social failure modes. Authors will ask you to skip the receipt because the diff is “obvious.” Reviewers will treat REPLAY-OK as a quality score. Neither request is in scope. The Replay Owner only answers whether a later human can re-run the work without the original chat.

Who should not use this SOP

Skip this process if you are a solo hobbyist with no handoff and no shared branch. Skip it if your company already requires experiment tracking with sealed artifacts and a compliance officer. Skip it for emergency production hotfixes that already have a named incident commander and a recorded rollback. Do not use it as cover for merging unreviewed patches, and do not use it to shame teammates who still think in a scratch buffer.

If you cannot name a human owner for the coming week, you do not have a replay practice. You have a Markdown file. Put a real handle on the wiki page first, then run the scripts on one small flaky test before you roll the SOP across every AI-assisted pull request.

Name a Pin Owner: A One-Page Wiki SOP for Shared Free Agent Runtimes

Alex Zhu — Wed, 16 Sep 2026 03:37:04 +0000

Last Friday your intern left a coding agent running overnight on the shared lab machine you all treat as a scratch box. Monday you open the branch and the patch looks plausible, tests are green, and nobody can say which prompt pack produced it. The free runtime has already moved on to another teammate's job, and the previous session log was rotated out. You are not blocked by missing talent so much as by a missing name on the wiki for this run.

Why a pin owner is not a diff signer

A diff signer attests that a patch is acceptable to merge after a human has actually read the change. A pin owner attests something earlier and quieter: which runtime, prompt pack, and input corpus created the candidate. Those jobs collide on small teams, and then Friday's agent output becomes Monday's folklore instead of an engineering artifact. You separate those jobs so a green test suite cannot hide an unrepeatable sandbox session from later review.

Shared free runtimes make the gap worse because the kitchen keeps changing cooks while you are asleep. If your lab already shares free model access and a free server, treat that box as a rented kitchen, not a personal workstation. Disclosure: This article was prepared as part of MonkeyCode's product outreach, and the product notes below stay limited to those two availability claims. MonkeyCode can host that shared lab with free model access and a free server option the whole team can reach.

You still need a named person who pins each run, because a free shared runtime will not pin itself. Write the four jobs on one wiki page so a weekend intern can follow them without a meeting. Keep the names human, because a channel mention is not ownership when the log has already rotated. Rotate the pin owner weekly if the lab is busy, but never leave the line blank during a run.

Four jobs in the handoff

You treat the handoff as a checklist, not a standup topic that dissolves after the call. Fill these four lines before the agent starts, and refuse to export files if any line is still empty. Read them aloud once if the intern is new, because the names are the whole control.

You name a Pin Owner who writes the pin file before any agent output leaves the shared box.
You name a Transcript Owner who archives the session log beside that pin, even when the log is messy.
You name a Contention Owner who grants a time window on the free server and records who occupies it.
You name a Promotion Gate who may move pinned artifacts toward review and must refuse unpinned ones.

These four names can be the same human on a two-person team, and that is still better than nobody. The wiki should show the current names at the top, plus the next rotation date in plain text. If a run spans a rotation, the original pin owner keeps the run until promotion or discard is recorded.

Paste this one-page wiki SOP

Copy the block below into your team wiki and fill the bracketed fields before the next shared agent session. Keep it to one page so people actually read it, and link the pin.json path from the run ticket. Do not replace this page with a slide deck, because decks do not travel with the intern at midnight.

# Shared free agent runtime — pin SOP

## Current roster (update weekly)
- Pin Owner: [name]
- Transcript Owner: [name]
- Contention Owner: [name]
- Promotion Gate: [name]
- Next rotation: [date]

## Purpose
A pin records which prompt pack, input pack, git SHA, and occupancy window produced a sandbox artifact.
Unpinned exports are folklore. They do not enter review.

## Before you occupy the free server
1. Ask the Contention Owner for a window, then write it on the ticket and in CONTENTION_WINDOW.
2. Set PIN_OWNER to your roster name. A handle in chat does not count.
3. Freeze the prompt pack and input pack directories. Do not edit them after the pin is written.
4. Run `python3 pin_run.py [run-id]` and confirm pin.json exists before launching the agent.
5. Start a transcript (`script` or `tee`) and leave it running until the agent stops.

## During the run
- Do not reuse the box for a second job inside someone else's window.
- If the runtime becomes unreachable, stop and record the failure on the ticket. Do not invent a pin after the fact.
- If you change prompts, you start a new run-id. You never amend a live pin.

## After the run
1. Transcript Owner stores `session.transcript.txt` next to `pin.json`.
2. Pin Owner confirms hashes still match the trees on disk.
3. Promotion Gate chooses promote, re-run, or discard using the table in the runbook.
4. Only then may a diff signer review a patch.

## Refuse list
- No pin.json beside artifacts/
- No human name on PIN_OWNER
- Transcript missing or truncated before the final agent message
- Customer data, secrets, or production credentials in the transcript
- Production incident work on the shared free server

A runnable pin file you can check in

Check in a tiny helper so pinning is a command, not a memory test after a long agent loop. The script below is a local proposal you run in your repo; it does not talk to any hosted API. It hashes the prompt pack and input pack, records git HEAD, and refuses to write a pin without a human name. You should still copy the session transcript beside the pin, because hashes cannot reconstruct a lost chat log.

#!/usr/bin/env python3
"""Write pin.json for a shared agent run. Local helper only."""
import hashlib
import json
import os
import subprocess
import sys
from datetime import datetime, timezone
from pathlib import Path


def sha256_tree(root: Path) -> str:
    digest = hashlib.sha256()
    if not root.exists():
        return 'missing'
    files = sorted(path for path in root.rglob('*') if path.is_file())
    for path in files:
        rel = path.relative_to(root).as_posix().encode()
        digest.update(rel)
        digest.update(bytes([0]))
        digest.update(path.read_bytes())
    return digest.hexdigest()


def git_sha() -> str:
    try:
        return subprocess.check_output(['git', 'rev-parse', 'HEAD'], text=True).strip()
    except Exception:
        return 'not-a-git-repo'


def main() -> int:
    run_id = sys.argv[1] if len(sys.argv) > 1 else datetime.now(timezone.utc).strftime('%Y%m%dT%H%M%SZ')
    prompt_dir = Path(os.environ.get('PROMPT_PACK', 'prompts'))
    input_dir = Path(os.environ.get('INPUT_PACK', 'inputs'))
    out = Path(os.environ.get('PIN_PATH', 'pin.json'))
    pin = {
        'run_id': run_id,
        'created_at_utc': datetime.now(timezone.utc).isoformat(),
        'git_sha': git_sha(),
        'prompt_pack_sha256': sha256_tree(prompt_dir),
        'input_pack_sha256': sha256_tree(input_dir),
        'hostname': os.uname().nodename if hasattr(os, 'uname') else 'unknown',
        'pin_owner': os.environ.get('PIN_OWNER', ''),
        'contention_window': os.environ.get('CONTENTION_WINDOW', ''),
        'notes': os.environ.get('PIN_NOTES', ''),
    }
    if not pin['pin_owner']:
        print('Set PIN_OWNER to a real human name before export.', file=sys.stderr)
        return 2
    out.write_text(json.dumps(pin, indent=2) + '\n')
    print('wrote', out)
    return 0


if __name__ == '__main__':
    raise SystemExit(main())

Set the owner and window in the shell, then generate the pin before you launch the agent loop. If the command exits with status 2, you forgot the human name and you must not export artifacts yet. Commit pin.json with the branch, or attach it to the ticket if the branch must stay free of lab noise. Either place is fine so long as the Promotion Gate can find the pin without asking in chat.

export PIN_OWNER='sam.lee'
export CONTENTION_WINDOW='2026-09-16T14:00Z/2026-09-16T16:00Z'
export PROMPT_PACK='./prompts'
export INPUT_PACK='./fixtures'
python3 pin_run.py weekend-lab-47
test -s pin.json || { echo 'pin missing'; exit 1; }

# Proposal: capture a transcript beside the pin.
script -q session.transcript.txt
# ... launch your agent in this shell ...
exit
ls -l pin.json session.transcript.txt

Gate the export with a test

Add a cheap test so CI fails when someone copies an artifacts directory without a pin file sitting beside it. This does not prove the run is correct; it only proves somebody stopped long enough to name the run. Label it as a process test, and keep it out of unit test counts that you use for product quality.

# test_pin_gate.py — process test, not a product quality test
from pathlib import Path


def test_artifacts_require_pin():
    artifacts = Path('artifacts')
    if not artifacts.exists():
        return
    pin = Path('pin.json')
    assert pin.is_file() and pin.stat().st_size > 0, (
        'exporting artifacts without pin.json is forbidden'
    )

How you promote a pinned run

Promotion is a separate hour from generation, even when the patch is small and the author is impatient. Walk the steps in order, and send the run back to the sandbox if any step cannot be answered from the pin. You do not debate taste at this stage; you only decide whether the sandbox output is eligible for review.

You open pin.json and confirm the Pin Owner name matches a real person on the current wiki roster.
You verify prompt and input hashes still match the trees on disk, or you record that the trees have moved.
You skim the transcript for secrets, customer data, and licenses the intern was not allowed to paste.
You replay or re-run only if the pin says the model access and server window are still the same kitchen.
You hand the diff to the signer only after the Promotion Gate writes promote or discard on the ticket.

Use the table when people argue about re-running versus shipping, because argument without a pin is only folklore. The table is a discussion aid, not a compliance control, and you should say that on the wiki page. If two rows apply at once, choose the more conservative action and leave a note for the next pin owner.

Situation	Action	Owner
pin.json missing	Discard the export. Do not review the diff.	Promotion Gate
Hashes match and transcript is present	Allow human review of the diff.	Promotion Gate, then diff signer
Hashes no longer match the trees	Re-run under a new run-id, or discard.	Pin Owner
Contention window expired and the server was reused	Treat as unreproducible and re-run.	Contention Owner
Secrets or customer data in the transcript	Delete artifacts and rotate the exposed secret.	Transcript Owner

Limitations and who should skip this

A pin is a lab note, not a cryptographic proof that a vendor runtime stayed frozen underneath you. Free model access and a free server can change behavior without a notice that your hash will ever see. The helper does not record GPU SKUs, quotas, model names, or uptime, because those claims would be invented here. You will still lose runs when disks rotate, so the transcript copy matters as much as the hash.

Skip this ritual if you work alone on one laptop and you can already replay every command from your shell history. Skip it if your team already runs experiment tracking with dataset hashes, runtime snapshots, and mandatory export gates. Do not use it as evidence in a regulated audit, because a wiki SOP cannot stand in for a controlled environment. Do not use a shared free server for production incident work, customer data, or anything that needs a locked network.

Shared agent labs fail in boring ways: missing names, rotated logs, and patches that nobody can honestly replay. Name a pin owner before the next overnight intern run, and keep the one-page SOP where the intern will actually look. If the wiki line is filled, Monday becomes a review problem instead of a ghost-story problem about a vanished runtime.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Name a Data Boundary Keeper: A One-Page Wiki SOP for What Leaves Your Laptop

Alex Zhu — Mon, 14 Sep 2026 21:29:25 +0000

Name a Data Boundary Keeper: A One-Page Wiki SOP for What Leaves Your Laptop

The failure mode this playbook prevents

The opening story is a composite scenario, not a specific incident I can cite.

Imagine your team gets free remote capacity for an agent session and moves fast to use it. On Thursday a teammate asks the agent to "make the failing auth test pass" and hands over the repository context, which quietly includes .env.staging. The session succeeds, the test passes, and a transcript containing the staging database URL lands in a shared channel three hours later. Nobody was careless in a dramatic way; the workflow simply had no named person who owned the sentence "this may leave the laptop."

That story is a composite, but the shape of it is familiar to anyone who watched remote AI tooling spread through a team faster than its rules did. The fix is not a smarter model or a stricter vendor. It is one named owner, one committed config file, and one command that runs before any context is uploaded.

Why the usual controls do not cover this

Most teams already have secret scanning at commit time and access reviews at quarter end, and both are genuinely useful. Neither answers the question a remote session raises, which is: what bytes are we about to send outside the building?

Local tooling makes that question invisible because nothing leaves your machine. Remote model access and hosted agent servers turn it into a per-task decision, and per-task decisions without an owner drift within a week. That is why a reviewer who checks "did we scan the repo" will keep missing the actual gap.

So treat the boundary as a routing problem rather than a trust problem. You are not deciding whether a provider is trustworthy; you are deciding which classes of data may go there, and who signs each decision.

Step 1 - Classify every task with a routing table

Write the table once, paste it into the wiki, and refuse to relitigate it during incidents. The right-hand column matters more than the middle one, because a rule without a signer is only a suggestion.

Data class	Typical files	Allowed destination	Signer
Public	Open-source code, published docs	Any model, any host	Task Author
Internal, no secrets	Private source, internal docs	Remote model or hosted server, after the check passes	Boundary Keeper
Secrets	`.env*`, keys, service accounts, dumps	Local session only	Boundary Keeper + Restore Reviewer
Regulated	Customer PII, health, payment data	Local only; this SOP is not sufficient	Platform or legal owner

Two rules keep the table honest over time. If a file is not on the allow list, it counts as secrets by default until the Keeper moves it. And if you cannot name the signer at that moment, the answer is local, not "probably fine."

Step 2 - Name four roles and one handoff line

Roles fail when they are fuzzy, so give every role exactly one verb and one artifact.

Task Author - describes the task, lists the minimum files the session needs, and never widens that list mid-run.
Boundary Keeper - owns .agentboundary.toml, runs the pre-flight check, and is the only person who edits the allow list.
Session Runner - starts the remote session with the approved file set only, and stops it when the ticket closes.
Restore Reviewer - reads the audit log weekly and confirms blocked patterns were fixed or promoted deliberately.

The handoff between Author and Keeper is a single line appended to the ticket, and it is the artifact auditors actually ask for later:

route=remote-ok keeper=@alex rule=v3 files=7 bytes=184320 at=2026-09-15T09:12:04Z ticket=ENG-4821

Keep that line out of chat and inside the ticket, so the decision survives the next reorg. A grep-friendly format beats a paragraph of prose describing what everyone agreed to.

Step 3 - Commit a boundary file

Put the policy inside the repository instead of inside someone's memory. This file is intentionally boring: globs, size caps, and secret patterns that reviewers can read in one pass.

# .agentboundary.toml
version = 3

[allow]
globs = ["src/**/*.py", "tests/**/*.py", "docs/**/*.md", "pyproject.toml"]
max_file_bytes = 200000
max_total_bytes = 2000000

[deny]
globs = ["**/.env*", "**/*.pem", "**/*.key", "**/id_rsa*", "**/secrets/**",
         "**/fixtures/prod/**", "**/*.sqlite", "**/*.parquet"]

[secrets]
patterns = ["AKIA[0-9A-Z]{16}", "sk-[A-Za-z0-9]{20,}", "-----BEGIN [A-Z ]*PRIVATE KEY-----"]
entropy_threshold = 4.0
min_token_length = 24

Deny always wins over allow, and the size caps matter because context is not free on any tier. The 200 KB per-file cap is arbitrary but enforceable, which is the point; choose your own number and keep it where a reviewer can see it.

Step 4 - Run the pre-flight check

The script below is a reference implementation you can adapt rather than a tool I ran against a live service while writing. It requires Python 3.11 or newer because it reads TOML with tomllib from the standard library.

#!/usr/bin/env python3
"""boundary_check.py - pre-flight gate before a remote agent session.

Reads .agentboundary.toml, checks a candidate file list, prints a verdict,
and appends one audit line. Exit codes: 0 = remote-ok, 2 = blocked, 3 = usage.
"""
from __future__ import annotations
import fnmatch, math, os, re, sys, tomllib
from collections import Counter
from datetime import datetime, timezone
from pathlib import Path

CONFIG = Path(".agentboundary.toml")
AUDIT = Path(".agentboundary.audit.log")


def shannon_entropy(value: str) -> float:
    counts = Counter(value)
    length = len(value)
    return -sum((n / length) * math.log2(n / length) for n in counts.values())


def matches_any(rel: str, globs: list[str]) -> bool:
    return any(fnmatch.fnmatch(rel, g) for g in globs)


def scan(path: Path, cfg: dict) -> list[str]:
    problems: list[str] = []
    try:
        text = path.read_text(encoding="utf-8")
    except (UnicodeDecodeError, OSError):
        return [f"{path}: not UTF-8 text, treat as local-only"]
    for pattern in cfg["secrets"]["patterns"]:
        if re.search(pattern, text):
            problems.append(f"{path}: matched secret pattern {pattern}")
    min_len = cfg["secrets"]["min_token_length"]
    for token in re.findall(r"[A-Za-z0-9+/=_\-]{%d,}" % min_len, text):
        if shannon_entropy(token) >= cfg["secrets"]["entropy_threshold"]:
            problems.append(f"{path}: high-entropy token, review before sending")
    return problems


def main(argv: list[str]) -> int:
    with CONFIG.open("rb") as fh:
        cfg = tomllib.load(fh)
    raw = argv or (sys.stdin.read().split() if not sys.stdin.isatty() else [])
    if not raw:
        print("usage: boundary_check.py FILE [FILE...]  (or pipe a file list on stdin)")
        return 3

    blocked: list[str] = []
    total = 0
    for name in raw:
        path = Path(name)
        rel = path.as_posix()
        if matches_any(rel, cfg["deny"]["globs"]):
            blocked.append(f"{rel}: denied by glob")
            continue
        if not matches_any(rel, cfg["allow"]["globs"]):
            blocked.append(f"{rel}: not on the allow list, ask the Boundary Keeper")
            continue
        size = path.stat().st_size
        if size > cfg["allow"]["max_file_bytes"]:
            blocked.append(f"{rel}: {size} bytes over the per-file cap")
            continue
        total += size
        blocked.extend(scan(path, cfg))

    if total > cfg["allow"]["max_total_bytes"]:
        blocked.append(f"total payload {total} bytes over cap")

    verdict = "BLOCK" if blocked else "REMOTE_OK"
    stamp = datetime.now(timezone.utc).isoformat(timespec="seconds")
    line = (f"{stamp} verdict={verdict} keeper={os.environ.get('BOUNDARY_KEEPER', 'unset')} "
            f"files={len(raw)} bytes={total}")
    with AUDIT.open("a", encoding="utf-8") as fh:
        fh.write(line + "\n")
    print(line)
    for item in blocked:
        print(f"  - {item}")
    return 2 if blocked else 0


if __name__ == "__main__":
    raise SystemExit(main(sys.argv[1:]))

Wire it into the ticket workflow with two commands, one for a quick audit of the whole tree and one for the exact file set a session will use:

# Audit what the tree currently contains
git ls-files -co --exclude-standard | BOUNDARY_KEEPER=@alex python3 boundary_check.py

# Approve one ticket's minimal payload
BOUNDARY_KEEPER=@alex python3 boundary_check.py src/agent/runner.py tests/test_runner.py
echo "exit=$?"

A blocked run should look boring and specific, which is what makes it actionable:

2026-09-15T09:12:04+00:00 verdict=BLOCK keeper=@alex files=7 bytes=0
  - .env.staging: denied by glob
  - tests/fixtures/urls.txt: high-entropy token, review before sending

Step 5 - Paste the one-page run into the wiki

A numbered run that fits on one screen beats a policy document nobody opens during a deadline push.

Task Author opens the ticket and lists the minimum files the session requires.
Boundary Keeper confirms each path against the routing table and runs the check.
If the verdict is BLOCK, Author trims the payload or swaps secrets for fixtures.
Keeper appends the handoff line and approves with an explicit timestamp.
Session Runner starts the remote session with only those paths available to it.
Runner closes the session at ticket close; no long-lived background loops.
Restore Reviewer reads the weekly audit log and proposes config changes as pull requests.
Anyone may revoke an approval; stop the session first, then ask questions.

Steps five and six are where most leaks happen in practice, because a session left running keeps reading whatever the working tree happens to contain. If your team only remembers one line from this article, remember that one.

Limitations, and who should not use this

Be honest about the ceiling here. Glob rules and entropy checks catch careless mistakes; they do not inspect what happens to data after upload, and they cannot enforce provider-side retention or training policies.

Entropy heuristics also cut both ways. Minified bundles and base64 fixtures trip the threshold, while a short but valid credential can slip underneath it, so expect to tune min_token_length for your repository instead of trusting the default. Treat every BLOCK as a conversation starter, not a verdict from a scanner.

The SOP is a social control, not a sandbox. If you need a hard technical boundary, put an egress proxy or a DLP tool in front of the session and keep this file as the human-readable companion.

Do not use this approach on its own for regulated data, contractual restrictions, or anything your security team classifies as sensitive. If you cannot get a named Boundary Keeper with real authority to say no, writing the role down will not change anything.

Where a free remote session fits into the workflow

MonkeyCode is relevant here because the project offers free model access and a free server option, and the free server option is exactly the case this playbook was written for: the session runs somewhere other than your laptop. The operator states an allowance of 10 million free tokens, so verify the current terms on the project itself before you plan capacity around it.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

A sane first run on any free tier looks like this. Pick one small ticket, commit .agentboundary.toml with a conservative allow list, run the checker, and send only the approved paths into the session. Then compare the audit log against what the session actually touched, and tighten the file before you widen it.

Most teams discover the same thing at that point, which is that the boundary file is the cheap part and the named owner is the hard part. If you want to try the workflow, start with the checker on a single ticket rather than migrating an entire repository into a hosted session.

Name a Tool Budget Owner: A One-Page Wiki SOP for Shared Agent Runs

Alex Zhu — Sun, 13 Sep 2026 19:36:07 +0000

You come back from lunch to a shared lab chat that looks like an incident channel. The agent you started before noon has issued dozens of shell commands against one failing test. A teammate cannot push a hotfix because repeated pytest artifacts have filled the shared workspace disk. Nobody can say who is allowed to kill the process, so the retry loop keeps going.

This is not a model-quality problem in most small teams; it is an ownership problem. Cheap retries feel harmless until the same failing command runs for an hour on a machine other people still need. Without a named Tool Budget Owner, every extra shell call looks reasonable in isolation. You need a one-page wiki SOP before the next shared spike, not another prompt tweak.

What actually burns the shared lab

Most coding agents are a planner wrapped around tools: shell, file edits, test runners, and sometimes git. The planner will keep calling those tools while the transcript still looks like progress. On a laptop you notice the fan. On a shared host you notice a blocked push, a wedged port, or a teammate asking who owns the process.

You should treat tool calls as a budget the same way you treat deploy windows. A budget needs a person, a written cap, and a stop action that does not require a meeting. If you skip those three pieces, the agent becomes an unpaid intern with root-adjacent shell and no timesheet.

Four roles you can paste into the wiki

Keep the roster tiny so a Friday spike still has names attached. Write full names and a backup, not a rotating emoji.

Tool Budget Owner — sets caps, allowlists, and the kill/park/transfer rule for this run.
Runner — starts the agent, records the transcript path, and stays reachable for one working session.
Reviewer — reads tool output that touched git, secrets, or production-shaped config before merge.
Host Steward — owns the shared machine or workspace and can reclaim disk, ports, and processes.

The Tool Budget Owner may be the Runner on a two-person team, but the names still go on the page. When one person holds two hats, write that explicitly so a third teammate does not assume coverage. If the Owner will be offline, transfer the page before the agent starts, not after the first retry storm.

The one-page run (copy this block)

Paste the following into your team wiki and fill the brackets. Do not start the agent until every bracket has a value or an explicit n/a.

# Tool budget run — [date] — [repo]

- Tool Budget Owner: [name] (backup: [name])
- Runner: [name]
- Reviewer: [name]
- Host Steward: [name]
- Workspace / host: [path or hostname]
- Goal (one sentence): [what done looks like]
- Hard stop time: [local time]
- Max tool calls: [n]
- Max consecutive retries of the same command: [n]
- Max files touched: [n]
- Command allowlist file: tool-budget.yaml
- Transcript path: [absolute path]
- On cap hit: kill | park | transfer to [name]
- Secrets rule: no .env, no cloud creds, no production kube contexts

Read the page out loud with the Runner before launch. If you cannot say the stop action in one sentence, the run is not ready. Update the page when you transfer ownership; a stale name is worse than a missing name.

A policy file the Owner can enforce

Proposed example — adapt the numbers to your lab; they are not measured defaults. Keep the file in the repo so the cap travels with the spike.

# tool-budget.yaml
version: 1
run:
  hard_stop_local: "18:00"
  max_tool_calls: 40
  max_same_command_retries: 3
  max_files_touched: 12
allow_commands:
  - pytest
  - ruff
  - python
  - git status
  - git diff
deny_path_globs:
  - ".env"
  - ".env.*"
  - "**/credentials.json"
  - "**/*kubeconfig*"
write_roots:
  - "src/"
  - "tests/"
  - "docs/"
on_cap:
  action: park   # kill | park | transfer
  park_dir: ".agent-park/"

The allowlist is supposed to feel tight. If your agent needs git commit or docker, add those lines in the wiki change, not in a hidden prompt. The Host Steward should reject a run whose policy includes bash -c with no further constraint, because that cap is theater.

A watchdog you run beside the agent

Proposed example — this script does not claim to sandbox the model. It only reads a JSONL transcript your Runner already writes, then prints a stop code the Host Steward can bind to kill.

# tools/check_tool_budget.py
from __future__ import annotations

import json
import sys
from collections import Counter
from pathlib import Path

import yaml

EXIT_OK = 0
EXIT_PARK = 2
EXIT_KILL = 3


def load_policy(path: Path) -> dict:
    return yaml.safe_load(path.read_text())


def load_events(path: Path) -> list[dict]:
    events = []
    for line in path.read_text().splitlines():
        if line.strip():
            events.append(json.loads(line))
    return events


def main() -> int:
    policy = load_policy(Path(sys.argv[1]))
    events = load_events(Path(sys.argv[2]))
    commands = [e.get("command", "") for e in events if e.get("type") == "tool"]
    files = [e.get("path", "") for e in events if e.get("type") == "write"]
    retries = Counter(commands)

    if len(commands) > policy["run"]["max_tool_calls"]:
        print("cap: max_tool_calls")
        return EXIT_PARK
    if retries and retries.most_common(1)[0][1] > policy["run"]["max_same_command_retries"]:
        print("cap: max_same_command_retries")
        return EXIT_PARK
    if len(set(files)) > policy["run"]["max_files_touched"]:
        print("cap: max_files_touched")
        return EXIT_PARK

    denied = tuple(policy["deny_path_globs"])
    for path in files:
        if any(Path(path).match(glob) for glob in denied):
            print(f"kill: denied path {path}")
            return EXIT_KILL
    print("ok")
    return EXIT_OK


if __name__ == "__main__":
    raise SystemExit(main())

Wire it to a loop the Runner already understands. Keep the commands boring so a substitute Host Steward can run them from the wiki.

python3 tools/check_tool_budget.py tool-budget.yaml /tmp/agent-transcript.jsonl
code=$?
if [ "$code" -eq 2 ]; then
  mkdir -p .agent-park
  mv /tmp/agent-transcript.jsonl ".agent-park/$(date +%Y%m%dT%H%M%S).jsonl"
  pkill -f "your-agent-entrypoint" || true
elif [ "$code" -eq 3 ]; then
  pkill -9 -f "your-agent-entrypoint" || true
fi

Label the transcript format in the wiki if your agent SDK uses different keys. The point is a stop code, not a perfect parser. If the agent cannot emit JSONL, have the Runner wrap each tool call with a one-line logger; that wrapper is cheaper than arguing about vendor formats.

Decision table for the stop action

Use this table when the watchdog prints a cap. The Tool Budget Owner picks one cell before the run, not during the storm.

Signal	Default action	You park when	You kill when
Tool call cap hit	park	Goal is still valid after lunch	Host is blocking other work
Same command retried past cap	park	Flaky test, human will rerun	Command mutates git or infra
File-touch cap hit	park	Extra files are docs or fixtures	Writes escaped `write_roots`
Denied path glob	kill	Never park on secrets	Always kill, then rotate if needed
Hard stop time	transfer or kill	Owner going offline	Steward needs the host back

Park means freeze the transcript and leave a note in the wiki. Kill means terminate the process and reclaim disk. Transfer means a named human takes the page before the next tool call.

Where a free model and free server fit

Shared labs often pick a workspace that already offers free model access and a free server option so a spike does not wait on a procurement thread. If that workspace is MonkeyCode, treat those two options as finite shared capacity: they make the first agent run easy, and they make an unbounded retry loop easy as well. Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The SOP does not depend on one vendor. You still name an Owner, write caps, and store transcripts outside the chat window. The free server is just another host the Steward can reclaim. The free model is just another planner that will keep calling tools until you stop it.

Put the wiki page URL in the workspace README so a guest Runner cannot claim they never saw the caps. If you change allowlists mid-run, stop the agent first; a live policy edit is how two teammates enforce different budgets on one process.

A short launch checklist

Walk these steps in order. If you skip a number, you do not have a run; you have a hope.

Create the wiki page from the template and fill every bracket.
Commit tool-budget.yaml on a branch that is not main.
Confirm the Host Steward can pkill the entrypoint without asking in chat.
Start transcript logging before the first tool call, not after the first failure.
Run the watchdog on an interval the Owner can live with, such as every two minutes.
Stop at the first cap, update the wiki with the stop action, then decide whether to continue.

You can keep a dry-run habit: feed a previous transcript into check_tool_budget.py before you trust the caps. That check is a proposed rehearsal, not a published benchmark. If the script parks a run you still liked, lower the ambition of the goal rather than raising the cap in secret.

Limitations

This playbook does not sandbox the model, and it does not replace code review. A determined agent with unrestricted shell can ignore your YAML unless the Steward wraps the process. The watchdog only sees events you log, so a silent side channel will not trip a cap.

It also does not decide whether the patch is correct. Pair it with whatever review gate you already use for AI-authored diffs. Caps prevent a retry storm; they do not prevent a confident wrong edit that fits inside the budget.

Do not treat the sample numbers as capacity planning. Forty tool calls may be huge for a docs-only spike and tiny for a flaky integration suite. Write numbers your Host Steward can defend on Monday.

Who should not use this approach

Skip this SOP if you are the only person on a personal laptop and you can hear the fans. You still might want a retry cap, but you do not need four wiki roles. Skip it if your company already wraps agents in a job scheduler with quotas, audit logs, and an on-call rotation that pages on runaway processes.

Do not use a free shared server for data you cannot paste into a vendor workspace. This article does not claim retention behavior, isolation, or uptime. If the repo contains production secrets, regulated patient data, or customer exports, keep the agent off that host and off those files.

If your teammates will not name an Owner in writing, stop there. A beautiful YAML file with no human attached is how Friday retry storms start. Name the Tool Budget Owner first, then let the agent touch the repo.

Name an Eval Owner: A Wiki SOP When AI Writes the Tests Too

Alex Zhu — Sat, 12 Sep 2026 17:39:41 +0000

You have already seen this pattern on a shared lab board more than once this quarter. A teammate pastes a failing ticket into a coding assistant, accepts a full patch, and watches generated tests turn green. Nobody notices that the same session invented both the implementation and the assertions that bless it. The next on-call person inherits a merge that looks locally green and then fails in production.

This playbook names one human as Eval Owner before any AI-heavy pull request can merge. You keep the assistant, including free model access on a shared host, but you stop treating model-written tests as independent evidence. The artifact below is a one-page wiki SOP, a four-role handoff, and a small Python check you can run without the original chat.

The failure you are actually debugging

Shared free coding loops make this failure cheap to create and later expensive to notice during review. Someone drives a long session on a borrowed machine, dumps a patch plus tests, and leaves before standup. Reviewers skim the diff, trust the green squares, and never ask who authored the oracle that blessed the change. When the oracle and the code share one prompt, you have a tautology rather than a real test.

You do not need a new framework to interrupt that loop on a busy shared board. You need a named owner, a written bar, and a handoff that survives the person who closed the laptop. Write those three things on one wiki page so the next reader can replay the check without hunting through chat logs.

Current discussion about assistants drafting entire diffs makes this independence gap louder rather than smaller for shared labs. Faster patches increase the chance that tests and code are born together inside a single unreviewed thread. Your job is not to ban the assistant; your job is to keep the oracle out of that generating thread.

Four roles you paste into the wiki

Assign these names on the ticket before the first assistant prompt, not after the pull request already exists. If you skip the naming step, the Implementer silently becomes the Eval Owner and the Reviewer inherits a circular suite. Put the four handles where a tired reviewer will see them without opening extra documents or old threads.

Eval Owner. This person writes or selects checks that the coding session is not allowed to edit after freeze. They own fixtures, expected errors, and the replay command that a newcomer must be able to run.
Implementer. This person may use an assistant for code, refactors, and throwaway sketches against the frozen bar. They must not rewrite expected outputs so a convenient patch can turn the suite green.
Reviewer. This person confirms the suite is independent, readable, and actually fails against a known-bad stub. They run the wiki command on a clean checkout, not on the Implementer's already-tuned laptop.
Rollback Contact. This person stays reachable until the change is parked, reverted, or transferred to a named successor. Silence from this role means the pull request stays parked instead of drifting into an unowned merge.

If one person holds two roles, write that exception on the wiki page before work starts. Dual-hat work is allowed for tiny chores, but Eval Owner and Implementer should stay split for money, auth, or retained user data. A missing backup name is a parked ticket, not an invitation for whoever is awake to rewrite fixtures.

Decision table: is the eval independent?

Use this table in the pull request body so reviewers can score independence without rereading the whole assistant thread. If you cannot fill a row honestly, you do not yet have an eval, and the merge bar stays closed. The table is a conversation tool, not a score that you game by renaming generated tests as fixtures.

Question	Pass	Fail	Owner action
Who authored the fixtures?	A human Eval Owner, or a frozen file from before the session	The same chat that wrote the code	Reject and regenerate fixtures offline
Does a known-bad stub fail?	At least one mutation fails the suite	Suite stays green against an empty parser	Add a negative case before review
Can a newcomer replay without the chat?	A wiki command produces the same pass or fail	Replay needs the original thread or a hidden prompt	Record the command and pin the fixture hash
May the Implementer edit evals?	No, except typo fixes the Reviewer accepts	Yes, freely, in the same commit as the feature	Split commits or block merge

Numbered run: one page you can paste today

Copy the steps into your team wiki and fill the names before anyone opens a coding assistant. Keep the page to one screen so tired people will still follow it during a late review. If the page grows past a scroll, you have started writing a process novel instead of a runbook.

Name the Eval Owner on the ticket. Write the person's handle, timezone, and a backup before anyone opens an assistant. Do not start prompting until this line exists on the wiki page for that ticket.
Freeze the oracle before the assistant sees the production code path. Put fixtures, example payloads, and expected errors in a directory the Implementer cannot treat as draft. Share that path in the ticket so review does not hunt through personal scratch files.
Record the replay command in the same page. One shell line should run the independent suite from a clean checkout. If it needs extra env vars, list them beside the command instead of burying them in chat.
Run a negative check that a stub must fail. Point the suite at an empty or mutated parser and confirm it fails loudly. If the stub still passes, your oracle is too weak to serve as a merge bar.
Allow the Implementer to use an assistant after freeze. They may generate code against the frozen bar and iterate on implementation details. They may not rewrite expected outputs to match a convenient patch during that session.
Reviewer executes the wiki command on a clean checkout. Green results from the Implementer's laptop are not evidence for merge. Replay from the recorded command is the only evidence the Reviewer should accept.
Rollback Contact stays named until the next written handoff. If the owner goes offline, park the pull request rather than silently inheriting the bar. Transfer requires a new Eval Owner handle on the same wiki page.

One-page wiki template

Paste this block, then replace the brackets with handles, hashes, and a replay command that actually runs. Do not grow it into a novel, and do not hide the rollback contact below a fold of extra commentary. A short page that people use will beat a perfect page that nobody opens during review.

# Eval bar — [ticket id] — [date]

- Eval Owner: [@name] (backup [@name])
- Implementer: [@name]
- Reviewer: [@name]
- Rollback Contact: [@name] until [date/time]

## Frozen oracle
- Fixture path: `evals/invoice_totals.v1.json`
- Fixture sha256: [hash]
- Replay: `python evals/check_invoice_totals.py --fixtures evals/invoice_totals.v1.json`
- Negative replay: `python evals/check_invoice_totals.py --fixtures evals/invoice_totals.v1.json --mutate empty --expect-fail`

## Session notes
- Assistant host: [shared lab / local / other]
- System prompt frozen? yes/no
- Implementer edited evals? yes/no — if yes, Reviewer sign-off: [@name]

## Handoff
- Park / merge / revert: [choice]
- Next owner if parked: [@name]

A replayable check the chat does not own

The script below is an example you can run locally against a frozen oracle the chat does not own. It treats fixtures as the source of truth and refuses to proceed when the oracle file is missing. Label it as a template if your domain objects, locales, or error types differ from this invoice sketch.

# evals/check_invoice_totals.py
from __future__ import annotations

import argparse
import json
import sys
from pathlib import Path


def load_oracle(path: Path) -> list[dict]:
    if not path.is_file():
        raise SystemExit(f"oracle missing: {path}")
    payload = json.loads(path.read_text(encoding="utf-8"))
    if not isinstance(payload, list) or not payload:
        raise SystemExit("oracle must be a non-empty list")
    return payload


def parse_total(raw: str) -> str:
    """Replace this stub with the implementation under test."""
    digits = [ch for ch in raw if ch.isdigit() or ch in ".-"]
    if not digits:
        raise ValueError("no amount found")
    return "".join(digits)


def parse_total_empty(raw: str) -> str:
    return ""


def main() -> int:
    parser = argparse.ArgumentParser(description="Independent invoice total oracle")
    parser.add_argument("--fixtures", required=True)
    parser.add_argument("--mutate", choices=["none", "empty"], default="none")
    parser.add_argument("--expect-fail", action="store_true")
    args = parser.parse_args()

    fn = parse_total_empty if args.mutate == "empty" else parse_total
    cases = load_oracle(Path(args.fixtures))
    failures = []
    for case in cases:
        sample = case["input"]
        expected = case["expected"]
        try:
            got = fn(sample)
        except Exception as exc:  # example path; tighten in real code
            got = f"error:{exc.__class__.__name__}"
        if got != expected:
            failures.append((sample, expected, got))

    if args.expect_fail:
        if failures:
            print("negative check passed: oracle rejected the stub")
            return 0
        print("negative check failed: stub unexpectedly matched oracle")
        return 1

    if failures:
        for sample, expected, got in failures:
            print(f"oracle mismatch input={sample!r} expected={expected!r} got={got!r}")
        return 1

    print(f"oracle ok cases={len(cases)}")
    return 0


if __name__ == "__main__":
    sys.exit(main())

A tiny fixture file keeps the session honest because the assistant did not author it during the coding loop. Commit that file in a separate change, or at least before the Implementer starts prompting against the ticket. If the model proposes new expected values, the Eval Owner treats that proposal as a change request, not as truth.

[
  {"input": "Total due: USD 1,204.50", "expected": "1204.50"},
  {"input": "amount missing", "expected": "error:ValueError"}
]

Run the positive path and the negative path as two commands so the wiki replay line stays obvious to newcomers. Keep both commands on the wiki page rather than inside a personal alias that nobody else can find.

python evals/check_invoice_totals.py --fixtures evals/invoice_totals.v1.json
python evals/check_invoice_totals.py --fixtures evals/invoice_totals.v1.json --mutate empty --expect-fail

Hash the fixture after the Eval Owner freezes it, then paste the digest into the wiki page next to the path. If the Implementer's branch changes that hash, the Reviewer treats the eval as contaminated and restarts from freeze. Do not accept a verbal claim that the file only changed whitespace; the digest is the handoff artifact.

shasum -a 256 evals/invoice_totals.v1.json

Where a free shared coding host fits

Teams often run these loops on whatever machine is idle, then lose the thread that produced the patch. A shared coding environment with free model access and a free server option can host the Implementer's session without mixing personal billing. The Eval Owner still keeps oracles in version control that the chat cannot quietly edit.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

MonkeyCode is one such option when you want the Implementer on a free model path. Keep fixtures in a repository the assistant does not control, even when the coding session runs on that shared host. If you try the SOP there, paste the wiki page first and keep the four handles visible in the pull request.

Limitations

This SOP does not prove the product is correct for every caller, locale, or production shape you will meet. It only proves the merge bar is independent of the generating session that proposed the patch. Property tests, contract tests, and production monitors still matter, and none of them replace a Reviewer who can read the domain.

Free model access and a free server option can change, pause, or move without notice from any vendor. You must not pin incident response to a particular host, queue, or assistant thread that might vanish overnight. Do not store secrets in the assistant thread, and do not treat generated tests as compliance evidence for auditors. The Python snippet is an example, not a library, and you should adapt types, errors, and locale rules before money paths.

Who should not use this approach

Skip this playbook if you already require independently authored tests and your review checklist already fails tautological suites. Skip it if a single volunteer is pretending to be four roles on a regulated workload with real customers. Skip it if your fixtures themselves come from production data you cannot legally copy onto a shared host.

You should also skip it when the change is a comment-only edit or a docs typo that cannot break behavior. Naming an Eval Owner for those chores trains people to ignore the page the next time a parser changes. Save the ritual for patches where a green suite could hide a shared wrong assumption about inputs.

Close the loop in standup, not in the chat

Ask one question at standup: who owns the eval bar for each open AI-heavy ticket on the board. If the answer is the model or whoever prompted last, you do not have a bar yet. Write the name, freeze the oracle, and only then let the assistant write code against a check it does not own.

DEV Community: Alex Zhu

When Agent Cleanups Erase the Why: A Comment Handoff You Can Paste Into the Wiki

The review gap this card closes

Four roles on one handoff line

Handoff timing you do not skip

Paste this one-page run into the wiki

Steps you follow on the cleanup branch

1. Freeze the comparison range

2. Build the comment inventory

3. Classify before you rewrite

4. Separate style cleanup from behavior risk

5. Sign, publish, and stop

Commands you keep beside the card

An unexecuted scanner you can adapt

Decision table for the keeper

Where a disposable model pass fits

Limitations and who should skip this

What you do before the next cleanup

Assign a Graduation Owner: A Wiki Playbook Before Sandbox Agent Jobs Touch Production

The failure is a missing handoff, not a missing comment

Name the role before you name the server

Roles you should print on one wiki page

Decision table you can paste beside the names

A one-page wiki run you can paste today

Numbered run for the next shared agent job

Commands and checks you can run locally

What this playbook will not do

Close the loop in the repo, not in the demo

Assign a Tool-Allowlist Owner: A Wiki Playbook Before Shared Agents Call Vendor APIs

The failure you are actually preventing

Roles and handoffs you write in public

When the page applies, and when you skip it

One-page wiki run you can paste today

Numbered preflight you run before every shared job

Decision table for grant requests

What you log, and what you never log

Limitations, and who should not use this

Close the loop after the first blocked job

Name a Lane Owner: A One-Page Wiki SOP Before Free-Model Drafts Mix With Ship Work

Why mixed lanes fail quietly

Roles and handoffs

Decide the lane before the first token

One-page wiki SOP (paste this)

What a receipt must contain

A reproducible lane check you can run today

Where a free model path belongs

Limitations, and who should skip this

A one-week rollout that stays small

Name an Oracle Owner: A One-Page Wiki SOP When Green Tests Stop Meaning Pass

Give the role a boring, visible name

What you freeze, and what you allow the agent to touch

Why a shared free runtime makes the job sharper

Numbered run you can execute this afternoon

Artifact: a guard, a pinned recipe, and a hash diff

Decision table for the afternoon handoff

Limitations, and who should skip this

One-page wiki paste

Name a Merge-Readiness Owner: A One-Page Wiki SOP Before Agent Diffs Hit Main

Why merge readiness is a job, not a feeling

Role, backup, and the handoff window

The one-page wiki SOP you can paste

A numbered run for the next agent pull request

Artifact: a merge-readiness contract and a local checker

Where a shared free runtime fits

Limitations, and who should not use this SOP

Name a Stop-Condition Owner: A One-Page Wiki SOP for Runaway Shared Agent Jobs

What a runaway job actually is

Roles you name before anyone launches

Seven steps you can run this afternoon

Artifact: halt contract, watchdog, and kill decision table

Halt contract template

halt.env for the wrapper

Proposed watchdog.py (unexecuted example)

Commands the on-call killer actually runs

Decision table for wait, kill, or refuse restart

One-page wiki block to paste

Limitations and who should skip this

Name a Replay Owner: A One-Page Wiki SOP When Chat History Is Not a Build

Chat history is not a build artifact

Name a Replay Owner for the current week

`halt.env` for the wrapper

Proposed `watchdog.py` (unexecuted example)