DEV Community: Jordan Huang

FAQ: Your Free Server Closed. What Actually Shipped?

Jordan Huang — Sun, 20 Sep 2026 07:31:23 +0000

You closed the tab. Did any patch survive?

I keep hearing the same claims. An agent ran on a free box. The prompt said done. Someone treats that as shipped work. That is a story, not a receipt.

This FAQ is about promotion. How does work leave a scratch session? What still counts after the process dies?

I use first-person here as a reviewer. I am not claiming a production outage. Every command below is a proposed check. Run it. Do not trust my memory.

Why this keeps happening

Agents now live on rented shells. Your laptop is not the only cwd. That is useful. It is also a trap.

A session can look complete and still own nothing durable. Git never saw the files. CI never saw the SHA. Your laptop never fetched the branch.

I sometimes park that loop on MonkeyCode. Disclosure: This article was prepared as part of MonkeyCode's product outreach. It offers free model access and a free server option. That is all I will claim. No model names. No quotas. No hardware story.

The product is not the point. The receipt is the point.

Q1. The remote prompt said success. Is that a merge receipt?

No. A sentence is not git.

Ask a ruder question. What refs exist after the praise?

git rev-parse --abbrev-ref HEAD
git rev-parse HEAD
git status --porcelain=v1
git log -1 --format='%H %D %s'

If HEAD is detached, you do not have a branch. If porcelain is empty, maybe the tree is clean. Maybe you are in the wrong clone.

Corrected model: success text is narration. A merge receipt is a SHA on a named ref that origin also has.

git rev-parse HEAD
git rev-parse @{u} 2>/dev/null || echo 'no upstream'
git cherry -v 2>/dev/null || true

No upstream? Nothing shipped. Stop talking about done.

Q2. Local HEAD matches origin. Am I finished?

Matching SHAs can still hide a dirty tree. People skip porcelain. Then they ship a lie.

Check three planes, not one hash.

Committed tree: git rev-parse HEAD
Index: git diff --cached --stat
Worktree: git diff --stat plus untracked files

echo "HEAD $(git rev-parse HEAD)"
echo "INDEX"
git diff --cached --name-only
echo "WORKTREE"
git diff --name-only
echo "UNTRACKED"
git ls-files --others --exclude-standard

Same SHA, dirty tree? You verified a commit, not the session. The agent may have written files after the commit. Those files die with the box.

Corrected model: a SHA is a photograph. A dirty tree is a room you have not photographed.

Q3. Untracked output on the free box counts as delivered?

Counts for whom? The disk that will vanish?

Agents love /tmp, ./out, and pytest cache folders. They print paths. Humans screenshot paths. Nobody adds the path to git.

Proposed sniff for junk that feels like a deliverable:

find . -maxdepth 3 \
  \( -name '*.xml' -o -name '*.html' -o -name 'coverage*' \
     -o -name 'dist' -o -name 'build' \) \
  -not -path './.git/*'

Label this as a hunt, not a linter. Tune the names. Then decide, file by file:

Commit it, because reviewers need it.
Upload it, because CI already rebuilds it.
Drop it, because it is noise.

Corrected model: if git cannot clone it, you did not deliver it. A free server is not an artifact store.

Q4. I ran it on the free server. Will CI match?

Maybe the tests passed. Which Python? Which locale? Which PATH?

Do not argue. Dump the runner identity into a file you actually keep.

{
  echo "date_utc $(date -u +%Y-%m-%dT%H:%M:%SZ)"
  echo "pwd $(pwd)"
  echo "uid $(id -u) gid $(id -g)"
  echo "uname $(uname -a)"
  echo "shell $SHELL"
  echo "path $PATH"
  command -v python3 && python3 -c 'import sys,locale; print(sys.version); print(locale.getlocale())'
  command -v node && node -v
  git --version
} > session_identity.txt

Commit that file only if your team wants runner forensics. Otherwise keep it next to the PR comment. Compare it to CI logs later.

PATH myths die here. Your laptop uses pyenv. The box uses distro Python. Both print green. They are not the same experiment.

Corrected model: a passing command is bound to an identity dump. No dump, no comparison.

Q5. Closing the session archives the evidence?

Closing a tab is not git push. It is not tar. It is not CI cache.

I treat session end like a power loss. Anything not pushed is gone. That includes:

rebase state in .git
stash entries nobody named
hook output
.env the agent invented
test databases in /tmp

Proposed last-minute grab, still unexecuted until you run it:

# proposed: run before you kill the session
mkdir -p "$HOME/receipts"
OUT="$HOME/receipts/receipt-$(date -u +%Y%m%dT%H%M%SZ).txt"
{
  git remote -v
  git status --porcelain=v1 -b
  git stash list
  git diff --stat
  git log -5 --oneline --decorate
} > "$OUT"
echo "wrote $OUT"

If the free server deletes $HOME on stop, this file dies too. Then the only archive is a push. Push or lose it. That is the whole FAQ.

Corrected model: archive means another machine can fetch it. Chat history is not that machine.

Artifact: a session receipt you can diff

Here is a small proposed script. I have not benchmarked it. It writes a receipt. You copy that receipt off the box. Then you diff two receipts.

#!/usr/bin/env bash
# session_receipt.sh — proposed helper, not a product claim
set -euo pipefail

root=$(git rev-parse --show-toplevel 2>/dev/null || true)
if [[ -z "$root" ]]; then
  echo "not a git worktree" >&2
  exit 2
fi
cd "$root"

receipt=${1:-/tmp/session-receipt.json}

python3 - <<'PY' "$receipt"
import json, os, subprocess, sys, time

def sh(args):
    p = subprocess.run(args, text=True, capture_output=True)
    return p.returncode, (p.stdout or "").rstrip("\n"), (p.stderr or "").rstrip("\n")

def out(args):
    code, stdout, _ = sh(args)
    return stdout if code == 0 else ""

receipt_path = sys.argv[1]
head = out(["git", "rev-parse", "HEAD"])
branch = out(["git", "rev-parse", "--abbrev-ref", "HEAD"])
upstream = out(["git", "rev-parse", "@{u}"])
porcelain = out(["git", "status", "--porcelain=v1"])
untracked = out(["git", "ls-files", "--others", "--exclude-standard"])

payload = {
    "generated_at_unix": int(time.time()),
    "cwd": os.getcwd(),
    "head": head,
    "branch": branch,
    "upstream": upstream or None,
    "dirty": bool(porcelain),
    "porcelain": porcelain.splitlines() if porcelain else [],
    "untracked": untracked.splitlines() if untracked else [],
    "head_subject": out(["git", "log", "-1", "--format=%s"]),
}
with open(receipt_path, "w", encoding="utf-8") as f:
    json.dump(payload, f, indent=2)
    f.write("\n")
print(receipt_path)
PY

Usage I actually want in a PR comment:

chmod +x session_receipt.sh
./session_receipt.sh /tmp/session-receipt.json
# then copy the json off the box before logout

Diff two receipts without drama:

python3 - <<'PY'
import json, sys
a = json.load(open(sys.argv[1]))
b = json.load(open(sys.argv[2]))
keys = ["head", "branch", "upstream", "dirty"]
for k in keys:
    if a.get(k) != b.get(k):
        print(f"MISMATCH {k}: {a.get(k)!r} vs {b.get(k)!r}")
print("untracked_a", a.get("untracked"))
print("untracked_b", b.get("untracked"))
PY
# python3 compare.py laptop.json server.json

If head matches and dirty is false on both, you may talk about shipping. If untracked lists differ, you are still arguing about files that git does not know.

Decision table: what still counts

Claim you heard	Evidence that would support it	If evidence is missing
"The agent finished."	Named branch, clean porcelain, pushed SHA	Treat as a draft on a dying disk
"Tests ran."	CI log for that SHA, not session stdout	Re-run on a runner you control
"Artifact is included."	Path in `git ls-files`	Rebuild or drop the claim
"Same as my laptop."	Two receipts, matching `head` and identity dump	You compared vibes
"We archived the session."	Fetchable URL or git remote	You archived a feeling

Print that table next to the PR. It kills most arguments in one scan.

A workflow I will actually follow

Short loop. No theatre.

Start the agent on the free server if you want cheap iteration.
Force every "done" through session_receipt.sh.
Push a named branch before you close anything.
Copy the receipt to your laptop.
Run the same receipt locally.
Only then paste a SHA into review.

Skip a step and you are demoing a ghost.

Want a brutal extra? Delete the remote working tree in your head the moment you log out. If the branch is not on origin, it never existed.

git fetch origin
git rev-parse origin/your-branch
# fail here? your free server did not ship

Limitations

This receipt is git-shaped. It will not catch data you wrote to a hosted database. It will not catch secrets in shell history. It will not freeze container layers.

It also will not make two Pythons identical. Identity dumps explain mismatch. They do not fix mismatch.

Do not commit receipts that leak host names you cannot share. Redact. Or keep them in the PR, not the repo.

The script uses python3 and git. If either is missing, the receipt is incomplete. That failure is useful. It means the box is not the runner you imagined.

Who should not use this

Skip this workflow if you already have locked-down CI as the only writer. Your agents should not push. Humans merge from CI green only.

Skip it if the free server may touch production credentials. A scratch box is a scratch box. Do not launder secrets through it.

Skip it if you need a legal archive. JSON in /tmp is not a records system.

And skip it if you wanted a model leaderboard. I did not provide one. I will not invent names or scores.

What I want you to repeat instead

Ask one question at logout. Can another machine fetch this SHA?

If yes, you shipped a candidate. If no, you rented a story. Free model access does not change that physics. A free server does not change that physics.

When a teammate pastes a chat ending in "all tests passed," ask for porcelain. Ask for origin. Ask for the receipt file. That is the whole job.

If you run the receipt script, paste the JSON. Do not paste the pep talk.

FAQ: Five Myths About the Agent's Remote Shell

Jordan Huang — Sun, 20 Sep 2026 04:26:49 +0000

The agent said the remote shell was fine. Can you prove that claim with a file?

It pasted a traceback, then a patch, then a shrug. I still needed a receipt from the box itself.

This FAQ kills five claims developers repeat. They sound like caution. They are not.

This is an identity problem, not a prompt problem. Which Python. Which clock. Which tree. Which exit.

What this FAQ is not

This is not a model bake-off. I will not name models.

This is not a quota thread. I will not invent limits.

This is not a speed story. I have no honest benchmark here.

That JSON file is the only bar I trust.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

A free model plus a free server is one convenient place to collect the remote side of this receipt. The checklist still works on any throwaway shell you already own.

Myth 1: Remote Python is "the same Python"

The claim

"It is Python 3. The box is fine." Why does that sentence feel finished to reviewers?

Why it spreads

The prompt said python3. The traceback looked familiar. One import worked.

What the claim skips

python3 is a name, not an identity. Distros ship different ABIs. SoABI can drift. The stdlib path can drift too.

Evidence I actually want

Run this on the remote shell. Run it on your laptop. Then diff the files.

# proposal: interpreter fingerprint, not a vibe
import hashlib, json, platform, sys, sysconfig
from pathlib import Path

exe = Path(sys.executable).resolve()
payload = {
    "executable": str(exe),
    "version": sys.version,
    "platform": platform.platform(),
    "implementation": platform.python_implementation(),
    "soabi": sysconfig.get_config_var("SOABI"),
    "json_module": str(Path(__import__("json").__file__).resolve()),
}
print(json.dumps(payload, indent=2))
print("exe_sha256_16", hashlib.sha256(exe.read_bytes()).hexdigest()[:16])

Did the executable path match? Did SOABI match? Did the stdlib path match?

If any of those drift, your "same traceback" is a coincidence.

Corrected mental model

Treat python3 as an alias. Demand a fingerprint before you compare failures.

Myth 2: `which python3` proves the interpreter that ran

The claim

"which returned /usr/bin/python3. We are good." Are you sure that path was the live process?

Why it spreads

PATH feels like a contract. It is only a search order. Agents mix python, python3, uv run, and venv hooks.

What the claim skips

which cannot see a process that already exited. It cannot see argv either.

Evidence I actually want

# proposal: inspect the live shell, not a memory of PATH
ps -o pid,ppid,user,lstart,cmd -p $$
readlink -f /proc/$$/exe || true
tr '\0' '\n' < /proc/$$/environ | egrep '^(PATH|VIRTUAL_ENV|PYTHON|PWD)=' || true
type python python3
command -v python3
bin=$(command -v python3)
ls -l "$bin"
sha256sum "$bin" || shasum -a 256 "$bin"
file "$bin"

Ask for the exact argv next. Compare it to history or script output. If you cannot, you do not know.

Corrected mental model

PATH is a rumor. Argv plus a hash is evidence.

Myth 3: The prompt said UTC, so the box used UTC

The claim

"We logged timestamps in UTC. The server is UTC." Who actually configured that clock on the box?

Why it spreads

Cloud screenshots look UTC. Dockerfiles set TZ. People assume the next box follows.

What the claim skips

Locale, TZ, and /etc/localtime are three knobs. Agents rarely print all three. A "passed at 16:00" line can be local, UTC, or leftover state.

Evidence I actually want

date -u +%Y-%m-%dT%H:%M:%SZ
date +%Y-%m-%dT%H:%M:%S%z
python3 - <<'PY'
import datetime, os, time
print("TZ", os.environ.get("TZ"))
print("tzname", time.tzname)
print("offset", datetime.datetime.now().astimezone().utcoffset())
print("aware_now", datetime.datetime.now().astimezone().isoformat())
PY
timedatectl 2>/dev/null || echo "timedatectl missing"
ls -l /etc/localtime

Compare laptop versus remote. If offsets differ, your merged log is fiction.

Labeled example, not a measured incident: both sides print 16:04 passed. One clock is PDT. One clock is UTC. That is a two-hour lie with matching strings.

Corrected mental model

Time is configuration. Print it on every run, both sides.

Myth 4: Exit code 0 belongs to the files you edited

The claim

"The remote command returned 0. Ship it." Returned zero against which git tree, exactly?

Why it spreads

Exit codes feel atomic. Agents narrate them with confidence. Chat wraps them in a bow.

What the claim skips

You may not know cwd. You may not know which clone. You may not know if the files match your editor. A green run against a stale tree is a green lie.

This is not a test-quality FAQ. I already wrote that one. This is a tree-identity FAQ.

Evidence I actually want

# proposal: bind the later exit code to a tree fingerprint
git rev-parse HEAD
git status --porcelain
git diff --stat
pwd
python3 - <<'PY'
import hashlib, json, pathlib
root = pathlib.Path(".").resolve()
files = sorted(
    p for p in root.rglob("*")
    if p.is_file()
    and ".git" not in p.parts
    and "node_modules" not in p.parts
    and p.stat().st_size <= 1_000_000
)
h = hashlib.sha256()
for p in files:
    h.update(p.relative_to(root).as_posix().encode() + b"\0")
    h.update(p.read_bytes())
print(json.dumps({
    "cwd": str(root),
    "file_count": len(files),
    "tree_sha256": h.hexdigest(),
}, indent=2))
PY
echo "now run your command"
echo "EXIT:$?"

Keep HEAD, the hash, and the exit in one artifact. If HEAD moved, the exit is about another world. If the tree hash moved, stop quoting it.

Corrected mental model

An exit code has no meaning without a tree hash.

Myth 5: The free box is your audit log

The claim

"The server kept the session. We can replay it." Can you still replay it after the tab closes?

Why it spreads

SSH feels permanent. Cloud shells feel like machines. Chat transcripts feel like journals.

What the claim skips

A free server is a convenience. It is not an evidence locker. I will not claim retention. I will not claim disk size. I will not claim uptime.

If you did not copy the receipt off the box, you do not have it.

Evidence I actually want

Write one JSON file. Leave with that file. Paste it into git if the run matters.

./receipt.sh /tmp/remote-receipt.json
# then copy that file off the box before you close anything

Corrected mental model

Logs you do not export did not happen. Chat is not storage.

The artifact: `receipt.sh`

This is the whole method. Remote once. Local once. Diff the JSON.

Treat the script as a proposal. I am not attaching a public dataset. I am not inventing a pass rate.

#!/usr/bin/env bash
# receipt.sh — proposal, not a product claim
set -euo pipefail
out="${1:-./shell-receipt.json}"

python3 - "$out" <<'PY'
import hashlib, json, os, platform, socket, subprocess, sys, sysconfig, time
from datetime import datetime, timezone
from pathlib import Path

def sh(cmd):
    try:
        p = subprocess.run(
            cmd, shell=True, capture_output=True, text=True, timeout=20
        )
        return {
            "cmd": cmd,
            "code": p.returncode,
            "stdout": p.stdout.strip(),
            "stderr": p.stderr.strip()[:400],
        }
    except Exception as e:
        return {"cmd": cmd, "error": str(e)}

root = Path.cwd().resolve()
exe = Path(sys.executable).resolve()
tree = hashlib.sha256()
count = 0
for p in sorted(root.rglob("*")):
    if not p.is_file():
        continue
    if ".git" in p.parts or "node_modules" in p.parts:
        continue
    if p.stat().st_size > 1_000_000:
        continue
    tree.update(p.relative_to(root).as_posix().encode() + b"\0")
    tree.update(p.read_bytes())
    count += 1

receipt = {
    "schema": "shell-receipt/v1",
    "utc": datetime.now(timezone.utc).isoformat(),
    "host": socket.gethostname(),
    "platform": platform.platform(),
    "uname": sh("uname -a"),
    "cwd": str(root),
    "uid": getattr(os, "getuid", lambda: None)(),
    "tz": os.environ.get("TZ"),
    "tzname": time.tzname,
    "python": {
        "executable": str(exe),
        "version": sys.version,
        "implementation": platform.python_implementation(),
        "soabi": sysconfig.get_config_var("SOABI"),
    },
    "path_python3": sh("command -v python3"),
    "git": {
        "head": sh("git rev-parse HEAD"),
        "status": sh("git status --porcelain"),
        "remote": sh("git remote -v"),
    },
    "tree": {"file_count": count, "sha256": tree.hexdigest()},
}

Path(sys.argv[1]).write_text(json.dumps(receipt, indent=2) + "\n")
print("wrote", sys.argv[1])
PY

How I compare the two files

chmod +x receipt.sh
./receipt.sh /tmp/remote.json
./receipt.sh /tmp/local.json
python3 - <<'PY'
import json
from pathlib import Path

a = json.loads(Path("/tmp/remote.json").read_text())
b = json.loads(Path("/tmp/local.json").read_text())
for k in ("platform", "python", "cwd", "tree", "tzname"):
    print(f"{k:10} match={a.get(k)==b.get(k)}")
if a["git"]["head"] != b["git"]["head"]:
    print("HEAD mismatch: stop sharing exit codes")
if a["tree"]["sha256"] != b["tree"]["sha256"]:
    print("tree mismatch: the remote exit is another world")
PY

Decision table

Check	If it matches	If it does not
Python fingerprint	Tracebacks are comparable	Do not paste remote stacks onto local code
`git` HEAD	You may discuss the exit	Fetch or re-clone first
Tree `sha256`	The command saw the same files	The green bar is another tree
TZ offset	You may merge logs	Split the timelines
JSON copied off-box	You have a receipt	You have a story

Print that table in the PR. Or refuse the PR. Those are the two options I respect.

Limitations

The script skips files over 1MB. That is a bias.

It skips node_modules on purpose. That is another bias.

It does not freeze the network. A remote install can still race.

It does not prove the agent used this interpreter. It proves the interpreter you invoked while collecting the receipt.

timedatectl may be missing. That absence is data too.

/proc may be missing. Then record that gap in the JSON.

A free server can disappear. Export that file before the session dies.

Do not dump full environ into git. Secrets hide in PATH-adjacent variables.

This habit does not make a model "safe." It makes your claims checkable.

Who should not use this approach

Do not use a shared free server for customer data.

Do not put private keys on that box.

Do not treat it as CI if you need retention, IAM, or an artifact store.

Do not use it when policy forbids unknown hosts.

If you need those controls, use your real CI. Pay for isolation in your real pipeline instead. This FAQ is not that design.

The only question I ask now

Where is the JSON for that remote claim?

No JSON means I reject the remote claim. Run receipt.sh on the throwaway box. Run it on the laptop. Diff the files.

The agent can still be wrong. At least the box cannot hide behind a shrug.

FAQ: A Free Session Is Not a Build Receipt

Jordan Huang — Sun, 20 Sep 2026 01:19:08 +0000

Did that patch leave a host you do not own?
I keep hearing five myths around complimentary coding boxes.

This FAQ is a receipt checklist, not a vibe check.
Would you merge a binary with no hash, ever?

What broke in my head

Agents now jump onto a remote box without asking.
The chat looks finished while the compile looks green.
Then the real CI job explodes on a different host.

Does that sequence feel uncomfortably familiar to you?
I wanted one file I could commit beside the patch.

The mental model I want

Treat every complimentary host as an untrusted compiler.
Treat every free model turn as unlabeled text until tagged.

The remote box is not your shipping laptop.
The agent session is not your release pipeline.

Myth 1: The complimentary host matches my laptop

People repeat this claim like it is weather.
They say it built there, so it builds here.

Can you show uname, python3 --version, and which cc?
Those three strings decide whether the claim survives.

Linux userland is not the same as macOS userland.
A container glibc is not your workstation glibc either.

Compare fingerprints before you copy any built artifact.
Without a fingerprint, you should refuse the copy.

uname -srm
python3 --version
command -v cc || true
command -v rustc || true
echo "PATH=$PATH"

Run that command block on the complimentary host.
Run the same block on the machine that ships.

Different strings mean you actually built different products.
Do not argue with those platform strings later.

Myth 2: Free model output needs no author tag

I hear this claim in almost every review.
People say the model is free, so skip tags.

Does authorship vanish because nobody sent an invoice?
A patch still entered your tree from somewhere unknown.

Log the turn, the prompt hash, and the diff hash.
You are logging provenance, not a marketing brand name.

mkdir -p .session
date -u +"%Y-%m-%dT%H:%M:%SZ" > .session/started_at_utc.txt
printf '%s\n' "${MODEL_LABEL:-unknown-free-model}" > .session/model_label.txt
git diff --stat > .session/diff_stat.txt
git rev-parse HEAD > .session/base_commit.txt

Write unknown-free-model when the label is missing.
An unknown label is honest, while a missing file is not.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.
This walkthrough uses MonkeyCode as the example complimentary host.

Free model access and a free server option are available there.
I still refuse to treat that session as CI.

Myth 3: A compiler on the box makes artifacts portable

This myth still burns teams during release week.
Someone copies a binary off the box and ships.

Would you ship a binary you cannot rebuild today?
Then you should not ship this copied binary either.

The remote compile is only a smoke signal.
Rebuild under your pinned toolchain, or do not ship.

mkdir -p .session
find dist -type f -print0 2>/dev/null | sort -z | xargs -0 sha256sum > .session/dist.sha256
# rebuild on the ship host, then diff the two hash files

Divergent hashes taught you something while the lesson was cheap.
A missing hash taught you nothing about portability.

Myth 4: No invoice means no paper trail

Compliance folks hate this myth for good reason.
Engineers repeat it because the dashboard said free.

Free compute still executed secrets if you pasted them.
Free compute still wrote files you might later commit.

Demand a bill of materials before the victory lap.
A session without that file is only a rumor.

I want one JSON file sitting beside the patch.
The next human should re-check facts, not feelings.

Myth 5: Copying files back equals a pipeline

The agent says done and dumps a tarball on disk.
You extract it over src and open the merge request.

Is a file copy with extra confidence a pipeline?
I do not treat that copy as a release builder.

Remote work stays a draft until your pipeline rebuilds it.
Your pipeline is the only builder you should trust.

git status --porcelain
git diff --name-only
# stop if unexpected lockfiles or binaries appear

Unexpected lockfiles are a smell worth stopping for.
Unexpected binaries are a hard stop, not a debate.

PATH lies more than people think

The complimentary host PATH often includes mystery toolchains.
Your laptop PATH includes another set of mystery toolchains.

Never trust a binary name without command -v plus a hash.
A pretty interpreter name is not a receipt by itself.

command -v python3
python3 -c "import sys; print(sys.executable)"
sha256sum "$(command -v python3)"

That hash is the interpreter you actually invoked.
A pretty python3 name on PATH is not evidence.

Artifact: a session bill of materials

Here is a small Python script you can copy.
Treat it as a proposal and run it yourself.

Save the file as session_bom.py in the repo root.
Run it on the complimentary host before copying files.

#!/usr/bin/env python3
"""Write .session/bom.json for a complimentary coding host."""
from __future__ import annotations

import hashlib
import json
import os
import platform
import subprocess
import sys
from datetime import datetime, timezone
from pathlib import Path

ROOT = Path.cwd()
OUT = ROOT / ".session" / "bom.json"


def sh(args: list[str]) -> str:
    try:
        p = subprocess.run(args, check=False, capture_output=True, text=True)
    except FileNotFoundError:
        return ""
    return (p.stdout or "").strip()


def sha256_file(path: Path) -> str:
    h = hashlib.sha256()
    with path.open("rb") as f:
        for chunk in iter(lambda: f.read(65536), b""):
            h.update(chunk)
    return h.hexdigest()


def pip_freeze_hash() -> str | None:
    text = sh([sys.executable, "-m", "pip", "freeze"])
    if not text:
        return None
    return hashlib.sha256(text.encode()).hexdigest()


def git_info() -> dict:
    return {
        "head": sh(["git", "rev-parse", "HEAD"]),
        "status": sh(["git", "status", "--porcelain"]),
        "user_email_set": bool(sh(["git", "config", "user.email"])),
    }


def main() -> int:
    tracked = []
    for rel in ("src", "app", "lib"):
        d = ROOT / rel
        if not d.is_dir():
            continue
        for p in sorted(d.rglob("*")):
            if p.is_file() and p.stat().st_size <= 1_000_000:
                tracked.append(
                    {
                        "path": str(p.relative_to(ROOT)),
                        "sha256": sha256_file(p),
                    }
                )

    bom = {
        "generated_at_utc": datetime.now(timezone.utc).isoformat(),
        "hostname": platform.node(),
        "platform": platform.platform(),
        "python": sys.version.split()[0],
        "executable": sys.executable,
        "cwd": str(ROOT),
        "path_head": os.environ.get("PATH", "")[:400],
        "pip_freeze_sha256": pip_freeze_hash(),
        "git": git_info(),
        "model_label": os.environ.get("MODEL_LABEL", "unknown-free-model"),
        "tracked_files": tracked[:200],
    }
    OUT.parent.mkdir(parents=True, exist_ok=True)
    OUT.write_text(json.dumps(bom, indent=2) + "\n", encoding="utf-8")
    print(f"wrote {OUT}")
    return 0


if __name__ == "__main__":
    raise SystemExit(main())

Export a model label even when the name is unknown.
Then copy .session/bom.json along with the patch.

export MODEL_LABEL="unknown-free-model"
python3 session_bom.py

The merge request now carries a receipt, not a story.
Reviewers can argue with files instead of chat memory.

A test that refuses a naked patch

This pytest file fails closed on purpose.
A missing BOM means the patch should not pass.

# test_session_bom.py — proposal, not a production suite
from pathlib import Path
import json


def test_session_bom_exists_and_names_a_host():
    p = Path(".session/bom.json")
    assert p.is_file(), "missing .session/bom.json; refuse the patch"
    data = json.loads(p.read_text(encoding="utf-8"))
    assert data.get("hostname"), "BOM hostname is empty"
    assert data.get("python"), "BOM python is empty"
    assert data.get("git", {}).get("head"), "BOM has no git head"
    assert data.get("model_label"), "BOM has no model label"


def test_session_bom_is_not_your_laptop_claim():
    data = json.loads(Path(".session/bom.json").read_text(encoding="utf-8"))
    # Fill this from YOUR ship host, not from memory.
    ship_platform_needle = "REPLACE_ME"
    assert ship_platform_needle != "REPLACE_ME", "set the ship host needle"
    print("remote_platform=", data.get("platform"))

Replace REPLACE_ME with a substring from your ship host.
A green test must prove the platforms match, or rebuild.

Diff the BOMs, then approve

I also diff two BOM files before I approve.
Hostname, platform, python, and pip freeze hash must be visible.

# diff_bom.py — proposal: print mismatched BOM keys
import json
import sys

a = json.load(open(sys.argv[1], encoding="utf-8"))
b = json.load(open(sys.argv[2], encoding="utf-8"))
keys = ("hostname", "platform", "python", "pip_freeze_sha256", "model_label")
for k in keys:
    if a.get(k) != b.get(k):
        print(f"MISMATCH {k}: {a.get(k)!r} vs {b.get(k)!r}")

python3 diff_bom.py .session/bom.json ship_host_bom.json

Any MISMATCH line means rebuild on the ship host.
Do not negotiate with a mismatched interpreter hash.

Decision table I actually use

I keep this table next to the merge request template.
Ask the author which row they claim to occupy.

Claim you hear	Evidence to demand	If missing, do this
It built on the free box	`platform` plus compiler path in the BOM	Rebuild on the ship host
The model wrote a small patch	`model_label` plus `git diff --stat`	Mark author unknown and review harder
Just copy dist/	`dist.sha256` versus a local rebuild	Throw the copy away
Nothing sensitive happened	a command history you actually control	Rotate anything you pasted
CI will catch it	CI uses the same pins you named	Diff the pins; do not guess

Each row is a claim, evidence, and a refusal action.
If evidence is missing, take the refusal action immediately.

What this does not prove

A BOM is not a security audit of the host.
A BOM is not a license scan of the tree.

It will not catch a model that lied in comments.
It will not freeze a moving complimentary image either.

A complimentary image can change under you without notice.
I do not claim uptime, hardware, quotas, or model names.

If you need those numbers, read product docs that day.
Do not copy capacity claims from a blog post later.

Who should not use this

Skip this if you already have a locked remote builder.
Skip this if policy forbids any complimentary host usage.

Skip this if you cannot write .session into the repo.
Skip this if your threat model needs attested hardware.

This workflow is for messy agent sessions on scratch boxes.
It is not a substitute for a real compliance program.

The questions I ask before merge

I ask five questions before I approve the merge.
Each one needs a file, not a shrug.

Which host produced these bytes, according to the BOM?
Which model label sits inside that same JSON file?
Did I rebuild the tree under the real ship pins?
Did unexpected binaries land inside the git diff?
What secret might have hit that complimentary box?

If any answer is a shrug, I block the merge.
A shrug from a free session is not a receipt.

Would you accept a shrug from a human intern today?
Then do not accept that shrug from a complimentary session.

Closing

Complimentary compute remains useful for drafts and spikes.
Unlabeled compute is how rumors sneak into main.

Write the BOM before you copy a single artifact.
Hash the outputs, then rebuild where you actually ship.

Which of these five myths are you still repeating tomorrow?

FAQ: Five Myths About Promoting Code Off a Free Agent Box

Jordan Huang — Sat, 19 Sep 2026 22:11:12 +0000

Did the free box just become your origin?

I keep catching that assumption in review. A remote agent workspace is a draft. It is not a release channel.

This FAQ is about promotion, not prompting. You drafted on a cheap remote box. Fine. Then someone treats that box like CI. That is the bug.

What this FAQ is not

I already wrote about empty scratch hosts. I already wrote about chat logs versus git status. This one is narrower.

How do you move bytes from a free agent box into a real repo without lying?

Where a free box actually helps

I still use a throwaway remote shell for first drafts.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

MonkeyCode offers free model access and a free server option. I treat both as a scratchpad with a network. I do not treat them as production. I do not treat them as CI.

The value is cheap iteration. The risk is a fake handoff. The rest of this post is the handoff.

The five myths

Each myth has a claim, a check, and a better mental model. None of these need a vendor benchmark. They need hashes, lockfiles, and a local command.

Myth 1: The transcript file is the whole file

Claim: "I copied the function from chat. Ship it."

Did the model truncate the tail? Did the UI fold imports? Chat is a viewport. It is not a filesystem.

Check I actually run:

# Label: proposed local check, not a production metric.
sha256sum src/billing/quote.py
wc -l src/billing/quote.py

On the remote box, write the same two numbers into a manifest. If either number drifts, you copied a window. You did not copy a file.

Corrected model: Promote paths, not bubbles. Hash the path. Then copy. Then hash again.

Myth 2: The free box runtime is your runtime

Claim: "It imported on the agent server. We are good."

Which Python? Which glibc? Which Node? A free server is one image. Your laptop is another. CI is a third.

I ask three questions before I trust a remote green run:

What does python -V or node -v print there?
What does it print here?
What does CI pin in the workflow file?

# Label: collect facts. Do not invent a match.
python -V; which python
node -v 2>/dev/null || true
cat .python-version 2>/dev/null || true
cat .nvmrc 2>/dev/null || true

Corrected model: A remote interpreter is a guest. Guest success is a clue. It is not a contract.

Myth 3: A remote install gave you a trustworthy lockfile

Claim: "The agent ran install. The lockfile is current."

Did it? Or did it mutate package.json and skip the lock? Or rewrite poetry.lock with a different resolver?

Green install on a throwaway box is not a pin. I already beat that drum for local installs. The remote case is worse. You cannot see the resolver logs unless you kept them.

Check:

git diff --stat -- lockfile.txt poetry.lock \
  package-lock.json pnpm-lock.yaml Cargo.lock

If the lockfile is missing, stop. If it changed without a reason, stop. If the agent "helpfully" regenerated it, treat that as a new change. Review it like code.

Corrected model: The lockfile is the artifact. The install log is commentary.

Myth 4: Remote green means skip the local smoke

Claim: "Tests already passed on the box. Why rerun?"

Wrong machine. Wrong caches. Wrong secrets. Wrong working directory.

This is not the old "did the agent really run tests" question. Assume they ran. They still ran over there.

I want one boring command on my checkout. Not a full suite. A smoke that proves the files I copied are the files I think I copied.

# Label: example smoke only. Swap in your real target.
python -m compileall src/billing/quote.py
pytest -q tests/test_quote.py -k parse --maxfail=1

If that fails locally, the remote green build was a tourist photo. Nice lighting. Wrong city.

Corrected model: Remote tests answer "could this work." Local smoke answers "did this land."

Myth 5: A workspace link is a pull request

Claim: "I will just share the free server URL."

With whom? For how long? With which secrets still in the env?

A workspace link is not review. It has no diff. It has no CI. It has no CODEOWNERS. It often has your paste buffer.

Check before you paste a URL:

Is .env in the tree?
Did the agent echo tokens in a log?
Can a stranger fetch the workspace?
Do you even own the retention story?

I do not know your provider's retention. I refuse to guess. If you cannot answer retention, do not share the box. Open a PR from your repo instead.

Corrected model: Git is the handoff protocol. HTTP to a scratch VM is a demo.

Artifact: a promotion manifest

I use a tiny two-file protocol. No special platform. No claimed speedup.

File one lives on the remote box: promo.manifest. File two is this local checker. The checker is a proposal. Run it on a throwaway branch first.

Remote side, after the agent stops talking:

# Label: proposed remote snapshot.
# Run inside the project root on the free box.
{
  echo "# promo.manifest"
  echo "pwd $(pwd)"
  echo "head $(git rev-parse --short HEAD 2>/dev/null || echo nogit)"
  echo "python $(python -V 2>&1)"
  echo "node $(node -v 2>/dev/null || echo none)"
  git ls-files -o -m --exclude-standard | sort | while read -r f; do
    [ -f "$f" ] || continue
    printf '%s  %s\n' "$(sha256sum "$f" | awk '{print $1}')" "$f"
  done
} > promo.manifest
cat promo.manifest

Copy promo.manifest home with the files. Not through chat if the tree is large. Use scp, a bundle, or a patch.

# Label: proposed local verifier. Save as scripts/check_promo.sh
set -euo pipefail
manifest="${1:?usage: check_promo.sh promo.manifest}"
fail=0

while read -r hash path; do
  case "$hash" in
    \#*|pwd|head|python|node) continue ;;
  esac
  if [ ! -f "$path" ]; then
    echo "MISSING $path"
    fail=1
    continue
  fi
  got=$(sha256sum "$path" | awk '{print $1}')
  if [ "$got" != "$hash" ]; then
    echo "DRIFT $path"
    echo "  remote $hash"
    echo "  local  $got"
    fail=1
  fi
done < "$manifest"

if [ ! -f package-lock.json ] && [ ! -f pnpm-lock.yaml ] \
   && [ ! -f poetry.lock ] && [ ! -f Cargo.lock ]; then
  echo "WARN no lockfile in this snapshot"
fi

exit "$fail"

Then the smoke you already trust:

bash scripts/check_promo.sh ./promo.manifest
git status --short
# only then: your local smoke command

Decision table I keep next to the script:

Hash match, lockfile unchanged, smoke green → open the PR from this repo.
Hash match, lockfile changed → review the lockfile as its own diff.
Hash drift → stop copying from chat. Recopy the path.
Smoke red → the remote green build does not travel. Fix locally.
No manifest → you do not have a promotion. You have a story.

Limitations

This protocol does not prove correctness. It proves the bytes moved. It does not pin a model. It does not pin a server image. It does not replace CI.

Hashes ignore file modes on some copies. Newlines can bite you. Windows versus Unix will lie. If you copy through Slack, you already lost.

Who should not use this?

Anyone shipping secrets that already touched the free box.
Anyone who needs a long-lived remote environment. I did not claim one.
Anyone without a real git remote they control.
Regulated work that forbids unknown hosted shells.

If MonkeyCode's free server is your draft box, export the manifest before you copy anything home. That is the whole ask.

The mental model I want stuck in your head

Ask four questions, in this order:

Which machine ran the command?
Which bytes am I about to commit?
Which lockfile travels with those bytes?
Which system will rerun the check besides me?

If you cannot answer all four, you are not promoting. You are hoping.

A free model can draft. A free server can compile. Your repo still has to receive the patch. CI still has to hate it in public.

That split is the job. Keep it.

FAQ: What a Free Agent Session Does Not Prove

Jordan Huang — Sat, 19 Sep 2026 18:59:18 +0000

Have you ever copied a patch from a free agent box?
Then watched local CI reject the same tree?

I keep hearing five claims in review threads.
They sound harmless until a merge fails locally.

This FAQ treats those claims as testable myths.
A free session can draft work at useful speed.
It still cannot freeze your actual toolchain.

What I mean by a free session

Some agents give free model access for drafting patches.
Some also give a free server to run shell commands.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

I use MonkeyCode's free models and free server as scratch.
They help me try commands I might later keep.
They do not pin a compiler, image, or locale.

The rest of this piece stays useful without that product.
The receipt workflow is just git, a shell, and honesty.

How to read each answer

Each myth has a claim, a check, and a better model.
Skip anything you cannot verify on your own machine.

Ask one rude question after every remote green build.
What evidence will still exist when that box is gone?

FAQ 1: Remote green means the PR is merge-ready?

Claim: The free server built it, so the PR is done.

Did the remote compiler match your CI image?
Did it use the same libc, locale, and umask?
You probably do not know, and that is the problem.

I run this after every copy, before any review ping:

git rev-parse HEAD
git status --porcelain
git diff --stat

Then I rerun the same command CI will run.
A remote green build is a hint, not a gate.

Better model: Treat remote green as an untrusted preview.
Merge only after the local runner or CI repeats it.

Quick list I keep next to the laptop:

The git HEAD matches the PR branch exactly.
git status --porcelain stays empty after the copy.
One CI command runs locally and exits zero.

FAQ 2: It was a free draft, so skip the model note?

Claim: It was only a free model. Logging is ceremony.

Which turn produced the patch you are about to commit?
If you cannot point to it, you cannot replay advice.
Free does not mean interchangeable across days or teammates.
It means the draft was unpaid, not that it was pinned.

I do not invent model names I cannot see in logs.
I still write a one-line note in the receipt file.

session_note: drafted on a free model; not a version pin

Why bother, if the name is not a hash?
Because future you will ask where the patch was born.
A note saying "free draft" beats a silent blob every time.

Better model: Record that the source was a free draft.
Never pretend the model string is a compiler hash.

FAQ 3: Copying files off the box equals a commit?

Claim: The files sit on the server, so they are saved.

Saved where, on whose disk, and under which umask?
A server path is not a branch you can push.
It is a rental workspace with a short memory.

I copy through a reviewed diff, never through hope.

# on your laptop, after scp or a download
git add -N .
git diff
# only then
git add -p

If you cannot git add -p it, you do not own it yet.

Better model: Treat the free filesystem as a clipboard only.
A commit is the first moment the work becomes yours.

Questions I ask before I even open scp:

Which paths changed, and which paths must stay untouched?
Did the agent rewrite lockfiles I never asked it to touch?
Can I explain every hunk in one sitting?

FAQ 4: Locale and timezone never leak into artifacts?

Claim: The box is generic. Timestamps and sorts will match.

Have you seen tests fail on filename sort order?
Have you seen tarballs change because tar saw local time?
Have you seen Python tests break on %c locale formats?

I capture the personality of the box before I leave it.

date
date -u
locale
umask
id
python3 -c 'import locale,time,sys; print(sys.platform); print(locale.getlocale()); print(time.tzname)'

Write those values into receipt.json and compare them at home.

Better model: The free server has a quiet personality of its own.
Capture it, or you will debug ghosts after the copy.

Leaks worth hunting in copied trees:

Sorted test output can depend on LC_COLLATE.
Archive metadata can be stamped with the server timezone.
Generated headers can embed local strftime text.
File modes may only make sense under that umask.

FAQ 5: A teammate can resume from your memory?

Claim: We share the same free product, so we share state.

Do you share the same cwd and the same dirty tree?
Do you share env names and the same uncommitted patch stack?
Product sameness is not session sameness, not even close.

Better model: Hand off a receipt, not a remembered vibe.

Minimum handoff packet I will accept from a teammate:

Keep the receipt.json captured on the remote box.
Keep receipt.json from the laptop after copy.
Write the exact command that must pass locally.
Attach a git diff you can explain line by line.

No packet means no resume, because memory is not a worktree.

The artifact: two receipts and a diff

I want a file I can attach to the PR.
It records what the free session refused to freeze.

Save the script below as scripts/session-receipt.sh.
Treat it as a labeled example, not a production service.

#!/usr/bin/env bash
# session-receipt.sh — labeled example, run from a git worktree
set -euo pipefail

OUT="${1:-receipt.json}"
CMD_LOCAL="${MUST_PASS_LOCALLY:-python -m pytest -q}"

if ! git rev-parse --is-inside-work-tree >/dev/null 2>&1; then
  echo "run this inside a git worktree" >&2
  exit 1
fi

python3 - "$OUT" "$CMD_LOCAL" <<'PY'
import json, os, subprocess, sys, time, locale

out, cmd = sys.argv[1], sys.argv[2]

def sh(args):
    p = subprocess.run(args, capture_output=True, text=True)
    return p.returncode, p.stdout.strip(), p.stderr.strip()

_, head, _ = sh(["git", "rev-parse", "HEAD"])
_, status, _ = sh(["git", "status", "--porcelain"])
_, branch, _ = sh(["git", "branch", "--show-current"])
_, umask, _ = sh(["bash", "-lc", "umask"])
_, who, _ = sh(["id"])

receipt = {
    "captured_at_utc": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime()),
    "git_head": head,
    "git_branch": branch,
    "git_status_porcelain": status.splitlines(),
    "dirty": bool(status),
    "umask": umask,
    "id": who,
    "cwd": os.getcwd(),
    "locale": list(locale.getlocale()),
    "tzname": list(time.tzname),
    "python": sys.version.split()[0],
    "platform": sys.platform,
    "env_names_only": sorted(os.environ.keys()),
    "command_that_must_pass_locally": cmd,
    "model_note": "free-model draft; not a pin",
    "server_note": "free server is scratch; not CI",
}
with open(out, "w", encoding="utf-8") as f:
    json.dump(receipt, f, indent=2)
    f.write("\n")
print("wrote", out)
if receipt["dirty"]:
    print("worktree is dirty; review git diff before you trust a copy")
PY

Run it on the free server before you copy files.
Run it again on your laptop after the copy lands.
Then diff the two JSON files; that is the whole method.

chmod +x scripts/session-receipt.sh
MUST_PASS_LOCALLY="python -m pytest -q" ./scripts/session-receipt.sh /tmp/remote-receipt.json
# after copy
MUST_PASS_LOCALLY="python -m pytest -q" ./scripts/session-receipt.sh /tmp/local-receipt.json
python3 - <<'PY'
import json
remote = json.load(open("/tmp/remote-receipt.json"))
local = json.load(open("/tmp/local-receipt.json"))
keys = ["python", "platform", "locale", "tzname", "umask", "git_head"]
for key in keys:
    same = remote.get(key) == local.get(key)
    print(key, "match=" + str(same), "remote=", remote.get(key), "local=", local.get(key))
if local.get("dirty"):
    print("LOCAL_DIRTY")
PY

If git_head matches and dirty is false, the tree is honest.
If locale or umask differ, inspect generated files with extra care.

Decision table

Use this table before you click merge.

Claim you heard	Evidence that would support it	What I do instead
Remote build was green	CI image digest matches the box	Rerun the CI command locally
Free model drafted it	A written note that it was a draft	Keep the note; do not pin a fake hash
Files exist on the server	`git add -p` hunks you can explain	Commit only explained hunks
Locale cannot matter	Receipt locale equals laptop locale	Diff receipts; inspect generated files
Teammate can continue	Two receipts plus an explained diff	Refuse a memory-only handoff

Read the middle column twice before you argue.
If you lack that evidence, you are holding a story.

What this receipt does not prove

The receipt does not pin a model weight or vendor SKU.
The receipt does not clone a CI container or a runner label.
The receipt does not redact secrets hiding in env values.
That is why I store env names only, never values.

Free servers go away, and free models change without a changelog.
I do not claim quotas, hardware, duration, or uptime here.
Those numbers were not supplied, so they are not in this FAQ.

Who should not use this workflow

Do not put customer data on a shared free server.
Do not treat this script as a substitute for real CI.
Do not use it if you need bit-for-bit model replay.
Do not use it for regulated builds or release signing.

Release engineers need pinned images and signed provenance.
This FAQ is for people copying drafts off scratch boxes.

If your threat model includes other tenants, stop here.
A free server is a convenience, not a tenancy boundary.

A local gate I refuse to skip

After the receipt diff looks boring, I still run this.

test -z "$(git status --porcelain)" || echo "DIRTY TREE"
python3 - <<'PY'
import json, os
cmd = json.load(open("/tmp/local-receipt.json"))["command_that_must_pass_locally"]
print("running:", cmd)
raise SystemExit(os.system(cmd))
PY

If that command fails locally, the remote green was theater.
I send the patch back and I do not merge on memory.

Closing

So what did the free session actually prove?
It proved a draft could run in one unpaid environment.
That result is useful, but it is not a freeze.
It is also not CI, no matter how green it looked.

Export the receipt before you copy anything down.
If you skip that step, you are merging a rumor.

FAQ: A Sandbox Build Is Not a Release Receipt

Jordan Huang — Sat, 19 Sep 2026 15:50:33 +0000

Did the agent compile your app on a scratch host?
Did you almost treat that log as a release?
I keep hearing both claims in pull requests.

This FAQ kills five claims I still hear.
Each claim dies with a command you already have.
No dashboard screenshot counts as evidence here.

What this FAQ covers

I am talking about code that only built remotely.
I am not talking about model leaderboard drama.
I am not replacing your real pipeline either.

A scratch box is a compiler for ideas.
A release is a hashed tree plus recorded commands.
Those two objects are easy to confuse today.

The loop I actually trust

I still let an agent thrash in a sandbox.
Sometimes that sandbox is a free remote server.
MonkeyCode is one place I use for that loop.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

It has free model access and a free server option.
I still refuse to ship whatever that box compiled.
The receipt workflow below works on any host.

Myth 1: A sandbox compile is a release candidate

Claim: If it built there, it can ship.

What people repeat: The compiler already blessed the tree.
What that log proves: Some compiler accepted some files.
What it does not prove: Your release toolchain accepts them.

Ask one rude question before you tag anything.
Which exact compiler binary produced that success line?
If you cannot name it, you do not have a receipt.

# On the sandbox, after the agent shouts success:
which cc || true
cc --version || true
npm -v && node -v || true
python -V && pip -V || true
go env GOVERSION GOTOOLCHAIN || true
git rev-parse HEAD
git status --porcelain=v1

Copy those lines into the pull request body.
If the agent cannot produce them, stop merging.
A green sentence in chat is not a compiler identity.

Corrected mental model

Treat the sandbox as a noisy preview compiler.
Treat your runner as the only compiler that counts.
Preview compilers do not get to mint versions.

Myth 2: The sandbox toolchain matches production

Claim: It is Linux, so the versions are close enough.

What people repeat: The container felt normal enough.
What you can measure: File names under /usr lie often.
What you still need: A pinned toolchain file you own.

# Record what the sandbox actually is.
uname -a
cat /etc/os-release
command -v node && node -p process.version
command -v python3 && python3 -c 'import sys; print(sys.version)'
command -v rustc && rustc -V

Now run the same block on your release image.
Diff the two text files without mercy or hope.
A missing minor version still breaks native addons.

I keep a tiny allowlist, not a vibe check.
If node drifts, the receipt must fail closed.
Close is boring. Boring is the point here.

# toolchain.allowlist example
node==20.17.0
python==3.12.6

Corrected mental model

OS family is not a toolchain pin.
Pins live in files you review and hash.
Everything else is tourism, not packaging.

Myth 3: The agent saying "build succeeded" is an artifact

Claim: The model reported success, so the process did.

What people repeat: The last message looked official.
What that string is: Generated prose about a command.
What an artifact is: Bytes plus an exit code you stored.

Would you accept a human saying "tests passed"?
Would you skip JUnit because the standup sounded confident?
Then do not accept a chat bubble as exit 0.

# Wrap the real command. Ignore the model's summary.
set -o pipefail
./scripts/build.sh > /tmp/build.stdout 2> /tmp/build.stderr
echo $? > /tmp/build.exit
wc -l /tmp/build.stdout /tmp/build.stderr
sha256sum /tmp/build.stdout /tmp/build.stderr /tmp/build.exit

Check the exit file, not the assistant paragraph.
If the wrapper never ran, you have theater.
Theater does not belong in a release checklist.

Corrected mental model

Summaries are commentary. Receipts are files.
Commentary can be wrong and still sound calm.
Files can be hashed. Calm cannot be hashed.

Myth 4: The UI file list is the working tree

Claim: The sidebar showed the patch, so disk matches.

What people repeat: I watched the diff render live.
What a UI list is: A view the tool chose to paint.
What git cares about: Blobs in the index and worktree.

I do not argue with screenshots anymore.
I ask for names, sizes, and hashes instead.
If those three disagree, the UI lost.

git diff --name-only
git diff --stat
git ls-files -v | awk '$1 ~ /[S?]/ {print}'
find . -type f -not -path './.git/*' -print0 | \
  sort -z | xargs -0 sha256sum > /tmp/tree.sha256

Then compare /tmp/tree.sha256 with your laptop copy.
One missing file ends the debate immediately.
Pretty file trees are not Merkle trees.

Corrected mental model

The working tree is the product.
The chat panel is a projector.
Projectors do not ship.

Myth 5: The same prompt rebuilds the same tree tomorrow

Claim: I can recreate this with the original request.

What people repeat: The prompt is the build script.
What a prompt is: A wish with unbounded inputs.
What a rebuild needs: Bytes, commands, and versions frozen.

Prompts do not pin package indexes.
Prompts do not pin compiler patches.
Prompts do not pin the agent's extra tool calls.

If you need the tree again, store the tree.
Store the commands that mutated it too.
Do not store a vibe and call it provenance.

git add -A
git diff --cached > /tmp/agent.patch
sha256sum /tmp/agent.patch
# Keep the patch. Throw away the prompt fan fiction.

Corrected mental model

Reproducibility starts after the agent stops talking.
Your job is to freeze the result, not the chat.
Chat is grease. Grease is not a spec.

Artifact: a local release receipt

I use one script after every sandbox session.
It does not care which host wrote the files.
It fails if you only have a story.

Save this as sandbox_receipt.py and run it locally.

#!/usr/bin/env python3
"""Record a sandbox session as a hashed receipt.

Label: example script. Run it on a git checkout you control.
It does not talk to any vendor API.
"""
from __future__ import annotations

import argparse
import hashlib
import json
import os
import subprocess
import sys
from datetime import datetime, timezone
from pathlib import Path


def run(cmd: list[str]) -> tuple[int, str]:
    proc = subprocess.run(cmd, text=True, capture_output=True, check=False)
    out = (proc.stdout or "") + (proc.stderr or "")
    return proc.returncode, out.strip()


def sha256_file(path: Path) -> str:
    h = hashlib.sha256()
    with path.open("rb") as fh:
        for chunk in iter(lambda: fh.read(65536), b""):
            h.update(chunk)
    return h.hexdigest()


def toolchain() -> dict[str, str]:
    keys = {
        "uname": ["uname", "-a"],
        "node": ["node", "-v"],
        "python": ["python3", "-V"],
        "git": ["git", "--version"],
    }
    found: dict[str, str] = {}
    for name, cmd in keys.items():
        code, text = run(cmd)
        found[name] = text if code == 0 else f"missing:{code}"
    return found


def changed_files() -> list[str]:
    code, text = run(["git", "diff", "--name-only", "HEAD"])
    if code != 0:
        raise SystemExit("git diff failed; are you in a repo?")
    extra_code, extra = run(["git", "ls-files", "--others", "--exclude-standard"])
    if extra_code != 0:
        raise SystemExit("git ls-files failed")
    names = [line for line in (text.splitlines() + extra.splitlines()) if line]
    return sorted(set(names))


def main() -> int:
    parser = argparse.ArgumentParser()
    parser.add_argument("--build-cmd", default="")
    parser.add_argument("--out", default="receipt.json")
    args = parser.parse_args()

    head_code, head = run(["git", "rev-parse", "HEAD"])
    status_code, status = run(["git", "status", "--porcelain=v1"])
    if head_code != 0 or status_code != 0:
        print("Need a git checkout you control.", file=sys.stderr)
        return 2

    files = changed_files()
    hashed = []
    missing = []
    for name in files:
        path = Path(name)
        if not path.is_file():
            missing.append(name)
            continue
        hashed.append({"path": name, "sha256": sha256_file(path), "bytes": path.stat().st_size})

    build = {"skipped": True, "exit": None, "tail": ""}
    if args.build_cmd:
        code, text = run(["bash", "-lc", args.build_cmd])
        build = {"skipped": False, "exit": code, "tail": text[-2000:]}

    receipt = {
        "recorded_at": datetime.now(timezone.utc).isoformat(),
        "cwd": os.getcwd(),
        "git_head": head,
        "git_status": status.splitlines(),
        "toolchain": toolchain(),
        "changed_file_count": len(files),
        "changed_files": hashed,
        "missing_paths": missing,
        "build": build,
    }
    Path(args.out).write_text(json.dumps(receipt, indent=2) + "\n")
    print(f"wrote {args.out} with {len(hashed)} hashed files")
    if missing:
        print("missing paths:", ", ".join(missing), file=sys.stderr)
        return 3
    if not hashed:
        print("no changed files; nothing to receipt", file=sys.stderr)
        return 4
    if args.build_cmd and build["exit"] != 0:
        return 5
    return 0


if __name__ == "__main__":
    sys.exit(main())

Run it on the machine you actually own.
Pull the patch first if the sandbox still holds it.
Then force a local rebuild you can name.

python3 sandbox_receipt.py --build-cmd 'npm test --silent' --out receipt.json
python3 -c 'import json; print(json.load(open("receipt.json"))["build"])'

Attach receipt.json to the pull request.
Argue with the JSON, not the chat transcript.
If the JSON is thin, the change is not ready.

Decision table

Signal you saw	What it proves	What it does not prove
Sandbox compile line	A compiler on that box accepted a tree	Your release compiler accepts it
Agent said success	The model emitted that sentence	A process exited zero
UI showed a file	The tool rendered a path	Bytes on disk match
Free server still answers	You can reach it today	The host exists next week
Same prompt reused	You asked again	The same patch bytes return
`receipt.json` with hashes	You froze a tree and a command	Performance, security, or license clearance

Print that table next to the PR template.
It stops a surprising number of fake merges.
People hate it, then they start using it.

Limitations

This receipt is not supply-chain provenance.
It is not SLSA, not Sigstore, not a bill of materials.
It will not catch a malicious registry swap.

It also will not freeze a vanished free host.
Do not keep secrets on a scratch server.
Do not treat a deleted box as an audit log.

Hashes of source are not hashes of images.
A local npm test is not production traffic.
If you need bit-for-bit builds, use a real hermetic system.

Who should not use this

Skip this if you already have hermetic CI.
You do not need another JSON souvenir then.
Skip this if you cannot run git locally.

Skip this for regulated binaries with named signers.
A receipt script is not a signing ceremony.
Skip this if the sandbox still holds production secrets.

What I want in review

I want three files, not a victory speech.
I want the patch, the receipt, and the command log.
I want the toolchain block to match the release image.

Can the agent still help? Yes, in the sandbox.
Can the sandbox mint a release? No, never.
That split is the whole FAQ.

Steal the script. Then argue with the JSON.

Lock Collation Before You Merge a Generated Concat Step

Jordan Huang — Sat, 19 Sep 2026 12:41:49 +0000

The merge request is green on the author's laptop. CI is not.

bundle.json has the wrong checksum, and the job log still claims every fragment was included. The generated pack.sh concatenates fragments/*.json. That script came from an agent session on a remote Linux box. You are about to debug a flake that is not a flake.

Collation did this. Path order is a build input. Treat it like one.

Read the runner locale before you reread the chat

Open the CI log first. Print the environment the job actually had, not the environment the agent described.

locale
python3 -c 'import locale, sys; print(sys.platform); print(locale.getlocale())'
echo "LANG=${LANG-}"
echo "LC_ALL=${LC_ALL-}"
echo "LC_COLLATE=${LC_COLLATE-}"

Print the same block on the machine that produced the patch. Do not argue from memory. en_US.UTF-8 and C do not sort the same byte strings.

A concat step that uses an unquoted glob is a sort step in disguise. The shell expands fragments/*.json in collation order. Change the locale and you change the bundle. A checksum test then fails even though every file is present.

Mixed machines make this look like CI noise

You review on macOS. The agent wrote the script on Linux. Many hosted runners look closer to C.UTF-8 than to a desktop GUI locale. None of those hosts is broken. They are simply different inputs.

Generated code makes the gap worse. The model copies a glob because the glob is short. Short is not deterministic. You want an explicit file list, or LC_ALL=C plus a documented sort, or both.

Names that differ only by case sit in the same bucket. Linux will keep Auth.json and auth.json as two files. Default macOS APFS may not. An agent on a Linux box can create both. Your laptop then reports one path, and CI reports two. That is not a flaky hasher. That is a filesystem contract bug sitting next to the collation bug.

Recreate the mismatch in a throwaway directory

Work somewhere disposable. Treat the next script as a demonstration template, not as a result you already measured.

# proposal: demo_collation.sh — unexecuted template
set -euo pipefail
rm -rf /tmp/collation-demo
mkdir -p /tmp/collation-demo/fragments
cd /tmp/collation-demo

printf '{"k":1}\n' > fragments/a.json
printf '{"k":2}\n' > fragments/B.json
printf '{"k":3}\n' > fragments/c.json

pack() {
  local out="$1"
  : > "$out"
  for f in fragments/*.json; do
    cat "$f" >> "$out"
    printf '\n' >> "$out"
  done
}

LC_ALL=en_US.UTF-8 pack bundle-utf8.json || LC_ALL=C.UTF-8 pack bundle-utf8.json
LC_ALL=C pack bundle-c.json

sha256sum bundle-utf8.json bundle-c.json
echo "desktop-like order:"
LC_ALL=en_US.UTF-8 bash -lc 'printf "%s\n" fragments/*.json' || true
echo "C order:"
LC_ALL=C bash -lc 'printf "%s\n" fragments/*.json'

Run it where both locales exist. If locale -a does not list en_US.UTF-8, do not install extra locales on a production runner just to imitate a laptop. Pin the script instead.

Watch the two checksums. If they differ, you have the bug in miniature. If they match, keep the uppercase file. C locale puts B.json before a.json. Many UTF-8 locales do not. That single swap is enough to poison a concatenated OpenAPI file, a SQL migration chain, or a changelog blob.

Make order a reviewed artifact

Do not set a locale in CI and hope the next generated script obeys it. Put the contract in the file that ships.

# proposal: pack.sh — manifest first, POSIX collation as belt-and-suspenders
#!/usr/bin/env bash
set -euo pipefail
export LC_ALL=C
export LANG=C

root="$(cd "$(dirname "$0")" && pwd)"
list="$root/fragments.manifest"
out="$root/bundle.json"

if [[ ! -f "$list" ]]; then
  echo "missing $list" >&2
  exit 2
fi

: > "$out"
while IFS= read -r rel || [[ -n "$rel" ]]; do
  [[ -z "$rel" || "$rel" == \#* ]] && continue
  path="$root/$rel"
  if [[ ! -f "$path" ]]; then
    echo "missing fragment $path" >&2
    exit 3
  fi
  cat "$path" >> "$out"
  printf '\n' >> "$out"
done < "$list"

Keep fragments.manifest in git. One relative path per line. Comments allowed. An agent may still draft that list. You still review the order as a product decision, because schema fragments and migrations are ordered documents.

If a manifest is truly impractical, sort with a stable key instead of trusting the glob:

# proposal: fallback only for generated snapshots, not schema history
export LC_ALL=C
mapfile -t files < <(find "$root/fragments" -name '*.json' -type f | LC_ALL=C sort)

find plus sort beats a raw glob. It still loses to a reviewed list. Use the fallback for snapshot dumps. Do not use it for anything whose sequence is load-bearing.

Gate the tree, not the transcript

Add a check that never needs the original agent host. It reads your scripts and your paths.

#!/usr/bin/env python3
"""proposal: collation_gate.py — template, not a shipped metric."""
from __future__ import annotations

import os
import re
import subprocess
import sys
from pathlib import Path

UNQUOTED_GLOB = re.compile(r"(?<![\"'\w])([A-Za-z0-9_./-]*\*[A-Za-z0-9_./-]*)")
SKIP_DIRS = {".git", ".venv", "node_modules", "__pycache__", ".mypy_cache"}


def tracked_files(root: Path) -> list[Path]:
    proc = subprocess.run(
        ["git", "-C", str(root), "ls-files", "-z"],
        check=True,
        capture_output=True,
    )
    out: list[Path] = []
    for raw in proc.stdout.split(b"\0"):
        if not raw:
            continue
        out.append(root / Path(raw.decode()))
    return out


def bash_glob(dir_path: Path, lc_all: str) -> list[str]:
    env = os.environ.copy()
    env["LC_ALL"] = lc_all
    env["LANG"] = lc_all
    quoted = dir_path.as_posix().replace("'", "'\\''")
    proc = subprocess.run(
        ["bash", "-lc", f"printf '%s\\n' '{quoted}'/*"],
        check=False,
        capture_output=True,
        text=True,
        env=env,
    )
    return [line for line in proc.stdout.splitlines() if line]


def collation_problems(root: Path) -> list[str]:
    hits: list[str] = []
    dirs = sorted({path.parent for path in tracked_files(root) if path.is_file()})
    current = os.environ.get("LC_ALL") or os.environ.get("LANG") or "C"
    for directory in dirs:
        if SKIP_DIRS.intersection(directory.relative_to(root).parts):
            continue
        c_order = bash_glob(directory, "C")
        here = bash_glob(directory, current)
        if c_order != here and len(c_order) > 1:
            rel = directory.relative_to(root).as_posix() or "."
            hits.append(f"collation drift in {rel}/ under LC_ALL={current!r}")
    return hits


def glob_problems(root: Path) -> list[str]:
    hits: list[str] = []
    for path in tracked_files(root):
        if path.suffix not in {".sh", ".bash", ".zsh"}:
            continue
        text = path.read_text(encoding="utf-8", errors="replace")
        for index, line in enumerate(text.splitlines(), 1):
            stripped = line.split("#", 1)[0]
            if "LC_ALL=C" in text and "sort" in stripped:
                continue
            match = UNQUOTED_GLOB.search(stripped)
            if match:
                rel = path.relative_to(root).as_posix()
                hits.append(f"unquoted glob {rel}:{index}: {match.group(1)}")
    return hits


def case_collisions(root: Path) -> list[str]:
    seen: dict[str, str] = {}
    hits: list[str] = []
    for path in tracked_files(root):
        rel = path.relative_to(root).as_posix()
        key = rel.lower()
        if key in seen and seen[key] != rel:
            hits.append(f"case collision {seen[key]} vs {rel}")
        else:
            seen[key] = rel
    return hits


def main() -> int:
    root = Path(sys.argv[1] if len(sys.argv) > 1 else ".").resolve()
    problems = collation_problems(root) + glob_problems(root) + case_collisions(root)
    if not problems:
        print("collation_gate: ok")
        return 0
    print("collation_gate: fail")
    for item in problems:
        print(f"- {item}")
    return 2


if __name__ == "__main__":
    raise SystemExit(main())

Run it on a clean checkout. Pass the repo root if you are not already there.

# proposal: local invocation
python3 collation_gate.py .

Wire the same command into GitLab CI so the gate does not depend on a laptop locale. Keep the image boring.

# proposal: .gitlab-ci.yml fragment — unexecuted template
collation_gate:
  image: python:3.12-bookworm
  script:
    - locale
    - python3 collation_gate.py .

The job should fail closed. A missing fragments.manifest is a product defect, not a warning you scroll past.

Decide with a table, then stop debating hosts

What you observed	Safe to merge?	What to require
Unquoted glob, no `LC_ALL=C`, checksum test only	No	Manifest or POSIX sort in the script
Glob plus `export LC_ALL=C` and a CI locale dump	Maybe	Human review of file order
Reviewed `fragments.manifest` committed next to `pack.sh`	Yes, as order	Still run the checksum in CI
Two tracked paths differ only by case	No	Rename on a case-sensitive clone
Laptop green, CI red, locales never printed	No	Print `locale` on both sides first
Agent said "all files included" and showed stdout	No	Compare git paths, not chat

"Maybe" still needs a reviewer. Tables do not merge requests.

A spare Linux shell helps. Its locale is not the contract

If you work on macOS and the runner is Linux, you need one case-sensitive shell to see the real tree. That is the whole reason to use a remote box for this check.

MonkeyCode offers free model access and a free server option, which is enough to draft pack.sh and then rerun the concat under Linux. Disclosure: This article was prepared as part of MonkeyCode's product outreach. Use that server to print locale, run collation_gate.py, and catch Auth.json versus auth.json. Do not use it as the definition of production order.

Copy the locale dump into the merge request. Leave the generated script behind only after the manifest exists. The free box is a rehearsal stage. The committed list is the show.

Numbered path from red CI to a mergeable script

Print locale in CI and on the machine that wrote pack.sh.
Recreate the two-locale concat in a throwaway directory.
Replace the glob with fragments.manifest or with find | LC_ALL=C sort.
Commit the manifest next to the script.
Run python3 collation_gate.py . on a case-sensitive clone.
Fail the pipeline if the gate exits 2.
Review the order as a behavior change, then merge.

Skip step 7 and you only moved the bug from the glob into a file nobody read.

What this gate does not prove

A stable sort is not a schema review. A manifest can still concatenate the wrong documents in a confident order. Checksums do not tell you the JSON is valid.

LC_ALL=C is also the wrong hammer for user-facing sort. If the product must sort names for German or Swedish readers, do that in application code with an explicit locale. Do not inherit it from a build host.

Skip this workflow when the artifact is a locale-aware UI list. Skip it when you cannot run a POSIX shell. Skip it for regulated trees you must not send to a third-party host. Solo spikes can live with a glob for an afternoon. Ordered migrations cannot.

Clocks, containers, and uname will not save you here. The file list will.

What belongs in the merge request

Paste the locale dump. Paste the manifest diff. Paste the gate output.

If the answer is a screenshot of an agent saying the bundle looks complete, reject the request. Completeness is not order. Order is data. Put that data in git before anyone concatenates another file.

The Agent Wrote Tests. Did Anything Actually Run?

Jordan Huang — Sat, 19 Sep 2026 10:23:22 +0000

Did your coding agent just add tests today?
Then I have one unkind follow-up question.

What process executed those newly added test files?
Which exit code did that process return?

I keep hearing four claims in review threads.
They sound like diligence, but they are theater.

This FAQ busts those four repeated claims.
Then it gives you a claimed-versus-actual proof pack.

The background nobody wants to name

Chat UIs still reward a complete-sounding story.
A model can narrate a green suite in one breath.

You paste that story into a pull request.
Busy reviewers treat the story as a result.

Have you shipped that story this month?

Trend pieces keep arguing about vibe coding lately.
The real engineering failure is smaller and meaner.

We confuse three objects that are not equal.
Text, files, and runner results are different objects.

Myth 1: A new test file means coverage

Claim: The agent added tests, so the change is safe.

What developers repeat: test_api.py exists. Merge it.

Evidence that would actually count:

Git tracks the test module
A collector imported the module
Assertions ran in a process
The process exit code was zero

Corrected mental model: Authorship of a test is not execution.
A test that never loaded is commented-out hope.

Ask one blunt question at review time.
Did an interpreter import this test module today?

Collect before you trust

# proposed check; not a timed benchmark
git ls-files '*test*.py' '*_test.py' > /tmp/tracked_tests.txt
python -m pytest --collect-only -q | tee /tmp/collected.txt
echo "tracked=$(wc -l < /tmp/tracked_tests.txt)"

If collect-only cannot see the file, stop.
You are reviewing prose that happens to use def.

Parametrize can also lie in the other direction.
One function can explode into twenty node ids.

Count nodes, not functions, when you compare claims.

Myth 2: The recap is the suite result

Claim: "All fourteen tests passed."

Who actually counted those fourteen tests, exactly?
Was it the model, pytest, or a screenshot of neither?

Natural language integers stay cheap to hallucinate.
JUnit XML is boring, and that is the point.

Corrected mental model: You should believe stored artifacts, not chat sentences.
A sentence cannot sign a test report.

Decision table

Chat claim	Evidence?	Why it fails or holds
"All tests passed"	No	No runner identity, no exit code
Pasted pytest summary	Weak	Transcripts are easy to fabricate
`junit.xml` on disk	Yes	The runner wrote it
CI log URL	Yes	A third system stored it
Pass count without skips	Careful	Skips can inflate a green story
Pass count without errors	Careful	`errors` are not `failures`

Pytest treats import crashes as errors, not failures.
Agents often fold errors into failed or hide them.

Did the recap mention skipped tests at all?
Did it mention xfail or collection errors?

If those words never appeared, distrust the integer.

Myth 3: Self-check is a second run

Claim: The agent re-read the tests and confirmed them.

That is one narrator grading its own homework.
Would you let the compiler mark its own exam?

Corrected mental model: The real oracle always lives outside the chat.
A host process is an oracle, and a recap is not.

You need a split, not a pep talk.
Generate fixtures in one place, then execute in another.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

A practical split can look like this.
Draft the harness with MonkeyCode's free model access.
Then execute that harness on the free server option.

The value is the split, not the logo.
Any model plus any scratch VM can do this.

Keep generation away from the oracle on purpose.
That split is the whole method here.

Myth 4: Skipping the runner saves time

Claim: The diff is tiny, so tests are overhead.

Tiny diffs still rename symbols under you.
Tiny diffs still invert a boolean in a guard.

The hour you save becomes incident time later.
That trade looks smart until production pages you.

Corrected mental model: The test runner remains the cheapest review you have.
A short pytest run beats a long outage thread.

Also, coding agents retry without telling you.
Retries without an oracle duplicate side effects you never measured.

You do not need a lab for this lesson.
You need an exit code on disk.

Artifact: claimed versus actual

You should stop arguing with the chat transcript.
Serialize both stories, then compare them on disk.

This script is only a proposed oracle.
It is not a vendor benchmark, and it is not timed.

#!/usr/bin/env python3
"""claimed_vs_actual.py — proposed oracle, not production proof."""
from __future__ import annotations

import json
import subprocess
import sys
import xml.etree.ElementTree as ET
from pathlib import Path

CLAIM = Path("claimed.json")
JUNIT = Path("junit.xml")
PROOF = Path("proof.json")


def load_claim() -> tuple[int, int, int]:
    data = json.loads(CLAIM.read_text())
    passed = int(data["passed"])
    failed = int(data["failed"])
    skipped = int(data.get("skipped", 0))
    return passed, failed, skipped


def run_suite() -> int:
    cmd = [
        sys.executable, "-m", "pytest", "-q",
        f"--junitxml={JUNIT}",
    ]
    return subprocess.run(cmd).returncode


def read_junit() -> tuple[int, int, int]:
    root = ET.parse(JUNIT).getroot()
    suites = list(root) if root.tag == "testsuites" else [root]
    passed = failed = skipped = 0
    for suite in suites:
        tests = int(suite.attrib.get("tests", 0))
        failures = int(suite.attrib.get("failures", 0))
        errors = int(suite.attrib.get("errors", 0))
        skipped_n = int(suite.attrib.get("skipped", 0))
        failed += failures + errors
        skipped += skipped_n
        passed += tests - failures - errors - skipped_n
    return passed, failed, skipped


def main() -> None:
    if not CLAIM.exists():
        print("missing claimed.json; copy the agent's counts first")
        sys.exit(2)
    exit_code = run_suite()
    if not JUNIT.exists():
        print("pytest wrote no junit.xml; collection probably died")
        sys.exit(3)
    actual_p, actual_f, actual_s = read_junit()
    claim_p, claim_f, claim_s = load_claim()
    report = {
        "claimed_passed": claim_p,
        "claimed_failed": claim_f,
        "claimed_skipped": claim_s,
        "actual_passed": actual_p,
        "actual_failed": actual_f,
        "actual_skipped": actual_s,
        "pytest_exit": exit_code,
        "counts_match": (
            claim_p == actual_p
            and claim_f == actual_f
            and claim_s == actual_s
        ),
        "runner_green": exit_code == 0,
    }
    PROOF.write_text(json.dumps(report, indent=2) + "\n")
    print(json.dumps(report, indent=2))
    ok = report["counts_match"] and report["runner_green"]
    sys.exit(0 if ok else 1)


if __name__ == "__main__":
    main()

Seed the claim from chat

{
  "passed": 14,
  "failed": 0,
  "skipped": 0,
  "source": "agent-recap",
  "note": "copy numbers from chat; do not trust them"
}

Run the pack

# proposed workflow on any scratch host
python -m venv .venv
. .venv/bin/activate
pip install pytest
printf '%s\n' '{"passed":14,"failed":0,"skipped":0}' > claimed.json
python claimed_vs_actual.py
cat proof.json

If counts_match is false, the chat drifted.
If the runner never wrote XML, you had no suite.

Commit junit.xml and proof.json with the PR.
Now reviewers can argue with files, not vibes.

One-hour checklist

Freeze the agent's pass, fail, and skip counts.
Write those integers into claimed.json before running.
Collect tests with --collect-only and read node ids.
Execute the proof pack on a real interpreter.
Attach proof.json plus the JUnit file to the PR.
Reject the change when runner_green is false.

No dashboard is required for this gate.
No proprietary runner is required for this either.

Limitations, said plainly

Matching counts still do not mean correct assertions.
Wrong tests that run are still wrong tests.

This pack does not measure flake rates.
It also does not measure a mutation score.
It does not measure runtime performance at all.
It does not measure any real security properties.

Free model access is only a convenience.
A free server option is only a convenience.
I will not invent token quotas or hardware claims.

Treat both as optional tools for the split.
Do not treat them as a contract or an SLO.

Who should not use this approach

Skip this if CI already stores JUnit on every merge.
Skip this when the code cannot leave a locked network.
Skip this if secrets must never reach a scratch host.
Skip this when no human still owns the contract.
Skip this if you want a recap to replace design work.

If CI already gates merges, do not add theater.
Wire the pack into CI, or skip the extra ritual.

Keep this mental model

Three objects exist after every agent coding session.

Text the model wrote in a transcript
Files git actually recorded
Results a runner produced on a host

Only object three can turn a suite green.
Object one is a story, and object two is an input.

So, did the agent write tests this time?
Good, now make a real process run them.

A Green Install Is Not a Pin: Five Dependency Myths

Jordan Huang — Sat, 19 Sep 2026 06:18:50 +0000

Did your coding agent just fix the install? I keep seeing that victory line in traces. The command then printed a clean exit code. Then someone copies node_modules like a relic.

That souvenir is not a real dependency pin. It is leftover dust from a scratch disk. Want the rude version? The graph was never frozen.

What this FAQ is actually about

This is not a packaging sermon for librarians. This is about agent loops on scratch hosts. The model edits package.json with a confident range. The host runs some flavor of install. Chat says the work is done.

Did the graph freeze? Did CI receive the same bytes? Usually no. The chat just narrated a feeling.

I wrote this as a myth list you can run. Each myth has a check. Each check has a command you can paste. Steal the script at the end. Then argue with hashes, not vibes.

Myth 1: A package.json edit is a pin

The agent bumped a version range. The range looks adult and responsible. You feel oddly safe.

A range is a wish. A lockfile is a map. Which file would you restore after a fire? If the answer is package.json alone, you will drift.

Evidence you can collect in two minutes

Run these in the repo the agent touched:

git diff --stat -- package.json package-lock.json yarn.lock pnpm-lock.yaml
git status --porcelain -- package.json '*lock*'

Look at the porcelain lines. Did the lockfile move with the manifest? If the lockfile stayed silent, the pin is theater.

Manifest-only diff: the agent wrote a wish.
Lockfile-only diff: someone regenerated in the dark.
Both files in one commit: you might have a pin.

Corrected model: treat the lockfile as the artifact. Treat the manifest as commentary on intent.

Myth 2: Install exit code 0 means the graph is stable

Green text feels like proof. Agents love green text. Humans do too. Why would we not?

Exit code 0 means the installer did not die. It does not mean the tree is replayable tomorrow. Optional deps fail open. Peer deps warn and continue. Platform binaries swap without a speech.

Lifecycle scripts can mutate files after the resolver stops. Did you capture that? Or only the last happy line?

A tiny probe

npm install --dry-run --json > /tmp/install-dry.json
# frozen path, when a lockfile already exists
npm ci --ignore-scripts --dry-run

Read the warnings. Count peer issues. Did a native addon pick a platform? If you only stored agent stdout, you stored a mood. You did not store a graph.

Corrected model: success is a frozen installer command. Add a hashed lockfile. Keep both.

Myth 3: The scratch host's node_modules will match CI

This one bites teams on free servers. The sandbox is some Linux userland. Your laptop is not that userland. CI might use another Node line.

The agent compiled a binary. The binary linked a glibc it found. Your Mac never will. So why copy node_modules off a scratch disk? You are copying an accident with extra steps.

Fingerprint before you trust the tree

uname -s -m
node -v
corepack --version || true
git rev-parse HEAD
sha256sum package-lock.json 2>/dev/null || shasum -a 256 package-lock.json

Put those five lines next to the test output. If CI disagrees on any line, the tree is local color. It is not a release.

I replay some of these checks on MonkeyCode. Disclosure: This article was prepared as part of MonkeyCode's product outreach. It offers free model access and a free server option. That pair is a scratch lane for me. It is not a substitute for CI. I still hash the lockfile. I still refuse to ship the sandbox tree.

Corrected model: the host is a lab bench. CI remains the court of bytes.

Myth 4: Tests passing means you can skip the lockfile

The agent ran the unit suite. All dots went green. Someone deleted the lockfile from the PR. Review got quieter. Supply chain got louder.

I get the temptation. Lockfiles are noisy diffs. Reviewers complain about churn. Tests sample behavior. They do not sample the entire graph. A transitive bump can stay green. It can still change who you trust.

The check I actually want in review

# Fail the job if the lockfile moves after a frozen install
npm ci
git diff --exit-code -- package-lock.json

If that exits 1, the agent did not pin. The PR is incomplete. Green tests with a dirty lock are a failed job. Say that in the template.

Corrected model: a suite is evidence of behavior. A lock hash is evidence of the graph. You need both on the same commit.

Myth 5: Re-running the same prompt rebuilds the same tree

Agents do not replay like compilers. Prompts are not makefiles. Registries move. Resolvers change defaults. Models improvise a second "fix".

You hit retry. You got another story. The second tree is not the first tree. Did you keep the first lockfile hash? If not, you cannot detect the drift. You can only reread the chat.

Capture once, compare forever

mkdir -p .replay
{
  echo "head=$(git rev-parse HEAD)"
  echo "node=$(node -v)"
  echo "lock=$(sha256sum package-lock.json | awk '{print $1}')"
} > .replay/pin.txt

Next run, diff the file. If lock= changed, you did not reproduce. You rolled the dice and named it science.

Corrected model: reproduction is hash equality. Chat similarity is noise with punctuation.

Artifact: a pin-proof script and a decision table

Do not copy the sandbox. Prove the pin. Here is a script I keep as scripts/pin_proof.sh. Treat it as a labeled workflow sample. Adapt the paths. I am not claiming production metrics.

#!/usr/bin/env bash
# pin_proof.sh — record whether an agent install is actually pinned
set -euo pipefail

out="${1:-.replay/pin_proof.json}"
mkdir -p "$(dirname "$out")"

lock=""
for cand in package-lock.json pnpm-lock.yaml yarn.lock; do
  if [[ -f "$cand" ]]; then
    lock="$cand"
    break
  fi
done

if [[ -z "$lock" ]]; then
  echo "no lockfile found" >&2
  exit 2
fi

hash="$(sha256sum "$lock" | awk '{print $1}')"
dirty="$(git status --porcelain -- "$lock" package.json || true)"
head="$(git rev-parse HEAD 2>/dev/null || echo unborn)"

node_v="$(node -v 2>/dev/null || echo missing)"
os="$(uname -s -m)"

install_ok="skip"
lock_after="not_npm"
if [[ "$lock" == "package-lock.json" ]] && command -v npm >/dev/null; then
  if npm ci --ignore-scripts; then
    install_ok="npm_ci_ok"
  else
    install_ok="npm_ci_failed"
  fi
  if git diff --exit-code -- package-lock.json; then
    lock_after="unchanged"
  else
    lock_after="mutated"
  fi
fi

python3 - <<PY
import json, os
doc = {
  "git_head": "$head",
  "os": "$os",
  "node": "$node_v",
  "lockfile": "$lock",
  "lock_sha256": "$hash",
  "lock_dirty_before": """$dirty""".strip(),
  "install": "$install_ok",
  "lock_after_ci": "$lock_after",
}
os.makedirs(os.path.dirname("$out") or ".", exist_ok=True)
with open("$out", "w") as f:
    json.dump(doc, f, indent=2)
print(json.dumps(doc, indent=2))
PY

if [[ "$install_ok" == "npm_ci_failed" || "$lock_after" == "mutated" ]]; then
  echo "pin proof failed" >&2
  exit 1
fi

Run it after the agent stops talking.

chmod +x scripts/pin_proof.sh
./scripts/pin_proof.sh .replay/pin_proof.json

Commit .replay/pin_proof.json only if your team wants the fingerprint. Do not commit node_modules. Ever. That directory is a host opinion.

Decision table

Observation	Trust the tree?	Next action
No lockfile	No	Generate one with a frozen installer
Lock dirty after `npm ci`	No	Send the agent back at the lock
`npm ci` fails, `npm install` works	No	The graph is not pinned
OS or Node mismatch versus CI	No	Replay on the CI image
Lock hash matches CI cache	Yes, as a pin	Still run the real pipeline
Tests green, lock missing	No	Tests are not a resolver

Print the table in review. Argue with the row. Do not argue with the chat transcript.

A practical replay workflow

Freeze the git HEAD the agent actually saw.
Require a lockfile in that same commit.
Run scripts/pin_proof.sh on the scratch host.
Run the same script on CI or a matching image.
Merge only when lock hashes match and npm ci stays quiet.

Need a scratch lane for step 3? A free server is enough for the fingerprint. A free model can propose the lock update. Neither one gets merge rights. Tools propose. Proof files dispose.

That is the whole trick. Keep the rights off the chat.

Limitations

This script does not audit the registry. It does not catch a compromised tarball with the same version. It does not replace npm audit. It does not emit a real SBOM.

It assumes a Node repo. Cargo, Go, and pip need the same idea. Swap the frozen command. Keep the hash. Keep the OS line.

npm ci --ignore-scripts skips lifecycle traps. It also skips needed native builds. If your package requires scripts, run a second pass in CI. Do not hide that pass on a laptop and call it policy.

I am not publishing timings. Timings rot by next week. Hashes do not.

Who should not use this

Do not use this as your only release gate. Do not use this if you cannot run npm ci. Do not use this to skip reviewers because the JSON looks official.

If you ship mobile binaries, this Node fingerprint is incomplete. If you vendor native addons, hash those blobs too. If you need a legal provenance chain, use signed attestations. This FAQ is a lab checklist, not a courtroom.

What I want you to remember

The agent did not pin that range. The installer did not freeze the universe. The scratch disk is not your product. Ask one rude question in review. Where is the lock hash?

If the hash is missing, the work is missing. If the hash moved, you did not reproduce. You just got another story with a green exit code.

Run the script on the next agent PR. Paste the JSON in the review. That is the only ask.

FAQ: Five Myths About Tests the Agent Wrote for Itself

Jordan Huang — Sat, 19 Sep 2026 03:12:00 +0000

Who actually graded the agent's homework on that run?
I keep seeing the same claim in review threads.
The agent wrote tests, they passed, so we ship.

That claim is circular because the grader sat nearby.
The same session wrote the answers and the key.

Why this FAQ exists

Green test output is not independent evidence of correctness.
The same loop wrote the code and the checks.
Ask a harder question before you merge.

Did the tests constrain the implementation at all?
Or did they photograph whatever the model just invented?

I use this FAQ when the test tree appears in one burst.
New files in src/ and tests/ often share one commit.
You still need a mental model, not a screenshot.

Myth 1: Agent-written tests are a second opinion

Standup version sounds responsible and even grown-up.
The model implemented the feature, then added tests as review.

What I actually see looks much less independent.
The tests import the same brand-new module.
They assert the happy path the prompt already described.

Demote that suite before you treat it as review.
Treat those tests as documentation of intended shape.
They are not a rival reviewer with rival incentives.

Same session, same context, same hunger for green.
Would you let a candidate grade their own take-home?
I still want the tests, but I demote them hard.

They are clues about intent, not a court verdict.

Commands I run on the commit

git log --diff-filter=A --summary --since='2 hours ago'
git diff --stat HEAD~1
git log --follow -- tests/ | head

Look at which files arrived together this hour.
Implementation and tests in one commit is a smell.
That pattern is a smell, not an automatic ban.

If src and tests move in lockstep, pause there.
Ask which file could fail without the other changing.
If the answer is none, you have a mirror suite.

Myth 2: A high coverage number pins behavior

People quote coverage like it were a lock on meaning.
Coverage is high, so the agent covered the behavior.

Coverage counts executed lines, not meaningful decision branches.
A tautology still paints a source line green.
Asserting a function equals itself pins nothing real.

Coverage is a flashlight, and it is not a lock.
Ask which inputs would change the assertion today.
If you cannot name two, the test is a tour.

Proposal: scan for assertions that cannot fail

This snippet is a proposal, not a scored benchmark.
Adapt the root path before you run anything.

# proposal: tautology_scan.py
from ast import Assert, Compare, dump, parse, walk
from pathlib import Path
import sys

def is_tautology(node: Assert) -> bool:
    test = node.test
    if not isinstance(test, Compare):
        return False
    left = dump(test.left)
    rights = [dump(c) for c in test.comparators]
    return any(left == r for r in rights)

def scan(path: Path) -> list[str]:
    hits = []
    tree = parse(path.read_text(encoding="utf-8"))
    for node in walk(tree):
        if isinstance(node, Assert) and is_tautology(node):
            hits.append(f"{path}:{node.lineno}")
    return hits

if __name__ == "__main__":
    root = Path(sys.argv[1])
    for path in root.rglob("test_*.py"):
        for hit in scan(path):
            print(hit)

python3 tautology_scan.py tests

Run it against the agent's test directory only.
Empty output is not safety, only one cheap filter.
You still need failing cases that you wrote.

Myth 3: Mocks mean the agent understood the boundary

Does a MagicMock prove the adapter was actually designed?
The agent mocked the database, so the tests are hermetic.

Mocks often return the shape the implementation already assumed.
The test never met the real adapter or the real error.
Isolation can freeze a fantasy instead of a contract.

A mock is a second copy of the same story.
Compare mock payloads with a real recorded fixture.
If they only exist in the test file, be suspicious.

Hunt the stuffed animals

grep -R -n "MagicMock\|AsyncMock\|mock.patch\|monkeypatch" tests || true
grep -R -n "return_value\|side_effect" tests || true

Search for return_value lines and read them slowly.
Would that payload survive a schema change next month?
If not, the mock is a stuffed animal, not a fence.

Prefer one real fixture over five agreeable mocks.
You do not need production data for that fixture.
Store it outside the session if the loop can edit tests.

Myth 4: Hiding tests from the prompt keeps them honest

Prompt folklore says secrecy equals scientific control.
I never pasted the tests, so the model cannot cheat.

The model still writes both sides of the bargain.
It does not need your old tests to invent mirrors.
The filesystem is in the loop with the repo.

If the agent can list tests, it can match names.
Honesty is a process, not a clever prompt trick.
Hold out a check the agent cannot edit later.

I keep a tiny golden file outside the working tree.
Or I run one human test after the loop stops.

Hold-out pattern

mkdir -p /tmp/holdout
cp tests/test_invariants.py /tmp/holdout/
# run the agent against the repo only
pytest /tmp/holdout/test_invariants.py -q -ra

The hold-out file must predate the agent session.
If the agent wrote it, it is not a hold-out.
Simple question: did you type that file yesterday?

If not, it does not count as external proof.
A copied file from this afternoon still shares authorship.
Time travel in git does not create a second mind.

Myth 5: Snapshot files are regression armor

Snapshot tooling feels like a museum alarm for JSON.
The agent added snapshots, so future diffs will save us.

First snapshots can encode the first hallucination cleanly.
A golden file of wrong JSON still fails later.
That failure protects the bug, not the user.

Snapshots freeze a story, so check the story first.
Read the snapshot instead of watching only the matcher.
If you cannot explain one field, delete the file.

find . -name '*.snap' -o -name '*__snapshots__*'
find . -name '*.snap' | head | xargs -I{} head -n 40 {}

Ask three questions out loud before you commit snapshots.

What field is required for the user contract?
What field is incidental formatting or noise?
What field would a wrong model still get right?

If every field is incidental, the snapshot is theater.
Theater can still fail CI and waste a whole afternoon.
That is not armor. That is costume jewelry.

Artifact: a decision table for the PR body

I keep this table next to the pull request template.
It is a workflow, not a benchmark with fake numbers.

Observation	Do not conclude	Conclude instead	Next command
Tests arrived with the impl	The suite is independent	Same author, same hour	`git log --follow -- tests/`
Coverage looks high	Behavior is pinned	Lines ran once	add one failing input
Mocks return nested dicts	The boundary is tested	The fantasy was serialized	`grep -R return_value tests`
Prompt omitted tests	Model cannot cheat	Model can still write both	hold-out file
Snapshots committed	Regressions are blocked	A story was frozen	`head` the snap
Collection count is zero	Nothing broke	Nothing ran	`pytest --collect-only -q`

Copy the table into the pull request body.
Fill the next-command column with real command output.
Output beats vibes, and that is the whole FAQ.

A full audit script you can copy

Label this as a template and adapt paths before running.

#!/usr/bin/env bash
# template: audit_agent_tests.sh
set -euo pipefail

ROOT="${1:-.}"
cd "$ROOT"

echo "== new files in the last day =="
git log --since='1 day ago' --name-only --pretty=format: | sort -u

echo "== test vs src lockstep =="
git diff --stat HEAD~1 || true

echo "== collection count =="
pytest --collect-only -q || true

echo "== skip report =="
pytest -q -rs || true

echo "== tautology scan =="
python3 tautology_scan.py tests || true

echo "== mock payload hunt =="
grep -R -n "return_value\|side_effect" tests || true

echo "== snapshot peek =="
find . -name '*.snap' -print | head

Run it twice, once on a scratch host first.
Run it again on your laptop after that.
Diff the two transcripts before you argue about coverage.

diff -u host-audit.txt laptop-audit.txt

If collection counts differ, stop the merge immediately.
If skip counts differ, stop the merge immediately.
If only the banner differs, you can keep reading.

Where a free model and free server fit

I sometimes need a throwaway box for this audit.
I do not want leftover modules from last week.
I also do not want laptop caches lying to me.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

MonkeyCode offers free model access and a free server option.
I use that pair when I want a clean sample host.
The model proposes test files while the server runs the audit.

Then I still replay locally because the free host is one sample.
It is not CI, production, or a trophy screenshot.
I will not invent quotas, hardware, or duration claims.

Those details change, and this FAQ does not depend on them.
The method is the two transcripts and the hold-out file.
Lose the product and the checklist still stands.

Limitations

This FAQ will not catch a clever assertion.
It will not replace a human invariant test.
It will not prove a distributed system correct.

AST tautology checks miss helper functions on purpose.
They miss equality helpers that compare a value to itself.
They miss generated tests in other languages entirely.

The bash script assumes git and pytest are installed.
It assumes a Python layout many repos do not share.
Do not treat empty scanner output as a blessing.

Who should not use this approach

Do not use this checklist as a hiring score.
Do not use this checklist as a security audit.
Do not use this on repositories you cannot clone.

Do not skip CI because a scratch host was green.
If you ship firmware, write the hold-out yourself.
If you handle money, write the hold-out yourself.

If you have no tests at all, start there first.
An audit of empty tests is empty theater only.
This FAQ assumes you already have a suite to interrogate.

What I want you to remember

The agent can write tests, and that is useful.
Useful is not the same thing as independent proof.
Same author, same hour, same incentive to go green.

Hold out one check the loop cannot edit.
Replay the suite on a second machine every time.
Read the snapshots like they might be wrong today.

They might be wrong, and that is the job.
Would you merge a self-graded exam without a second booklet?
I would not, and your users would not either.

If you already have a scratch host, run the two-transcript diff tonight.

Agent stdout Is Not Your Test Plan

Jordan Huang — Sat, 19 Sep 2026 00:03:07 +0000

Did your agent print All tests passed again today? Did you merge because the log felt complete? That habit is how fake green sneaks in.

I keep hearing four myths about agent logs. They sound like engineering. They are not a test plan.

This is a myth-busting FAQ. You can delete every product name. The checklist still works.

Why logs became a fake CI

Agents write confident sentences. Humans treat sentences like gates. That is a category error, right?

A log line is a claim. A claim is not a process. Who launched the process you trust?

I want a boring gate. I want it copyable. I want it on a scratch machine.

Myth 1: A printed pass means the suite ran

Have you grepped agent.log for the word passed? I have. It lies in three ways.

The model can narrate a run it skipped. It can paste an old snippet. It can praise intent as if it were output.

What evidence remains then? Only a process you started. Only bytes you captured. Only an exit code you stored.

Corrected mental model: stdout is untrusted speech. Speech is not a run.

You need four fields or you have nothing:

the argv you actually invoked
the working directory
the numeric exit code
hashes of stdout and stderr

No fourth field? You are reading a story.

Myth 2: Exit code 0 means the change is finished

Zero is one integer. Which contract did it sign?

A formatter can return 0. A skipped pytest session can return 0. A --dry-run flag can return 0. Did you name the contract first?

Ask a sharper question. Did this exact argv fail? That is all zero answers.

Corrected mental model: exit code is narrow. Completeness lives elsewhere. Completeness needs file lists and git votes.

Myth 3: Complete-looking logs mean a clean tree

Logs sit in a buffer. The repo sits on disk. They drift fast.

Can the agent print a diff it never wrote? Yes. Can it write files it never mentioned? Also yes. Which artifact did you review?

Corrected mental model: git status is a vote. git diff --stat is a vote. The chat transcript is not a vote.

If HEAD is unknown, you are not reviewing a change. You are reviewing a vibe.

Myth 4: One green scratch run is portable

You got a clean VM. The command passed once. Will CI share PATH, lockfiles, and OS packages?

A scratch host is a lab. A lab is not memory. Do not ship the lab as proof.

Corrected mental model: a green run is one sample. Samples need a recipe. Recipes need pins you can paste.

The artifact: refuse speech, record votes

I do not want another dashboard. I want a script that ignores English.

Label this as a runnable example. I am not attaching fake timings or fake pass rates.

Save accept_run.py at the repo root. Make it the only gate after an agent session.

#!/usr/bin/env python3
"""Acceptance harness: treat agent speech as untrusted."""
from __future__ import annotations

import hashlib
import json
import os
import subprocess
import sys
from pathlib import Path

ROOT = Path(os.environ.get("ACCEPT_ROOT", ".")).resolve()
CMD = os.environ.get("ACCEPT_CMD", "python -m pytest -q")
REPORT = Path(os.environ.get("ACCEPT_REPORT", "accept_report.json"))


def run(cmd: str, cwd: Path) -> dict:
    proc = subprocess.run(
        cmd,
        cwd=cwd,
        shell=True,
        text=True,
        capture_output=True,
    )
    return {
        "cmd": cmd,
        "cwd": str(cwd),
        "exit_code": proc.returncode,
        "stdout_sha256": hashlib.sha256(proc.stdout.encode()).hexdigest(),
        "stderr_sha256": hashlib.sha256(proc.stderr.encode()).hexdigest(),
        "stdout_bytes": len(proc.stdout.encode()),
        "stderr_bytes": len(proc.stderr.encode()),
    }


def git(args: list[str]) -> str:
    proc = subprocess.run(
        ["git", *args],
        cwd=ROOT,
        text=True,
        capture_output=True,
        check=False,
    )
    return proc.stdout.strip()


def tree_votes() -> dict:
    diff = git(["diff", "HEAD"])
    return {
        "head": git(["rev-parse", "HEAD"]),
        "status_porcelain": git(["status", "--porcelain"]),
        "diff_stat": git(["diff", "--stat"]),
        "diff_sha256": hashlib.sha256(diff.encode()).hexdigest(),
    }


def decide(run_info: dict, tree: dict) -> list[str]:
    failures: list[str] = []
    if run_info["exit_code"] != 0:
        failures.append("command_failed")
    if not tree["head"]:
        failures.append("not_a_git_repo")
    if tree["status_porcelain"]:
        failures.append("working_tree_dirty")
    if run_info["stdout_bytes"] == 0 and run_info["stderr_bytes"] == 0:
        failures.append("empty_stdio")
    return failures


def main() -> int:
    run_info = run(CMD, ROOT)
    tree = tree_votes()
    failures = decide(run_info, tree)
    report = {
        "run": run_info,
        "tree": tree,
        "failures": failures,
        "accepted": failures == [],
    }
    REPORT.write_text(json.dumps(report, indent=2) + "\n")
    print(json.dumps(report, indent=2))
    return 0 if report["accepted"] else 1


if __name__ == "__main__":
    sys.exit(main())

Wrap it if you want a human summary:

#!/usr/bin/env bash
set -euo pipefail
export ACCEPT_CMD="${ACCEPT_CMD:-python -m pytest -q}"
python3 accept_run.py
python3 - <<'PY'
import json
from pathlib import Path
rep = json.loads(Path("accept_report.json").read_text())
print("accepted:", rep["accepted"])
print("failures:", ", ".join(rep["failures"]) or "(none)")
print("head:", (rep["tree"]["head"] or "n/a")[:12])
print("dirty:", bool(rep["tree"]["status_porcelain"]))
PY

Want dirt to fail closed? Keep working_tree_dirty. Want intentional edits? Drop that flag. Make the policy explicit. Do not hide it in chat.

Wrong gate, for contrast:

# Do not do this.
if grep -qi 'pass' agent.log; then
  echo ship_it
fi

That grep is how myth 1 becomes a merge.

A labeled false-green test also helps teach the team:

# labeled example: pytest can exit 0 while proving nothing
def test_placeholder():
    print("All tests passed")
    assert True

Run pytest. Watch exit code 0. Then run the harness. Notice how little you proved besides zero. That is the lesson.

Decision table: what each signal proves

Signal	Proves	Does not prove
Agent said "passed"	The model emitted text	Any process ran
Exit code 0	That argv did not fail	The suite was complete
stdout hash	Bytes were captured	Bytes are correct
`git status --porcelain`	Disk disagrees with HEAD	The diff is good
`git diff --stat`	Size of the delta	Intent of the delta
Empty stdio	You captured nothing	Silence is success
Green on a scratch host	One sample existed	CI will match

Argue with the table. Stop arguing with the chatbot.

A workflow that keeps the model off the grade

Start from a known HEAD.
Let the agent work on a branch.
Ignore victory speech in the log.
Run ACCEPT_CMD='python -m pytest -q' python3 accept_run.py.
Read failures in accept_report.json.
Only then inspect git diff.

Need a cheap lab for that loop? Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode's free model access and free server option are the only product facts I rely on here. That pair is a scratch lab for rehearsing the harness. The model may draft commands. The harness still grades the run. Do not let the speaker mark its own exam.

How to read the JSON without lying to yourself

Open accept_report.json. Read failures first. Then read exit_code. Then read status_porcelain.

If accepted is true and you never looked at the diff, you skipped a vote. The script cannot review intent. You still can.

If empty_stdio is present, your command printed nothing. Silence is not success. Fix the command. Do not praise the agent.

If working_tree_dirty fires on every good session, change the policy. Record dirt as a flag. Review it. Do not hide policy in a prompt.

Compare two reports only after you pin ACCEPT_CMD. Different argv means different contracts. Hash mismatches then prove nothing useful.

What this will not catch

This harness is small on purpose. It is not a security scanner. It is not a flake oracle.

It will miss:

tests skipped by a bad glob
mocks that hide production failures
secrets echoed into captured logs
hashes that churn because of timestamps
tools installed outside the working tree

Nondeterministic stdout will fight hash comparison. Pin seeds. Pin locale. Or drop cross-host hash equality. Keep exit code plus git votes.

Who should not use this

Skip this if you need a signed audit trail. Skip this if tests cannot run headless. Skip this if a shared host would see real secrets.

Do not copy .env onto a free server. Do not call a lab gate production verification. If CI already stores these four votes, do not duplicate the ritual.

FAQ

Q: Can I parse the agent's English instead?

No. English is not a schema. Emit JSON from your process.

Q: Is a screenshot of the run enough?

Did it show cwd, argv, and exit code? Usually it did not.

Q: May the agent write the harness?

It can draft. You freeze the policy. Then you run it.

Q: What if a dirty tree is expected?

Then dirty is a flag, not a failure. Review the diff as its own vote.

Q: Does a scratch host make the result official?

No. It makes the sample cheap. Official still means CI plus pins.

Limitations, said plainly

I did not benchmark hosts. I did not name models. I did not promise quotas, hardware, or uptime.

A free server can vanish. A free model can be wrong. Your harness must remain copyable.

Remove every product mention and the method should still stand. If it does not, I wrote a brochure. I tried not to.

Try the harness on a throwaway branch. Paste a redacted failures list if you want a second pair of eyes. Bring votes. Leave the victory speech behind.

A Successful Import Is Not a Runtime Receipt

Jordan Huang — Fri, 18 Sep 2026 20:58:48 +0000

Did the agent really install your dependencies today?
I keep seeing the same false comfort in chats.
An import works. Then someone ships. Then CI explodes.

This is not a rant about model quality.
It is a runtime-identity problem instead.
The chat log is not sys.executable and never was.

I will knock down five claims I still hear.
Then you get a receipt you can rerun.
You will see no mystery scores or fake hardware lists.

What this piece refuses to do

I will not quote a latency number I did not measure.
I will not name models I cannot verify here.
I will not treat a transcript as a build log.

The scripts below are a proposed workflow.
Label them that way in your own notes.
Run them on the same host the agent used.

Myth 1: A clean import means the package is pinned

Claim. "It imported httpx. We are fine."

Really? What version loaded into memory?
Which extra was present on disk?
Which interpreter performed the import?

An import only proves one module object exists.
It does not prove a pin or a content hash.
It also does not prove tomorrow's host will match.

Ask one blunt question after every import.
Which file recorded that wheel on disk?
If the answer is the chat, you have theater.

Corrected model. An import is a smoke test.
A lockfile is the actual install contract.
Keep those two jobs in separate boxes.

Myth 2: The host Python is your project Python

Claim. "The loaner box already has Python. Skip the venv."

System Python is only a bootstrap tool.
It is not your project's runtime profile.
sys.prefix and sys.base_prefix tell the truth.

If those two strings match, you are not in a venv.
The agent may still print a friendly "installed."
That print is social talk rather than isolation.

Run this on the machine the agent used:

python3 -c "import sys; print(sys.executable); print(sys.prefix); print(sys.base_prefix)"
command -v python3
type python

Do those lines agree with each other?
If they disagree, the agent guessed the binary.
Guessing the binary is not a packaging step.

Corrected model. The project owns the interpreter.
The host only loans a bootstrap binary.
Create a venv before any install command runs.

Myth 3: "Already satisfied" means the right version

Claim. "pip said already satisfied. Stop nagging."

That phrase only means a name was found.
It may still be the wrong version.
It may be a user-site install instead.
It may be leftover cache from another job.

Name match is not the same as version match.
Version match is not the same as extra match.
Extra match is still not a lockfile.

Check the two sources that actually exist:

python3 -m pip show httpx
python3 -m pip freeze | grep -E '^httpx=='

If show drifts from your lock, you drifted.
If freeze is empty, you queried the wrong Python.
Wrong Python is the usual failure, not "pip is broken."

Corrected model. Satisfied is only a cache message.
It is not a reproducibility statement at all.
Diff pip freeze against the lock every session.

Myth 4: The chat remembered the virtualenv

Claim. "We activated .venv two prompts ago."

Did we? In which process, exactly?
Shell state dies with the process that created it.
A new command often gets a brand new shell.

VIRTUAL_ENV is an environment variable.
It is not model memory in the transcript.
The transcript cannot export it for you.

printf '%s\n' "${VIRTUAL_ENV:-unset}"
.venv/bin/python -c "import sys; print(sys.executable); print(sys.prefix != sys.base_prefix)"

If the first line prints unset, activation died.
If the second binary is missing, no venv exists.
If the boolean is false, you are on host Python again.

Corrected model. Activation is strictly per process.
Re-export it in every command block you run.
Or call .venv/bin/python by absolute path.

Absolute path beats folklore every time.
Folklore is how agents "forget" the venv.
Do you still trust a sentence that says "activated"?

Myth 5: Cheap rehearsal means you can skip the receipt

Claim. "It is only a loaner host. Why record it?"

Because loaner hosts are where drift hides.
A free model can still emit unpinned pip install.
A free server can still pick another default Python.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

I reach for MonkeyCode when a free model and a free server belong in the same rehearsal loop. That pairing is handy for trying the receipt without burning a laptop. It does not pin wheels for you, and it does not turn a chat into CI.

Corrected model. Cost and identity are different questions.
Cheap rehearsal still needs a written receipt.
Free does not mean the runtime goes unrecorded.

The artifact: write a runtime receipt

Here is a small proposed checker.
I am not publishing timings for it.
Copy it. Run it. Keep the JSON.

#!/usr/bin/env python3
"""Write runtime-receipt.json for the current interpreter.

Proposed workflow. Run it yourself before you trust it.
"""
from __future__ import annotations

import hashlib
import json
import os
import subprocess
import sys
from pathlib import Path
from shutil import which


def sha256_if_exists(path: Path) -> str | None:
    if not path.is_file():
        return None
    digest = hashlib.sha256()
    digest.update(path.read_bytes())
    return digest.hexdigest()


def pip_freeze_lines() -> list[str]:
    completed = subprocess.run(
        [sys.executable, "-m", "pip", "freeze"],
        check=False,
        capture_output=True,
        text=True,
    )
    if completed.returncode != 0:
        return [f"ERROR:{completed.returncode}"]
    return [line for line in completed.stdout.splitlines() if line.strip()]


def main() -> int:
    root = Path.cwd()
    receipt = {
        "cwd": str(root.resolve()),
        "sys_executable": sys.executable,
        "sys_version": sys.version.replace("\n", " "),
        "sys_prefix": sys.prefix,
        "base_prefix": sys.base_prefix,
        "in_venv": sys.prefix != sys.base_prefix,
        "which_python3": which("python3"),
        "which_python": which("python"),
        "VIRTUAL_ENV": os.environ.get("VIRTUAL_ENV"),
        "PATH_head": os.environ.get("PATH", "").split(os.pathsep)[:8],
        "requirements.txt": sha256_if_exists(root / "requirements.txt"),
        "requirements.lock": sha256_if_exists(root / "requirements.lock"),
        "pyproject.toml": sha256_if_exists(root / "pyproject.toml"),
        "pip_freeze": pip_freeze_lines(),
    }
    out = root / "runtime-receipt.json"
    out.write_text(json.dumps(receipt, indent=2) + "\n", encoding="utf-8")
    print(out)
    if not receipt["in_venv"]:
        print("warning: not in a virtualenv", file=sys.stderr)
        return 2
    return 0


if __name__ == "__main__":
    raise SystemExit(main())

Need a tiny test beside it?
Yes. Assert identity, not vibes.

# test_runtime_receipt.py
from __future__ import annotations

import json
import sys
from pathlib import Path


def test_receipt_file_exists():
    assert Path("runtime-receipt.json").is_file()


def test_receipt_matches_running_interpreter():
    payload = json.loads(Path("runtime-receipt.json").read_text(encoding="utf-8"))
    assert payload["sys_executable"] == sys.executable
    assert payload["in_venv"] == (sys.prefix != sys.base_prefix)


def test_pin_file_was_recorded():
    payload = json.loads(Path("runtime-receipt.json").read_text(encoding="utf-8"))
    has_req = payload["requirements.txt"] is not None
    has_lock = payload["requirements.lock"] is not None
    has_pyproject = payload["pyproject.toml"] is not None
    assert has_req or has_lock or has_pyproject

What the JSON keys actually mean

sys_executable: the binary that actually ran.
in_venv: prefix inequality, not a vibe check.
which_python3: whatever PATH says right now.
VIRTUAL_ENV: empty means activation did not survive.
pip_freeze: the live view of site-packages.
file hashes: did the pin files change under you?

If sys_executable and which_python3 disagree, stop.
You are debugging the wrong interpreter now.
Everything after that disagreement is noise.

Proposed commands

Do not paste this blindly into production.

python3 -m venv .venv
.venv/bin/python -m pip install -U pip
.venv/bin/python -m pip install -r requirements.txt
.venv/bin/python runtime_receipt.py
.venv/bin/python -m pytest test_runtime_receipt.py -q

What should you commit after that?
Commit runtime_receipt.py and the tests.
Commit the lockfile you actually used.
Treat runtime-receipt.json as session evidence.
Redact paths if they leak a username.

Decision table

Use this before you trust an agent install.

Situation	Write a receipt?	Free model + free server okay?	Ship from that host?
Exploring a script	Yes	Yes, as rehearsal	No
Pinning dependencies	Yes	Yes, if you copy freeze out	No
App tests after pins	Yes	Only if the receipt matches CI	No
Production release	Yes, inside your CI	Not as the release runner	No
Secrets in the tree	Yes, on a machine you own	Avoid shared loaner hosts	No

Read the last column again, slowly.
Rehearsal is not a release channel.
That is the whole table.

Who should not use this

Skip this workflow if you need a full SBOM.
Skip it if you cannot run Python at all.
Skip it if the host must never see your lockfile.
Skip it if legal wants an attested builder.

Also skip the shared free server when secrets sit in .env.
A receipt will not unsay a leaked token.
Keep secrets off loaner disks on purpose.

Limitations I will not hide

pip freeze is incomplete for some extras.
It does not replace uv lock or Poetry.
It does not understand Conda environments.
It does not prove your app tests passed.
It can record absolute paths. Redact them.

The venv check uses prefix inequality.
That is the usual CPython signal today.
It is not a law of every embedding.

I also do not claim the receipt is tamper proof.
It is a snapshot you can diff later.
Diffs beat vibes. That is enough for rehearsal.

FAQ, compressed

Did the import pass? Then you have smoke, not a pin.
Did pip look happy? Then you have a cache message.
Did the chat mention a venv? Then you have a story.
Do you have JSON for sys.executable? Then we can talk about installs.

Stop asking "did it import?" as the finish line.
Start asking "which interpreter, which freeze, which file?"
Three answers. One JSON. Then the debate is real.

The agent can help type the commands.
You still own the receipt after that.
If those two roles blur, you are back in myth land.

If you rehearse on a free server, commit the receipt.
That is the only ask.

DEV Community: Jordan Huang

FAQ: Your Free Server Closed. What Actually Shipped?

Why this keeps happening

Q1. The remote prompt said success. Is that a merge receipt?

Q2. Local HEAD matches origin. Am I finished?

Q3. Untracked output on the free box counts as delivered?

Q4. I ran it on the free server. Will CI match?

Q5. Closing the session archives the evidence?

Artifact: a session receipt you can diff

Decision table: what still counts

A workflow I will actually follow

Limitations

Who should not use this

What I want you to repeat instead

FAQ: Five Myths About the Agent's Remote Shell

What this FAQ is not

Myth 1: Remote Python is "the same Python"

The claim

Why it spreads

What the claim skips

Evidence I actually want

Corrected mental model

Myth 2: which python3 proves the interpreter that ran

The claim

Why it spreads

What the claim skips

Evidence I actually want

Corrected mental model

Myth 3: The prompt said UTC, so the box used UTC

The claim

Why it spreads

What the claim skips

Evidence I actually want

Corrected mental model

Myth 4: Exit code 0 belongs to the files you edited

The claim

Why it spreads

What the claim skips

Evidence I actually want

Corrected mental model

Myth 5: The free box is your audit log

The claim

Why it spreads

What the claim skips

Evidence I actually want

Corrected mental model

The artifact: receipt.sh

How I compare the two files

Decision table

Limitations

Who should not use this approach

The only question I ask now

FAQ: A Free Session Is Not a Build Receipt

What broke in my head

The mental model I want

Myth 1: The complimentary host matches my laptop

Myth 2: Free model output needs no author tag

Myth 3: A compiler on the box makes artifacts portable

Myth 4: No invoice means no paper trail

Myth 5: Copying files back equals a pipeline

PATH lies more than people think

Artifact: a session bill of materials

A test that refuses a naked patch

Diff the BOMs, then approve

Decision table I actually use

What this does not prove

Who should not use this

The questions I ask before merge

Closing

FAQ: Five Myths About Promoting Code Off a Free Agent Box

What this FAQ is not

Where a free box actually helps

The five myths

Myth 1: The transcript file is the whole file

Myth 2: The free box runtime is your runtime

Myth 3: A remote install gave you a trustworthy lockfile

Myth 4: Remote green means skip the local smoke

Myth 5: A workspace link is a pull request

Artifact: a promotion manifest

Limitations

Myth 2: `which python3` proves the interpreter that ran

The artifact: `receipt.sh`