DEV Community

Morgan Xu
Morgan Xu

Posted on

File an Evidence Packet Before Agent Code Merges

Agent output is not a merge. Compile success is not a review. Teams that skip a written evidence packet treat vibe output as engineering. This playbook keeps that fiction off the default branch.

The packet is a short wiki card. It records who ran the agent session. It names the human who witnessed the raw diff. Those two names freeze the change for a stated window.

Cheap model access makes the gap louder. A free server lets a whole desk spawn patches overnight. Volume then outruns judgment by morning standup. The loading dock fills with unsigned boxes.

Inventory becomes a story nobody can audit. Reviewers later argue from chat scraps. The merge already landed in production traffic. The packet exists to stop that sequence.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. Some desks run exploratory agents through MonkeyCode free model access and the free server option. That pair is fine for drafts. It does not replace a witness on the diff.

The SOP still holds if another tool wrote the patch. The product is only the cheap draft lane. Promotion still needs human stamps on paper.

Roles live on one wiki page. The duty engineer owns the prompt and the working tree. The witness reads the diff without the original prompt. The freeze clerk holds the merge button for a named window.

A tiny team may share two hats across people. One person cannot wear all three on one change. That is the two-chair rule in this shop.

A cashier should not audit the till just counted. An agent runner should not witness the packet they filed. Distance is the whole point of the second chair.

Handoff is a card, not a hallway story. The duty engineer pastes the packet before stepping away. The witness refuses unsigned packets in the same hour. The freeze clerk ignores pings that skip the card.

Night runs make the handoff sharper. The next shift inherits a file, not a vibe. Analogies help here more than slogans. Think of a sealed evidence bag at a crime desk.

The bag is boring on purpose. Boring bags survive weekends and timezone gaps. Flashy dashboards do not survive a tired Monday. Paste the bag into the team wiki as-is.

# Agent Evidence Packet
Status: draft | witnessed | frozen | merged | rejected
Change-id: <git short hash or worktree name>
Repo: <name>
Branch: <name>
Duty engineer: <handle>
Witness: <handle, different person>
Freeze clerk: <handle>
Freeze window: <start ISO-8601> .. <end ISO-8601>
Draft lane: free-model / free-server / other
Prompt hash: <sha256 of prompt file>
Command log: <path to sanitized shell transcript>
Tests run: <command + exit code>
Risk note: <one sentence, no adjectives>
Witness verdict: accept-with-nits | reject-rewrite | reject-unsafe
Enter fullscreen mode Exit fullscreen mode

The card is the one-page run. Nobody needs a second document for the night. Extra pages become a graveyard of good intentions. One page either exists or the merge waits.

A reproducible check belongs next to the card. The check is a small script, not a platform. It refuses to tag a merge when the packet is missing. Teams can run it locally before any hook debate.

#!/usr/bin/env bash
# evidence_gate.sh — refuse merge without a complete packet.
set -euo pipefail
PACKET="${1:-./EVIDENCE_PACKET.md}"
BRANCH="$(git rev-parse --abbrev-ref HEAD)"
HEAD_SHORT="$(git rev-parse --short HEAD)"

if [[ ! -f "$PACKET" ]]; then
  echo "missing evidence packet: $PACKET" >&2
  exit 2
fi

need() {
  local key="$1"
  grep -E "^${key}:" "$PACKET" | grep -vE '<.*>' | grep -q .
}

for key in Duty\ engineer Witness Freeze\ clerk Freeze\ window Prompt\ hash Tests\ run Witness\ verdict; do
  if ! need "$key"; then
    echo "packet field incomplete: $key" >&2
    exit 3
  fi
done

DUTY="$(sed -n 's/^Duty engineer: //p' "$PACKET" | head -n1)"
WITNESS="$(sed -n 's/^Witness: //p' "$PACKET" | head -n1)"
if [[ "$DUTY" == "$WITNESS" ]]; then
  echo "two-chair rule failed: duty equals witness" >&2
  exit 4
fi

VERDICT="$(sed -n 's/^Witness verdict: //p' "$PACKET" | head -n1)"
case "$VERDICT" in
  accept-with-nits) ;;
  reject-rewrite|reject-unsafe)
    echo "witness rejected the packet" >&2
    exit 5
    ;;
  *)
    echo "unknown witness verdict" >&2
    exit 6
    ;;
esac

echo "evidence ok for $BRANCH@$HEAD_SHORT"
Enter fullscreen mode Exit fullscreen mode

Mark the script executable in the repo. Keep it next to the wiki paste, not in a private gist. A gist dies when the author changes teams. A tracked script survives the next rotation.

chmod +x evidence_gate.sh
./evidence_gate.sh ./EVIDENCE_PACKET.md
Enter fullscreen mode Exit fullscreen mode

A pre-push hook can call the same gate. The hook is optional on day one. The script is not optional if the team claims engineering. Optional theater is how vibe patches sneak through.

#!/usr/bin/env bash
# .git/hooks/pre-push
set -euo pipefail
ROOT="$(git rev-parse --show-toplevel)"
"$ROOT/evidence_gate.sh" "$ROOT/EVIDENCE_PACKET.md"
Enter fullscreen mode Exit fullscreen mode

Prompt hashing keeps the packet honest. The witness should not need the raw prompt. The hash still proves which instruction produced the tree. Teams store the prompt file beside the packet.

sha256sum prompts/change-17.txt > prompts/change-17.sha256
# then paste the hex digest into Prompt hash on the wiki card
Enter fullscreen mode Exit fullscreen mode

Sanitized transcripts matter as much as hashes. Secrets in shell logs turn the packet into a leak. Strip tokens before the witness ever opens the file. The duty engineer owns that redaction step.

# labeled example only — adjust patterns for the local stack
sed -E 's/(token|key|secret|password)=[^ ]+/\1=REDACTED/Ig' \
  raw_session.log > session.sanitized.log
Enter fullscreen mode Exit fullscreen mode

Tests in the packet are commands, not feelings. "Looks good" is not a test line. The duty engineer pastes the exact command and the exit code. The witness may rerun that command on a clean tree.

npm test -- --runInBand
echo "exit:$?" 
Enter fullscreen mode Exit fullscreen mode

If the stack has no tests yet, the packet says so in one sentence. Missing tests are a risk note, not a hidden skip. The freeze clerk then lengthens the window. Silence is not an acceptable risk note.

Decision paths stay on the same page. They read as a tiny matrix in a fence. The matrix is an artifact, not a slide. People follow it at 1 a.m. without a meeting.

verdict            freeze window         merge allowed
accept-with-nits   2 working hours       after nits land
reject-rewrite     none                  no, new packet required
reject-unsafe      none                  no, duty files incident note
Enter fullscreen mode Exit fullscreen mode

Accept-with-nits is not a shrug. Nits become a second tiny commit with the same packet id. The witness checks that the nits match the note. Scope creep restarts the whole card.

Reject-rewrite is the common honest outcome. The draft lane did its job by being cheap. The default branch did its job by staying closed. Pride has no field on the card.

Reject-unsafe is rarer and louder. The freeze clerk pages the incident channel, not the author chat. The duty engineer stops further agent loops on that tree. Loops without a new packet are just more fog.

A worked example keeps the SOP concrete. Suppose a night agent patches a rate limiter. The duty engineer files the card before logging off. The morning witness reads the diff on a cold laptop.

The witness finds a default that fails open. Verdict becomes reject-unsafe in one line. The freeze clerk never opens the merge queue. The free draft lane still earned its keep by failing early.

Another example goes the other way. The agent adds a retry around a flaky client. Tests pass twice on a clean clone. The witness accepts with nits on log wording.

The nits land in a follow-up commit under the same change-id. The freeze window covers lunch, not a week. The merge then looks like ordinary engineering. That is the point of the stamps.

Limitations are part of the playbook. The script does not prove the code is correct. It only proves a second human signed a card. Liars can still type two names.

The hash does not prove the prompt was wise. It only binds a file to a digest. A poisoned prompt still hashes cleanly. Wisdom stays a human job.

The freeze window does not replace load tests. It only slows a rushed merge after a cheap draft. High-stakes systems still need their real gates. This SOP is a receiving clerk, not a lab.

Who should not use this approach is equally plain. Solo hobby repos with one author gain little. The two-chair rule cannot clone a person. Those repos should keep ordinary personal review habits.

Regulated codebases with existing change-advisory boards should not swap boards for this card. The packet can feed those boards. It should not pretend to be one. Compliance theater is worse than no packet.

Teams that cannot redact transcripts should not file them. A packet full of secrets is a worse incident. They can store hashes and test commands only. Empty secret fields beat leaked ones.

Desks that treat free draft compute as production compute should stop first. A free server is a sandbox with a clerk. It is not a silent path to main. The airlock is social, then technical.

The trend this week is loud on engineering versus vibe. This shop does not need that debate as a slogan. It needs a bag with names and a clock. Names and clocks still work when models change.

Paste the card. Run the gate. Keep the second chair empty of the first. That is the whole night run, and it fits on one wiki page.

Desks already drafting on the free model lane can drop the packet template into the wiki before the next shared night run.

Top comments (0)