A Review Packet for OSS Patches: Reproduce, Patch, Test, Then Ask a Model

The review half of an OSS patch stalls more contributions than the code does. A reviewer cannot merge what they cannot rerun. A patch that arrives with a captured failing transcript, a size-bounded diff, and a short claim ledger clears triage faster than a larger patch with a better story.

Model review can shorten the loop, but only inside a contract. Free model access is now common enough to be part of a normal workflow. The useful question is not "which model" but "what gets sent, and what is allowed back in".

This article builds a five-step loop around one artifact: the review packet. Nothing below assumes a specific language, and every command is plain shell and git.

What a reviewer actually needs

Three artifacts cover most of the ground:

1. A transcript that reproduces the bug and its exit status.
2. A diff small enough to read in one pass.
3. A claim ledger mapping each changed line to evidence.

Everything else is commentary. A model can help produce commentary, and can flag gaps. It cannot substitute for the transcript.

Step 1: Capture the failing transcript first

Run the failing case before touching the source. Record the command, the output, and the exit code. The exit code matters as much as the message.

mkdir -p transcripts

record() {
  local name=$1; shift
  local log="transcripts/${name}.log"
  printf '$ %s\n' "$*" >"$log"
  "$@" >>"$log" 2>&1
  local code=$?
  printf 'exit=%d\n' "$code" >>"$log"
  return "$code"
}

# Expected to fail before the patch.
record before npx vitest run test/parse.spec.ts || true

sha256sum transcripts/*.log > transcripts/SHA256SUMS
cat transcripts/before.log

Two rules keep this honest. The transcript must come from a clean checkout of the base commit, not a modified working tree. And the failing assertion must name the surface behavior, not an internal detail.

Step 2: Build a size-bounded packet

Large diffs get skimmed. Skimming is where review quality dies. Cap the packet and let the stat summary carry the rest.

#!/usr/bin/env bash
# packet.sh BASE [LINE_BUDGET]
set -euo pipefail

BASE="${1:-origin/main}"
BUDGET="${2:-400}"

if ! git rev-parse --verify "$BASE" >/dev/null 2>&1; then
  echo "unknown base: $BASE" >&2
  exit 2
fi

echo "# diffstat"
git diff --stat "$BASE"...HEAD

echo
echo "# commits"
git log --oneline --no-decorate "$BASE"..HEAD

echo
echo "# patch (first ${BUDGET} lines)"
git diff "$BASE"...HEAD | head -n "$BUDGET"

echo
echo "# transcript hashes"
sha256sum transcripts/*.log 2>/dev/null || echo "none"

Run ./packet.sh origin/main 400 > review-packet.md. If the patch exceeds the budget, split the change. A patch that cannot be capped is usually two patches.

Order files by review risk, not by directory. Public API and schema changes go first. Generated files and lockfiles go last, summarized by diffstat only.

Step 3: Write the claim ledger

Every behavioral claim in the pull request description needs a pointer. This is the artifact that catches the most errors, because it makes vague claims expensive.

Claim	Evidence	Command	Status
Parser accepts trailing commas	transcripts/before.log, transcripts/after.log	npx vitest run test/parse.spec.ts	verified
No change to public exports	git diff --stat, api-extractor run	npm run api:check	verified
Faster on large inputs	not measured	—	removed

That last row is the important one. A claim that cannot be evidenced gets deleted, not softened. Softened claims still waste reviewer time.

Step 4: Add a model as a second reader, inside a contract

A model pass is useful on one narrow task: find places where the diff and the ledger disagree. It is poor at judging intent, and it has no access to the failing case unless the packet includes it.

This is where MonkeyCode fits the workflow. The operator advertises free model access, including a free token allowance (currently stated as ten million tokens) and a free server option. Treat both numbers as time-sensitive and confirm them on the project page before planning around them. The relevant property for this loop is availability: a second reader that costs nothing per pass can be run on every patch instead of only the risky ones.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The contract below is a prompt template. It is not executed output.

You are reviewing a patch packet. Do not rewrite code.

Inputs: diffstat, commit list, capped patch, claim ledger,
transcript hashes (contents omitted).

Return at most 12 items. Each item must have:
- file and hunk
- type: missing-test | ledger-mismatch | unhandled-input | scope-creep
- one sentence on what evidence would settle it

Do not propose refactors. Do not comment on style.
If the packet is insufficient to judge, say so and stop.

Strip transcripts before sending. Send hashes and the failing assertion line instead. Bug reports can contain user data, and a hosted model is a third party.

Step 5: Triage every suggestion the same way

Model output is a list of hypotheses, not findings. Each one gets one of three verdicts, and only one verdict allows a code change.

Verdict	Condition	Action
Accept	A test fails before the change and passes after	Patch, add transcript
Defer	Plausible, but no reproducible input yet	Open an issue, link the packet
Reject	Contradicts the transcript or the ledger	Note it in the PR thread

A suggestion is never applied because it sounds right. It is applied because a new failing test exists for it. That single rule prevents most of the damage a confident model can do to a small patch.

Two failure modes worth naming

Ledger drift. The patch changes during review, the ledger does not. Fix: regenerate the packet and hashes after every rebase, and diff the ledger against the patch in CI.

Scope creep by suggestion. A model flags an adjacent problem, and the patch grows to fix it. Fix: hard-cap the diff budget, and move every accepted out-of-scope item into a follow-up issue in the same session.

Both failures are structural, not model-specific. They appear with human reviewers too, just slower.

Limitations and who should skip this

• The free tier terms can change without notice. Pin the workflow to self-hosted or paid capacity before depending on it in a schedule.
• Model review does not read the whole repository. Cross-file invariants and generated code are out of scope.
• Repositories with an AI-assistance policy, CLA restrictions, or embargoed security fixes need a human-only path. Do not paste embargoed reports into a hosted model.
• The loop costs setup time. For a one-line typo fix, the packet is overhead; ship the patch.
• Contributors who cannot run the test suite locally get no value from steps 1 and 5. Fix the environment first.

The packet itself is the durable part. Models change, and free allowances change with them. A capped diff with a hash-verified transcript and a ledger that admits what was not measured stays reviewable either way.

If the loop is useful, run it once end to end on a small patch before adopting it. The first packet usually reveals which claims were never evidenced to begin with.