DEV Community: Taylor Wang

A Review Packet for OSS Patches: Reproduce, Patch, Test, Then Ask a Model

Taylor Wang — Sat, 17 Oct 2026 06:31:10 +0000

The review half of an OSS patch stalls more contributions than the code does. A reviewer cannot merge what they cannot rerun. A patch that arrives with a captured failing transcript, a size-bounded diff, and a short claim ledger clears triage faster than a larger patch with a better story.

Model review can shorten the loop, but only inside a contract. Free model access is now common enough to be part of a normal workflow. The useful question is not "which model" but "what gets sent, and what is allowed back in".

This article builds a five-step loop around one artifact: the review packet. Nothing below assumes a specific language, and every command is plain shell and git.

What a reviewer actually needs

Three artifacts cover most of the ground:

A transcript that reproduces the bug and its exit status.
A diff small enough to read in one pass.
A claim ledger mapping each changed line to evidence.

Everything else is commentary. A model can help produce commentary, and can flag gaps. It cannot substitute for the transcript.

Step 1: Capture the failing transcript first

Run the failing case before touching the source. Record the command, the output, and the exit code. The exit code matters as much as the message.

mkdir -p transcripts

record() {
  local name=$1; shift
  local log="transcripts/${name}.log"
  printf '$ %s\n' "$*" >"$log"
  "$@" >>"$log" 2>&1
  local code=$?
  printf 'exit=%d\n' "$code" >>"$log"
  return "$code"
}

# Expected to fail before the patch.
record before npx vitest run test/parse.spec.ts || true

sha256sum transcripts/*.log > transcripts/SHA256SUMS
cat transcripts/before.log

Two rules keep this honest. The transcript must come from a clean checkout of the base commit, not a modified working tree. And the failing assertion must name the surface behavior, not an internal detail.

Step 2: Build a size-bounded packet

Large diffs get skimmed. Skimming is where review quality dies. Cap the packet and let the stat summary carry the rest.

#!/usr/bin/env bash
# packet.sh BASE [LINE_BUDGET]
set -euo pipefail

BASE="${1:-origin/main}"
BUDGET="${2:-400}"

if ! git rev-parse --verify "$BASE" >/dev/null 2>&1; then
  echo "unknown base: $BASE" >&2
  exit 2
fi

echo "# diffstat"
git diff --stat "$BASE"...HEAD

echo
echo "# commits"
git log --oneline --no-decorate "$BASE"..HEAD

echo
echo "# patch (first ${BUDGET} lines)"
git diff "$BASE"...HEAD | head -n "$BUDGET"

echo
echo "# transcript hashes"
sha256sum transcripts/*.log 2>/dev/null || echo "none"

Run ./packet.sh origin/main 400 > review-packet.md. If the patch exceeds the budget, split the change. A patch that cannot be capped is usually two patches.

Order files by review risk, not by directory. Public API and schema changes go first. Generated files and lockfiles go last, summarized by diffstat only.

Step 3: Write the claim ledger

Every behavioral claim in the pull request description needs a pointer. This is the artifact that catches the most errors, because it makes vague claims expensive.

Claim	Evidence	Command	Status
Parser accepts trailing commas	`transcripts/before.log`, `transcripts/after.log`	`npx vitest run test/parse.spec.ts`	verified
No change to public exports	`git diff --stat`, `api-extractor` run	`npm run api:check`	verified
Faster on large inputs	not measured	—	removed

That last row is the important one. A claim that cannot be evidenced gets deleted, not softened. Softened claims still waste reviewer time.

Step 4: Add a model as a second reader, inside a contract

A model pass is useful on one narrow task: find places where the diff and the ledger disagree. It is poor at judging intent, and it has no access to the failing case unless the packet includes it.

This is where MonkeyCode fits the workflow. The operator advertises free model access, including a free token allowance (currently stated as ten million tokens) and a free server option. Treat both numbers as time-sensitive and confirm them on the project page before planning around them. The relevant property for this loop is availability: a second reader that costs nothing per pass can be run on every patch instead of only the risky ones.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The contract below is a prompt template. It is not executed output.

You are reviewing a patch packet. Do not rewrite code.

Inputs: diffstat, commit list, capped patch, claim ledger,
transcript hashes (contents omitted).

Return at most 12 items. Each item must have:
- file and hunk
- type: missing-test | ledger-mismatch | unhandled-input | scope-creep
- one sentence on what evidence would settle it

Do not propose refactors. Do not comment on style.
If the packet is insufficient to judge, say so and stop.

Strip transcripts before sending. Send hashes and the failing assertion line instead. Bug reports can contain user data, and a hosted model is a third party.

Step 5: Triage every suggestion the same way

Model output is a list of hypotheses, not findings. Each one gets one of three verdicts, and only one verdict allows a code change.

Verdict	Condition	Action
Accept	A test fails before the change and passes after	Patch, add transcript
Defer	Plausible, but no reproducible input yet	Open an issue, link the packet
Reject	Contradicts the transcript or the ledger	Note it in the PR thread

A suggestion is never applied because it sounds right. It is applied because a new failing test exists for it. That single rule prevents most of the damage a confident model can do to a small patch.

Two failure modes worth naming

Ledger drift. The patch changes during review, the ledger does not. Fix: regenerate the packet and hashes after every rebase, and diff the ledger against the patch in CI.

Scope creep by suggestion. A model flags an adjacent problem, and the patch grows to fix it. Fix: hard-cap the diff budget, and move every accepted out-of-scope item into a follow-up issue in the same session.

Both failures are structural, not model-specific. They appear with human reviewers too, just slower.

Limitations and who should skip this

The free tier terms can change without notice. Pin the workflow to self-hosted or paid capacity before depending on it in a schedule.
Model review does not read the whole repository. Cross-file invariants and generated code are out of scope.
Repositories with an AI-assistance policy, CLA restrictions, or embargoed security fixes need a human-only path. Do not paste embargoed reports into a hosted model.
The loop costs setup time. For a one-line typo fix, the packet is overhead; ship the patch.
Contributors who cannot run the test suite locally get no value from steps 1 and 5. Fix the environment first.

The packet itself is the durable part. Models change, and free allowances change with them. A capped diff with a hash-verified transcript and a ledger that admits what was not measured stays reviewable either way.

If the loop is useful, run it once end to end on a small patch before adopting it. The first packet usually reveals which claims were never evidenced to begin with.

Test-First OSS Patches: Write the Failing Fixture Before You Touch the Code

Taylor Wang — Fri, 16 Oct 2026 05:33:58 +0000

Test-First OSS Patches: Write the Failing Fixture Before You Touch the Code

A bug report is not reproducible until a test fails on your machine. Before that moment it is just a story. This workflow converts the story into a fixture, a captured failure signature, and a two-commit patch. The first commit holds only the test. The second holds only the fix.

The loop takes about twenty minutes on a small parser bug. It saves maintainer attention, which is the scarcest resource in open source. It also makes review faster, because the reviewer sees the broken behavior encoded as code.

Why "it does not work" is not a bug report

Three patterns consume most open-source issue threads. The reporter describes an outcome without an input. The maintainer reproduces on a different version and sees nothing. The patch lands without a test, so the bug returns in six months.

A failing test removes all three. It pins the input, the version, and the expected output in one file. The test then guards the fix forever, at zero recurring cost.

The fixture contract

A usable fixture satisfies four properties. It is deterministic, so repeated runs give the same result. It is offline, so it needs no network or credentials. It is ordered, so it never depends on filesystem enumeration. It is minimal, so one assertion explains the whole failure.

If a candidate fixture breaks any of these properties, it is a flaky test in disguise. Fix the fixture before touching product code.

Step 1: freeze the reporter's input

Create a branch and a fixture directory. Copy the smallest input that still triggers the bug.

git switch -c repro/4821-trailing-comma
mkdir -p tests/regression/fixtures/issue_4821

Store the raw input exactly as the reporter supplied it. Strip secrets, absolute paths, and hostnames. Add the expected output beside it, taken from documented behavior rather than from current output.

The expected value matters. Copying current output encodes the bug as correct behavior. Derive the expectation from the spec, the docs, or a sibling parser that behaves properly.

Step 2: assert the wrong behavior on purpose

The test module below is illustrative. Adjust imports and names to your package. It was not executed in this article.

# tests/regression/test_issue_4821_trailing_comma.py
from pathlib import Path

from yourpackage.config import parse_record  # adjust import

FIXTURES = Path(__file__).parent / "fixtures" / "issue_4821"


def read(name: str) -> str:
    return (FIXTURES / name).read_text(encoding="utf-8")


def test_trailing_comma_keeps_previous_field():
    record = parse_record(read("input.conf"))
    assert record.as_text() == read("expected.txt")

One test, one assertion, one fixture pair. Name the file after the issue number so the link between code and thread stays obvious.

Run it and expect a failure:

python -m pytest -q tests/regression/test_issue_4821_trailing_comma.py \
  2>&1 | tee /tmp/4821.before.txt

If this test passes, you have not reproduced the bug. Stop. Do not write a patch against an unproven theory.

Step 3: capture the failure signature

Extract the assertion lines into a small file. This becomes the before-image of the fix.

grep -E '^(E |FAILED|AssertionError)' /tmp/4821.before.txt | head -20

The signature should mention the exact field and value that differ. A signature of "assert False" tells you the fixture is too coarse. Split it until the diff names one behavior.

Step 4: patch the smallest surface

Change the least code that makes the assertion true. Resist refactoring nearby functions in the same branch. Reviewers approve small diffs faster, and small diffs revert cleanly.

Most parser bugs need a guard clause or an index fix. If the patch grows past roughly fifteen lines, the fixture may be testing two bugs. Split it into two issues and two branches.

Step 5: verify with two commands, then commit in two parts

Run the targeted test first, then the full suite.

python -m pytest -q tests/regression/test_issue_4821_trailing_comma.py \
  2>&1 | tee /tmp/4821.after.txt
python -m pytest -q 2>&1 | tail -5

Diff the failure signatures to prove the assertion lines disappeared:

diff <(sed -n '/^E /p' /tmp/4821.before.txt) \
     <(sed -n '/^E /p' /tmp/4821.after.txt)

Then commit the test and the fix separately:

git add tests/regression
git commit -m "test: reproduce #4821 with trailing comma fixture"
git add src/
git commit -m "fix: ignore trailing comma in config parser"

Two commits let a reviewer verify the test fails before the fix and passes after it. That is the whole argument, expressed as git history.

Step 6: use model assistance where it cannot lie

Drafting fixtures is repetitive work, which is where model assistance helps most. Useful prompts ask for edge cases the reporter did not mention: empty values, duplicate keys, unicode boundaries, very long lines. A model can also summarize a long issue thread into candidate inputs.

Verification stays human. Every proposed fixture is confirmed by running the pytest command above and reading the output. A model cannot observe your failure signature, so it cannot confirm reproduction.

MonkeyCode provides free model access and a free server option, per the operator. That matters for this loop because the workflow needs only a shell, git, and Python. Free model access lowers the cost of generating candidate edge-case fixtures. The free server option lets the reproduce-patch-verify cycle run on a remote machine, keeping a local checkout clean while the fixture set grows. Free-tier limits and server terms change over time, so check the current product page instead of trusting a number in a blog post. Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Decision table: patch, ask, or close

Report state	Action	Reason
No version string or no input	Ask for both	Nothing is reproducible yet
Fixture fails on main branch	Write the patch	Reproduction is proven
Fails only on the reporter's machine	Request environment dump	Likely platform-specific
Test passes after the patch, suite green	Open the pull request	Evidence is complete
Cannot reproduce, suite green	Comment with the fixture, keep the issue open	Guessing produces bad fixes

Post the fixture in the thread even when reproduction fails. A maintainer on another platform can run it in one command.

Limitations and who should skip this

Fixtures do not fit every bug class. Flaky concurrency bugs, hardware-dependent crashes, and performance regressions need repeated runs and benchmark harnesses instead. Private data cannot be committed as a fixture, so a redacted or synthetic input is required.

Security fixes need extra care. A public failing test can describe a vulnerability before a patched release exists. In that case keep the reproduction private and coordinate disclosure with the maintainers first.

Teams with a strict contribution guide should follow it. Some projects require tests in a specific directory or forbid new fixture folders. Read CONTRIBUTING.md before creating structure.

Closing

Write the fixture, watch it fail, fix the smallest surface, watch it pass. That order is the entire method, and it scales from a two-line parser bug to a protocol change. If a reader wants a second opinion on a candidate fixture, the free model access in MonkeyCode is one reasonable place to look for edge cases before running them locally.

Exit Codes First: A Reproduce-Patch-Verify Loop for OSS Bug Triage

Taylor Wang — Thu, 15 Oct 2026 11:34:33 +0000

Exit Codes First: A Reproduce-Patch-Verify Loop for OSS Bug Triage

Most open-source issues do not die in code. They die in triage. A reporter says it crashes on their machine. Nobody reproduces it. The thread goes quiet for six weeks.

Three states fix most of that. They are R (reproduce), P (patch), and V (verify). Each state writes an exit code into a ledger. Nothing advances on opinion.

The loop takes about twenty minutes per issue. It produces artifacts a reviewer can argue with. It also survives maintainer turnover.

1. Why issue queues stall

Unreproducible reports dominate most backlogs. Volume is not the only problem. A recent DEV trend post counted thousands of open-source AI projects landing in a single weekend (https://dev.to/dailycontext/one-weekend-3000-open-source-ai-projects-3g65). Supply grows faster than triage capacity.

Reviewers then guess. A guessed fix becomes a guessed test. The patch merges, and the same bug returns three releases later.

The loop replaces guessing with three checkable facts. The bug fails at commit X. The patch is minimal. The same check passes after the patch.

2. Step 1: Pin the reporter's commit

Never triage on main. Check out the commit named in the report. Branch drift hides half of all reproduction failures.

git clone --filter=blob:none https://github.com/<org>/<repo>.git work
cd work
git checkout <reporter-sha>
git rev-parse --short HEAD   # record this value

Build and install dependencies at that commit. If the build fails, the issue is a build report, not a bug report. Say so in the thread.

3. Step 2: Record states, not opinions

One script drives the loop. R expects a non-zero exit. V expects zero. P is a diff, so it has no exit code.

#!/usr/bin/env bash
# triage.sh <repo-dir> <check-command>
set -uo pipefail

REPO="${1:?usage: triage.sh <repo-dir> <check-command>}"
CHECK="${2:?usage: triage.sh <repo-dir> <check-command>}"
LEDGER="triage-ledger.ndjson"

cd "$REPO" || exit 2
commit=$(git rev-parse --short HEAD)
: > "$LEDGER"

record () {
  printf '{"commit":"%s","state":"%s","want":%s,"exit":%s,"seconds":%s}\n' \
    "$commit" "$1" "$2" "$3" "$4" >> "$LEDGER"
  echo "state=$1 exit=$3 want=$2"
}

run_state () {
  local state="$1" want="$2" t0 t1 code
  t0=$(date +%s)
  bash -c "$CHECK" >"/tmp/triage-$state.log" 2>&1
  code=$?
  t1=$(date +%s)
  record "$state" "$want" "$code" "$((t1 - t0))"
  [ "$code" -eq "$want" ]
}

run_state R 1 || { echo "R did not fail; not reproducible here"; exit 1; }

git apply --check patch.diff && git apply patch.diff

run_state V 0 || { echo "V did not pass; patch is incomplete"; exit 1; }

Run it with a narrow check first, never the full suite.

bash triage.sh ~/work/parser \
  "python -m pytest tests/test_offset.py::test_zero_width -q"

Output is two lines, plus one NDJSON row per state. Paste both into the issue.

4. Step 3: Gate the ledger before review

A ledger nobody reads is a diary. Add a verifier to CI and to the local script chain. Missing states must fail the build.

# verify_ledger.py
import json
import sys

WANT = {"R": 1, "V": 0}
rows = [json.loads(l) for l in open("triage-ledger.ndjson") if l.strip()]
seen = {row["state"]: row for row in rows}

for state, want in WANT.items():
    row = seen.get(state)
    if row is None:
        sys.exit(f"missing state {state}")
    if row["exit"] != want:
        sys.exit(f"{state}: exit {row['exit']}, want {want}")

print("ledger ok:", {s: seen[s]["exit"] for s in WANT})

One useful second check: reverse the patch and rerun R. If the failure does not return, the test never covered the bug.

5. Step 4: Split review work by cost

Not every review question deserves a human. Sort them by cost of a wrong answer.

Review question	Cheapest reliable signal	Draft model review	Maintainer sign-off
Does it reproduce?	exit code R	no	no
Is the diff minimal?	`git diff --stat` line count	yes	only if contested
Does the test cover the fix?	revert-and-rerun R	yes	no
Is the API change acceptable?	none	draft only	yes
Security, licensing, data handling	none	no	yes

The rule is simple. Machine checks own the left column. Humans own the right one.

6. Where free model access fits

Rows two and three are cheap model work. A wrong answer there wastes a comment, not a release.

MonkeyCode offers free model access and a free server option, per the project operator. That helps contributors without a paid API budget run the reviewer step. Point the model at the diff and the ledger, not at the whole repository.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The allowance and server terms can change. Check the project page before planning a workflow around them. Nothing in this article was benchmarked against a paid tier.

7. Limitations and who should skip this

The ledger proves environment, not correctness. A green V on a weak test is still a weak test.

Flaky checks poison the loop. Fix nondeterminism before trusting R or V. Network-dependent tests break the exit-code contract by design.

Model reviewers tend to rubber-stamp a diff that compiles. Keep the exit-code gate in front of them. Treat their output as a draft, never as approval.

Skip this loop if failures need private data or specific hardware. Skip it if CI minutes are the bottleneck and two extra runs per issue are unaffordable. Skip it for one-off scripts nobody depends on.

8. Apply it to one stalled issue

Pick a single issue this week. Pin the commit. Write the narrowest failing check you can. Record R, apply a small patch, record V.

If the reviewer step is the bottleneck, MonkeyCode's free model access and free server option are a reasonable place to start. Keep the exit codes as the source of truth.

Two exit codes beat a forty-comment thread of opinions.

Set an Eighty-Point Gate Before an OSS Model Review

Taylor Wang — Sun, 11 Oct 2026 22:16:10 +0000

An OSS contribution should clear a local score before model review. The score freezes reproduction, patch scope, tests, and rollback. A model may comment later, but it must not set the bar.

This note gives a proposed gate for ordinary bug-fix pull requests. The checker below is a template, not a recorded benchmark. No runtime metric in this draft comes from a live maintainer host.

The failure this gate stops

A loose patch invite produces comments about tone and naming. The maintainer still lacks a failing command and a bounded diff. Review time goes to prose, not to the defect.

A model amplifies that failure when the prompt is vague. It will happily discuss style while the bug stays open. The gate blocks that prompt until local evidence exists.

What the scorecard stores

The packet lives in one directory beside the clone. Five notes carry fixed weights that sum to one hundred. A missing note scores zero for that weight.

repro.md holds the failing command and the observed result. patch.diff holds the candidate change and nothing beyond it. tests.md holds the fail command and the later pass command.

scope.txt holds git diff stat output plus a path allowlist. rollback.md holds the branch name and the drop command. The weights are twenty, forty, twenty, ten, and ten.

Forty points sit on the diff because scope creep is common. Twenty points sit on reproduction because hearsay is common. The remaining forty points cover tests, scope, and rollback.

Step 1. Freeze the reproduction

The contributor creates a clean directory and a shallow clone. The branch name carries the issue id and the word repro. Work does not start on the default branch.

The clone URL below is a placeholder, not a real project. The contributor replaces it with the target repository. A depth of one is enough for a small bug note.

mkdir -p oss-scorecard
cd oss-scorecard
git clone --depth 1 https://example.com/org/project.git src
cd src
git switch -c issue-1842-repro

The contributor runs the smallest command that shows the bug. The note stores the command, the exit code, and key lines. Tokens, emails, and internal host names are redacted first.

set -o pipefail
./scripts/repro.sh > ../repro.md 2>&1
printf 'exit:%s\n' $? >> ../repro.md

A needed fixture is committed on the throwaway branch. A private database dump never enters the note. A maintainer should rerun the command on a normal laptop.

Step 2. Bound the patch

The fix starts only after the failing command is saved. The diff stays inside the path allowlist in scope.txt. Extra refactors move to a separate pull request.

The contributor writes scope.txt from git diff stat. An allowlist of paths is appended under that stat. The full diff is saved as patch.diff in the packet.

git diff --stat > ../scope.txt
printf '\nallow:\n' >> ../scope.txt
git diff --name-only >> ../scope.txt
git diff > ../patch.diff

A lockfile edit without a stated reason fails review. A hand edit of a generated file also fails review. Those two checks stay human after the score passes.

Step 3. Attach the test evidence

tests.md lists the fail command and the pass command. The same script name should appear on both lines. A renamed test with no behavior change does not count.

The note records the expected exit code for each run. One line names the assertion that holds after the patch. Timing claims stay out unless a local clock was recorded.

cat > ../tests.md << 'EOF'
fail: ./scripts/repro.sh
pass: ./scripts/repro.sh
expect_fail_exit: 1
expect_pass_exit: 0
assertion: the parser rejects an empty name field
EOF

Step 4. Write the rollback note

rollback.md tells a reviewer how to abandon the branch. It includes the branch name and any files the patch adds. It omits credentials for any shared staging host.

cat > ../rollback.md << 'EOF'
branch: issue-1842-repro
drop: git switch main && git branch -D issue-1842-repro
added_files: none
data_loss: none expected
EOF

Step 5. Run the local checker

The checker looks at presence, weight, and rough length. It does not execute the project tests by itself. A passing score only permits a later comment request.

The script below is proposed code for a local laptop. It has not been certified against a specific language runtime. The contributor should read it before trusting the exit code.

#!/usr/bin/env python3
'''Proposed local gate for an unexecuted template.'''

import pathlib
import sys

REQUIRED = {
    'repro.md': 20,
    'patch.diff': 40,
    'tests.md': 20,
    'scope.txt': 10,
    'rollback.md': 10,
}

MIN_CHARS = 40
HOLD_BELOW = 80

def score_packet(root: pathlib.Path) -> int:
    total = 0
    for name, weight in REQUIRED.items():
        path = root / name
        if not path.is_file():
            print('missing ' + name)
            continue
        text = path.read_text(encoding='utf-8').strip()
        if len(text) < MIN_CHARS:
            print('thin ' + name)
            continue
        total += weight
    return total

def main() -> int:
    root = pathlib.Path(sys.argv[1] if len(sys.argv) > 1 else '.')
    total = score_packet(root)
    print('score ' + str(total))
    if total < HOLD_BELOW:
        print('hold the model prompt')
        return 1
    print('scorecard is ready for a model pass')
    return 0

if __name__ == '__main__':
    raise SystemExit(main())

python3 gate_check.py ..

A score under eighty stops the workflow at once. The contributor repairs the thin note and reruns the checker. The threshold is not lowered merely to ship faster.

A missing ten-point file can still clear eighty. The illustrative output below shows that scoring hole. The contributor should not treat ninety as full evidence.

missing rollback.md
score 90
scorecard is ready for a model pass

Step 6. Send the card to a model

MonkeyCode enters only after the local gate has passed. Disclosure: This article was prepared as part of MonkeyCode's product outreach. The product is a review aid here, not the acceptance authority.

The operator states that free model access is available. The operator also states that a free server option exists. This draft does not treat either offer as permanent.

Current limits belong on the product page, not in this note. No token quota, model name, or machine size is assumed. Those details change, and stale numbers would mislead a team.

The free server fits when the maintainer host must stay clean. The upload should contain the five notes, not the whole clone. A smaller packet is easier to redact and easier to audit.

The model task should stay narrow on purpose. It scores each file against the written weights. It lists gaps, contradictions, and risky file paths.

It does not rewrite the patch as the first action. The prompt below is a template for that narrow task. The contributor pastes the five notes under that prompt.

Review the five scorecard files only.
Score each file from 0 to its weight.
Cite a line for every deducted point.
List contradictions between repro.md and tests.md.
Do not propose a new feature.
Stop if a secret or credential appears.

Step 7. Sort every model comment

A fixed decision table keeps the human owner visible. Each comment maps to exactly one local action. The model does not merge, rebase, or close the issue.

Comment type	Local action	Model role
Missing command	Return to reproduction	Point at the empty field
Path outside allowlist	Split the diff	Name the extra path
Test rename only	Demand a behavior check	Flag the mismatch
Speculative refactor	Defer to another issue	Must not expand scope
Possible secret	Stop and redact	Must not repeat the secret
Logic doubt with a citation	Reproduce on the laptop	May suggest a check command

The contributor applies the table before editing the commit text. A comment is accepted only after a local command confirms it. Comments that cite files outside the packet are discarded.

Limits of the gate

File length is only a weak proxy for truth. A long incorrect repro.md can still earn twenty points. The reviewer reruns the command instead of trusting the score.

The checker ignores comment quality and project license terms. Some projects forbid sending code to an external host. That policy is read before any free server upload.

Free model access can rate-limit, queue, or change without notice. A sprint plan should not depend on an unchecked offer. The same packet keeps a fully local review path.

This gate is a poor fit for urgent security patches. Those diffs may contain exploit detail that should stay private. A trusted maintainer channel replaces a general model host.

Generated files and binary assets need extra local rules. The sample checker does not parse diff headers. The team extends it before use on a release branch.

Who should skip this workflow

Teams skip it when the bug is not reproduced locally. Teams skip it when the license blocks external model review. Teams skip it when they need measured model accuracy.

Auth changes and credential grants also sit outside this gate. A scorecard cannot prove that an auth change is safe. That proof belongs in a dedicated security review.

New contributors may still use the file list as a writing aid. They ask a human maintainer which files may be shared. The model pass is optional, not a requirement to contribute.

Keep the merge with the maintainer

The useful order is reproduce, patch, test, then model review. The eighty-point checker enforces that order on disk. The human maintainer still owns the merge decision.

A redacted scorecard is the only packet that should leave the laptop. A contributor checks MonkeyCode's current free model and server offer.

Hold OSS Model Notes Behind a Local Review Contract

Taylor Wang — Sat, 10 Oct 2026 18:53:40 +0000

An OSS model note stays non-binding until a local contract passes. The contract locks one command, one diff scope, and two exit codes. A free model may comment only after that gate.

Why unbound notes fail

A long model reply can sound like a finished review. It still lacks a rerun of the failing command. Maintainers then merge model confidence instead of local evidence.

This workflow splits the work into three lanes. Local reproduction owns the claim about the bug. A disposable rerun and a model read stay secondary.

Contract fields that matter

Keep the contract in review-contract.json next to the patch. Record only facts that a stranger can check. Leave coding style opinions out of that file.

{
  "repo": "example/widget",
  "base": "origin/main",
  "command": "python -m unittest tests.test_widget",
  "expect_before": 1,
  "expect_after": 0,
  "allowed_paths": ["widget/parse.py", "tests/test_widget.py"],
  "model_role": "non_binding_reader"
}

The expect_before field stores the exit before the patch. The expect_after field stores the exit after the patch. Both numbers must come from commands, not from chat.

The widget paths and example.invalid host are placeholders, not a real project. Swap in the public repository before a real review run.

Three lanes, one owner

Lane	Where it runs	What it proves	Can it approve the patch
Reproduce	Local worktree	Command exit and log	No, it only supplies evidence
Rerun	Disposable free server	Same command, clean tree	No, it only confirms the exit
Read	Free model session	Risks outside the command	No, comments stay non-binding

Use the free server as a clean rerun host. Use free model access as a narrow second reader.

The operator presents MonkeyCode as an open-source project. Disclosure: This article was prepared as part of MonkeyCode's product outreach. Those claims cover the rerun lane and the read lane only.

This draft does not name models, quotas, hardware, or duration. Cite a current primary source before publishing any of those figures.

Step 1. Freeze one failing command

Choose a command that fails on the base commit today. Write it into the contract before any edit. Ban prompts, watches, and commands that never exit.

git fetch origin
git switch --detach origin/main
python -m unittest tests.test_widget
echo "before=$?"

Copy the printed exit code into the expect_before field. A zero means this command does not show the bug. Replace that command before writing any code patch.

Run the snippet without errexit, or the shell stops early. A stopped shell never prints the before marker. The missing marker is a failed reproduction, not a pass.

Step 2. Limit the patch to named paths

Open a detached worktree for the patch experiment. Check the diff, then apply it only if the path list matches. A wider diff makes the written contract false.

git worktree add --detach ../review-wt origin/main
git -C ../review-wt apply --check patch.diff
git -C ../review-wt apply patch.diff
git -C ../review-wt diff --name-only > changed.txt

Compare changed.txt with allowed_paths before the next lane. Extra paths mean the contract is already stale. Update the list or remove the extra file.

Step 3. Capture the local after log

Run the locked command inside the review worktree. Save the combined log and the exit marker. Never trim a failure just to save space.

cd ../review-wt
python -m unittest tests.test_widget > after.log 2>&1
echo "after=$?" | tee -a after.log

Require the after marker when expect_after equals zero. A different marker blocks every later review lane. The local log stays the primary review evidence.

Step 4. Confirm on a disposable server

Push nothing private into the disposable server session. Clone the public base, then apply the same patch file. Run the same command and record the server exit beside the local one.

Reuse the same patch.diff on the server lane. Do not rebuild the patch from memory on that host.

# Proposal only. Replace the placeholder before any real run.
git clone --depth 1 https://example.invalid/widget.git review-srv
cd review-srv
git apply --check ../patch.diff
git apply ../patch.diff
python -m unittest tests.test_widget
echo "server=$?"

Label the server block as an unexecuted proposal. A free server may lack compilers, tokens, or long job time. Record that gap and keep the local log as authority.

Step 5. Build a redacted read packet

Copy four items into a folder named packet. Include the contract, the path list, both exit lines, and a scoped diff. Strip secrets, env files, and absolute home paths first.

mkdir -p packet
cp review-contract.json changed.txt packet/
git -C ../review-wt diff -- widget/parse.py tests/test_widget.py > packet/scope.diff
python3 -c 'from pathlib import Path; text = Path("after.log").read_text(encoding="utf-8", errors="replace"); lines = [ln for ln in text.splitlines() if ln.startswith("after=")]; Path("packet/exits.txt").write_text("\n".join(lines) + "\n", encoding="utf-8")'

That packet is the only model input. A full repository upload adds risk without review value. Delete the packet after the note is written.

Step 6. Request a non-binding read

Ask the free model to list uncovered risks. Require each bullet to cite a path or a contract key. Forbid a verdict of correct, safe, or ready.

Role: non_binding_reader
Files: review-contract.json, changed.txt, exits.txt, scope.diff
Task: list risks the locked command does not cover
Rule: cite a path or a contract key in every bullet
Rule: do not approve the patch

A free model session can serve this narrow read. The model can still miss hidden fixtures and local conventions. A human maintainer still decides whether the patch lands.

Step 7. Check the contract before the note is posted

Run the checker on the contract, the path list, and the local log. Post model bullets only when the checker prints success. Prefix each bullet with the non-binding label.

python3 check_review_contract.py review-contract.json changed.txt after.log

The script below is a local unexecuted proposal. It checks fields, role, path scope, and the exit marker. It never contacts a network and it never opens a pull request.

#!/usr/bin/env python3
"""Proposal: validate an OSS review contract before model notes."""
import json
import sys

def main() -> int:
    if len(sys.argv) != 4:
        print("usage: check_review_contract.py CONTRACT CHANGED LOG")
        return 2
    contract_path, changed_path, log_path = sys.argv[1:4]
    contract = json.loads(open(contract_path, encoding="utf-8").read())
    required = [
        "command",
        "expect_before",
        "expect_after",
        "allowed_paths",
        "model_role",
    ]
    missing = [key for key in required if key not in contract]
    if missing:
        print("missing fields:", ", ".join(missing))
        return 2
    if contract["model_role"] != "non_binding_reader":
        print("model_role must stay non_binding_reader")
        return 2
    changed = [
        line.strip()
        for line in open(changed_path, encoding="utf-8")
        if line.strip()
    ]
    extra = sorted(set(changed) - set(contract["allowed_paths"]))
    if extra:
        print("paths outside contract:", ", ".join(extra))
        return 3
    log = open(log_path, encoding="utf-8").read()
    marker = "after=%s" % contract["expect_after"]
    if marker not in log:
        print("local log lacks", marker)
        return 4
    print("contract ok; model notes remain non-binding")
    return 0

if __name__ == "__main__":
    raise SystemExit(main())

Step 8. File the maintainer note

Use a fixed note shape so reviewers can scan it. Put local evidence first and model text last. Keep the model section short enough to scan.

Command: python -m unittest tests.test_widget
Local before: 1
Local after: 0
Server after: not run
Checker: contract ok
Model role: non-binding
Model notes:
- non-binding: tests/test_widget.py covers only the named parser path

A missing server exit line is still acceptable. A missing local exit line is not acceptable. Readers should rerun the command before they trust the note.

How failed checks should end

Exit 2 means required fields or the model role are wrong. Exit 3 means the diff left the allowed path list. Exit 4 means the local log lacks the expected marker.

Repair the failed lane before another model call. Do not ask a second model to overwrite a bad log. Edit the contract only when the command itself changed.

Limits that stay in force

One command can pass while a neighbor behavior fails. A clean server can still miss private fixtures or system packages. Free model wording can be fluent and still incomplete.

A log line that merely contains the marker can fool this check. Treat the checker as a gate, not as full test proof. Read the unittest summary before accepting the marker.

Normalize relative paths before running the set comparison. A leading prefix can cause a false extra-path failure.

This article states no quota, latency, or uptime figure. Those values were not verified against a primary source here. A free tier is optional assistance, not a promised production host.

Who should not use it

Skip the flow when the patch holds secrets or customer data. Skip it when project rules ban external model reads. Skip it when the bug depends on devices the server cannot attach.

Skip it for broad refactors that no single command represents. Skip it when the review must be a formal security audit. A non-binding model reader cannot replace that audit.

Practice to keep

Store the contract, the log marker, and the checker result with the patch. Mark every model sentence as non-binding in the review thread. Ask one maintainer to rerun the locked command locally.

Optional reruns and model reads can support that second look. They do not replace the local exit code. Operators who keep evidence separate from commentary can try the optional lanes in MonkeyCode.

Cite a Clean Host Log in Every OSS Review Note

Taylor Wang — Fri, 09 Oct 2026 15:15:06 +0000

An OSS review note should cite a clean host log. A model draft stays private until that citation exists. This workflow keeps the patch small and the comment checkable.

Start from the review risk

October often brings a rush of first open-source patches. Speed without a host log creates weak review notes. A stranger should be able to replay the cited command.

A laptop log can hide a local toolchain quirk. A clean host log reduces that doubt before a public comment. The contributor treats the remote run as the review witness.

Where the helper tools fit

MonkeyCode's free model access is an operator-stated option. Disclosure: This article was prepared as part of MonkeyCode's product outreach. A free server option is operator-stated as well.

The operator describes the project as open source. This article does not invent a repository URL. Readers should use the link published in current docs.

Those options can draft notes and host one clean run. They do not decide which patch is correct. The host log and the diff remain the evidence.

Token caps, model lists, and server lifetime change. The contributor reads the current project docs before relying on either option. This article states no quota, no hardware size, and no duration.

Artifact this method leaves behind

The method produces a small file set before any public comment. None of them is a pull request by itself. Each file has one job in the review path.

File	Job	Public?
repro.sh	Runs the failing command	Yes, if secrets are absent
host.redacted.log	Stores the cleaned host output	Yes
evidence.md	Maps commands to exit codes	Yes
redact.diff	Shows every redaction change	No
review.draft.md	Holds unfiltered model notes	No

The example below is a proposed and unexecuted workflow. It was not executed against a live upstream. Replace the sample paths before any real use.

1. Freeze one public reproducer

The reproducer should fail for one stated reason. It should avoid network calls that are not required. It should print the exit code on purpose.

#!/usr/bin/env bash
set -eu

ROOT="$(cd "$(dirname "$0")" && pwd)"
cd "$ROOT"

echo "host=$(uname -s)"
echo "pwd=$(pwd)"
echo "commit=$(git rev-parse --short HEAD)"

set +e
python -m pytest tests/test_parse_edge.py -q --tb=line
status=$?
set -e

echo "exit=${status}"
exit "${status}"

The script prints host, commit, and exit code. Those three lines anchor the later review notes. A missing line means the log is incomplete.

A failing test under set -e would stop the script early. The sample disables that stop only around pytest. The exit line still prints after a failure.

2. Run the script on a clean host

The contributor copies the public project onto the clean host. The free server option can supply that clean host. The laptop remains the place where the patch is written.

The host alias devbox stands for the chosen clean server. It is not a fixed product hostname here. The contributor substitutes the real alias before running.

git rev-parse HEAD > commit.txt
ssh devbox 'git clone https://example.invalid/org/project.git "$HOME/oss-repro"'
scp commit.txt repro.sh devbox:"$HOME/oss-repro/"
ssh devbox 'cd "$HOME/oss-repro" && git checkout --detach "$(cat commit.txt)" && bash repro.sh' | tee host.log

The clone uses a placeholder URL in this example. The contributor replaces it with the real public repository. A private URL does not belong in the pasted log.

A shallow clone may omit the pinned commit. The example uses a full clone for that reason. The contributor may switch to a shallow clone only after a fetch check.

The pinned commit must already exist on the public remote. A local-only commit cannot be checked out there.

3. Redact the log before anyone else sees it

The contributor redacts tokens, emails, and internal host names. The log should stay safe to paste in a public issue. A secret in the log blocks public publication.

sed -E \
  -e 's/(token|password|secret)[[:space:]]*[:=][[:space:]]*[^[:space:]]+/\1=[REDACTED]/Ig' \
  -e 's/[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Za-z]{2,}/[EMAIL]/g' \
  host.log > host.redacted.log
diff -u host.log host.redacted.log > redact.diff || true

The redaction pass runs on the local copy only. The contributor reviews every replaced line by hand. A script cannot know every private secret shape.

The local diff output is the redaction review. A diff exit code of 1 means lines changed. It does not mean the redaction command failed.

The contributor deletes the raw log after that check. The public file is the redacted host log. Later evidence rows cite that redacted file only.

4. Build an evidence table from the log

Each row names a command, an exit code, and a file. The table accepts only lines present in the redacted log. A remembered result does not earn a row.

# evidence.md

| id | command | exit | file | log line |
| --- | --- | --- | --- | --- |
| E1 | python -m pytest tests/test_parse_edge.py -q --tb=line | 1 | tests/test_parse_edge.py | exit=1 |
| E2 | git rev-parse --short HEAD | 0 | commit.txt | commit=abc1234 |

The evidence ids become the only legal citations. A review sentence without an id is incomplete. The contributor adds rows only after a fresh host run.

5. Ask a free model to draft notes

The prompt includes evidence.md and a short patch diff. It excludes unrelated repository files from the prompt. The model may suggest wording, not new product claims.

Draft review notes for this open-source patch.
Use only ids from evidence.md.
Each bullet must end with an evidence id in brackets.
Do not invent commands, versions, or test results.
If a suggestion lacks an id, write SKIP.

MonkeyCode's free model access can run that drafting step. Any other local or hosted model can run it too. The choice does not change the citation rule.

The draft stays in review.draft.md until the filter runs. It is not pasted into the issue yet. A fluent sentence is still untrusted review text.

6. Filter the draft before a maintainer sees it

A small script drops bullets that lack a known id. It also drops bullets that cite a missing id. The maintainer should see the filtered file only.

# Proposed filter. This example was not executed as a benchmark.
from pathlib import Path
import re

evidence = Path("evidence.md").read_text(encoding="utf-8")
known = set(re.findall(r"\|\s*(E\d+)\s*\|", evidence))
kept, dropped = [], []

for line in Path("review.draft.md").read_text(encoding="utf-8").splitlines():
    ids = re.findall(r"\[(E\d+)\]", line)
    if line.startswith("- ") and ids and set(ids) <= known:
        kept.append(line)
    elif line.startswith("- "):
        dropped.append(line)

Path("review.public.md").write_text("\n".join(kept) + "\n", encoding="utf-8")
Path("review.dropped.md").write_text("\n".join(dropped) + "\n", encoding="utf-8")
print(f"kept={len(kept)} dropped={len(dropped)}")

python filter_review.py

The filter accepts only dash bullets with brackets. A star bullet is dropped even with a valid id.

A dropped line can still be useful locally. It must not reach the maintainer as a fact. The contributor either gathers a new log row or discards the line.

7. Patch only lines the table can support

The patch should touch the files named in the evidence table. A drive-by cleanup belongs in a separate change. The commit message names the supporting evidence id.

git checkout -b fix-parse-edge
git add tests/test_parse_edge.py src/parser.py
git commit -m "Fix parse edge case cited by E1"
git show --stat HEAD

The contributor reruns repro.sh on the same clean host. The new log should show the expected exit code. A laptop-only pass is not enough for the review note.

ssh devbox 'cd "$HOME/oss-repro" && bash repro.sh' | tee host-after.log
sed -E \
  -e 's/(token|password|secret)[[:space:]]*[:=][[:space:]]*[^[:space:]]+/\1=[REDACTED]/Ig' \
  host-after.log > host-after.redacted.log
grep '^exit=' host.redacted.log host-after.redacted.log

The compare step looks only at marked lines. A changed exit code is the result that matters here. Extra warnings need a new evidence row first.

The after log gets the same redaction pass first. The grep command then reads only redacted files. A raw after log stays on the laptop.

8. Keep the draft out of the commit

The public branch contains the patch, the reproducer, and the redacted log. The model draft stays untracked on purpose here. A published draft invites readers to treat guesses as findings.

printf '%s\n' 'review.draft.md' 'review.dropped.md' >> .git/info/exclude
git add repro.sh host.redacted.log evidence.md src/parser.py
git status --short

A printf append can duplicate exclude lines on a second run. Run that exclude command only once per clone.

The status output should omit the draft files. The contributor stops if a draft path appears. That stop is cheaper than a public correction.

9. Post a comment that a stranger can replay

A public sentence should contain one evidence id. Two ids in one sentence hide a weak claim. The contributor splits that sentence before posting it.

The filtered file becomes the only comment source. The contributor copies it only after the reread. No new sentence is added during the copy.

Clean host log shows exit=1 for tests/test_parse_edge.py [E1].
The patch commit changes src/parser.py and records exit=0 on that host.
Repro script: repro.sh. Pasted lines come from host.redacted.log.

The public comment lists the host fact, the evidence id, and the commit. It links the reproducer rather than pasting a long model essay. The public tone stays factual, short, and specific.

The contributor does not claim the model found the bug. The contributor claims the log and the diff agree. That distinction protects the maintainer from unsupported claims.

Decision table

Situation	Action	Reason
Log lacks an exit line	Rerun on the clean host	The witness is incomplete
Model note has no id	Drop the note	Citation rule failed
Patch touches extra files	Split the change	Review scope grew
Remote run differs from laptop	Trust the clean host for the note	Local state may be dirty
Secret appears in the log	Redact and rotate the secret	Public paste is unsafe
Docs show no free host today	Use another clean host	Availability is not a promise

Limits of this method

The filter checks citation shape rather than technical truth. A cited note can still be technically wrong. A human must read the diff before opening the request.

A clean host can still miss a production dependency. The method does not prove thread safety or performance. It proves one command result on one host.

Free model access can fail, rate-limit, or change models without notice. The free server option can be unavailable or short-lived. The workflow must still run with a local model and any clean shell.

This article does not name a model, a token cap, or a machine size. Those facts belong to the current project documentation. Outdated numbers would mislead a careful open-source contributor.

Who should not use this

Maintainers with a required CI replay do not need a second host note. They should follow the project CI contract instead. A parallel log can confuse that required contract.

Security patches that include exploit details do not belong in this template. Private disclosure channels should carry those security reports. This method is meant for ordinary failing tests.

Contributors who cannot redact secrets should not paste host logs. They should fix the reproducer before any paste. A public log is optional only after redaction.

Close

The review note is ready when every bullet maps to the redacted log. The model draft is a private input, not the published comment. A clean host plus a citation filter keeps that order stable.

Readers can check the current MonkeyCode project docs. Those docs describe the free model and free server options. Current terms there override any memory of older limits.

Open an OSS Patch From a Checked Four-File Bundle

Taylor Wang — Thu, 08 Oct 2026 10:50:41 +0000

An open-source patch is not ready at the first green test. It is ready when four frozen files agree with each other. A pull request waits until a local checker accepts them.

Those files are the issue note and the repro log. The other pair is the patch file and the test log. A model may comment, but a maintainer still merges.

Hold the request until the bundle is complete

Each stage writes one file and then stops. A checker script blocks the request until those files exist. The contributor does not open the request from a chat summary.

A long chat thread hides the failing command and exact output. A maintainer cannot replay a paragraph of advice. A frozen bundle can be cloned, diffed, and run again.

The bundle lives beside the clone, not inside the product tree. That split keeps generated logs out of the product patch. The layout below is a proposed template for local use.

contrib-bundle/
  issue.md
  repro.sh
  repro.log
  patch.diff
  test.sh
  test.log
  review-notes.md
  check-bundle.sh

1. Write the issue note before editing code

The issue note states expected behavior in one short paragraph. It names the command that should fail on the current tree. It names the single file the contributor expects to change.

Keep the note under forty lines so the bug stays narrow. A long note often hides a second unrelated defect. Add the upstream issue URL when one already exists.

# Issue note

Command `python -m unittest tests/test_slug.py` fails.
Observed call returns hello-- with the trailing separators intact.
Expected result is hello with trailing separators removed.
Suspected file is pkg/slug.py only.
Upstream issue: replace this line with the real URL.

Create the bundle directory before the first command runs. Keep that directory outside the product commit on purpose. A later status check should ignore the bundle path.

mkdir -p contrib-bundle
printf '%s\n' '# Issue note' > contrib-bundle/issue.md

2. Freeze a reproduction that exits non-zero

Place the product clone in a sibling directory named repo. The sample scripts resolve that path from their own folder. Change the path if the clone uses another directory name.

The repro script must exit non-zero on the unpatched tree. It must print the observed value with a stable marker. It must not edit source files or install new packages.

#!/usr/bin/env bash
set -euo pipefail
cd "$(dirname "$0")/../repo"
python - <<'PY'
from pkg.slug import slugify
got = slugify('Hello--')
print('observed=' + got)
raise SystemExit(0 if got == 'hello' else 1)
PY

Run the script once and redirect both streams into the log. Do not edit that log by hand after the run. A hand-edited log will not match a later run.

chmod +x contrib-bundle/repro.sh
./contrib-bundle/repro.sh > contrib-bundle/repro.log 2>&1 || true
test -s contrib-bundle/repro.log
grep -n '^observed=' contrib-bundle/repro.log

The failure guard keeps the shell alive after a real miss. The log must still contain the observed marker line. A missing marker means the script never reached the bug.

This script is an unexecuted example, not a measured run. Adapt the import path before using it on a real clone. Replace the sample if the project exposes another entry point.

3. Export a diff that stays on one path

Change only the suspected file during this pass. Stage nothing else before exporting the unified diff. Write that diff into the bundle under a fixed name.

cd repo
git diff -- pkg/slug.py > ../contrib-bundle/patch.diff
test -s ../contrib-bundle/patch.diff
git diff --stat -- pkg/slug.py

Reject the diff when it touches lockfiles or generated assets. Reject it when unrelated docs change in the same hunk set. A wide diff is a new task, not a review bundle.

The sample below is an illustration, not a tested upstream fix. The real project may already collapse repeated separators. Replace the sample with the actual diff from git.

--- a/pkg/slug.py
+++ b/pkg/slug.py
@@
 def slugify(value):
-    return value.strip().lower().replace(' ', '-')
+    text = value.strip().lower().replace(' ', '-')
+    while '--' in text:
+        text = text.replace('--', '-')
+    return text.strip('-')

4. Record the test command and the full log

The test script runs the narrow test or the full suite. It writes stdout and stderr into one log file. Its exit status must match the test runner status.

#!/usr/bin/env bash
set -euo pipefail
cd "$(dirname "$0")/../repo"
python -m unittest tests/test_slug.py

set +e
./contrib-bundle/test.sh > contrib-bundle/test.log 2>&1
status=$?
set -e
printf 'exit=%s\n' "$status" | tee -a contrib-bundle/test.log
test "$status" -eq 0

Read the full log before asking any model for notes. A green summary with a skipped test is not success. Record skips in the issue note instead of hiding them.

5. Redact the bundle before it leaves the machine

Search the bundle for obvious secret markers before any upload. Keep env files and private keys out of that directory. A hosted review is optional, and a leaked secret is not.

grep -RInE 'API_KEY|SECRET|TOKEN|PASSWORD|BEGIN PRIVATE' contrib-bundle && echo 'stop: possible secret' || echo 'no-obvious-secrets'
find contrib-bundle -name '.env' -o -name '*.pem' -o -name 'id_rsa'

The search is a tripwire, not a complete secret scanner. It will miss renamed credentials and encoded blobs. The contributor still reads every diff line before upload.

6. Gate the request with a shape checker

The checker does not understand the bug itself. It only enforces bundle shape and a few markers. That limit is enough to stop a half-finished upload.

Edit the suspected path inside the checker before the first run. The sample path is only pkg/slug.py and will miss most repos. A hardcoded path is useful because the scope stays explicit.

#!/usr/bin/env bash
set -euo pipefail
root="$(cd "$(dirname "$0")" && pwd)"
need=(issue.md repro.sh repro.log patch.diff test.sh test.log)
for f in "${need[@]}"; do
  test -s "$root/$f" || { printf 'missing:%s\n' "$f"; exit 1; }
done
grep -q '^diff --git ' "$root/patch.diff" || { echo 'not-a-diff'; exit 1; }
grep -q '^observed=' "$root/repro.log" || { echo 'no-observed-marker'; exit 1; }
if grep -E '^\+\+\+ b/' "$root/patch.diff" | grep -v 'b/pkg/slug.py' | grep -q .; then
  echo 'diff-out-of-scope'
  exit 1
fi
if find "$root" -type f -size +200k | grep -q .; then
  echo 'file-too-large'
  exit 1
fi
echo 'bundle-ok'

Run the checker before any pull request command. Stop when the script prints anything other than success. Fix the named file, then rerun the checker from scratch.

chmod +x contrib-bundle/check-bundle.sh
./contrib-bundle/check-bundle.sh

The size cap is a local reading policy, not a host limit. Raise it only after a maintainer asks for a larger log. Generated coverage pages do not belong in this folder.

7. Ask a model to review the four frozen files

A model review helps when the input is the frozen bundle. It gets noisy when the input is the entire checkout. Attach the note, both logs, the diff, and the test script.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode's free model access can read that small bundle. Its free server option can host the review away from tests.

No quota, model name, or hardware figure is stated here. Those details change and belong on the current product page. Confirm both options there before treating them as available.

Use one fixed prompt and store the reply beside the logs. Do not treat that reply as a merge approval. A later human still compares the notes against both logs.

Review this local contribution bundle only.
Read issue.md, repro.log, patch.diff, and test.log.
List mismatches between the issue note and the diff.
List claims that test.log does not actually show.
Do not claim the patch is correct.
Do not propose new features.
End with: human maintainer decides.

That prompt is a proposal, not a scored evaluation. Discard the reply when it invents a path absent from the diff. Rerun with the same four files rather than the whole repo.

8. Assign each check to one owner

Some checks are mechanical, and some must stay human. The table below assigns one owner to each check. Hosted model notes stay inside the advisory row only.

Check	Owner	Rule
Bundle files exist and are non-empty	Checker	Missing files block the request.
Repro log contains the observed marker	Checker	A silent log is not a reproduction.
Diff starts with a git diff header	Checker	A note file is not a patch.
Diff stays on the suspected path	Checker	Extra paths start a separate task.
Test log shows no hidden skips	Human	Runners phrase skips in different ways.
Patch style looks consistent	Model note, then human	Style notes stay advisory.
Project will accept the design	Maintainer	A model cannot speak for the project.
Secrets stay out of the bundle	Contributor	Never upload tokens or private keys.

The hosted review is for notes, not for the test suite. Keep the passing test log on the machine that produced it. A remote comment never replaces that local evidence.

9. Name the limits before requesting review

This bundle does not prove thread safety or load behavior. It does not cover bugs that need private credentials. Leave those cases on a maintainer-approved private repro.

A green test log can still miss a sibling package. The checker reads file shape, not coverage data. Add a coverage line only when the project already prints one.

Model notes can be wrong, stale, or simply overconfident. Keep each note dated inside the review file for comparison. Never paste model wording into the commit message as fact.

The two availability options are claims supplied for this draft. They are not a promise of capacity, speed, or future price. Read the current product page before relying on either option.

10. Skip the workflow when it outweighs the fix

Skip this workflow for a one-line documentation typo. Four frozen files would outweigh that small documentation change. Skip it when a failing test and target branch already exist.

Skip the model step when the diff holds secrets or customer data. Skip it for unpublished vulnerability details as well. Review those patches with a human on a trusted machine.

Skip any hosted server when the license bars third-party review. Read that license before uploading any project file. A convenient host does not grant permission to upload code.

11. Commit the product change, then open the request

After the checker succeeds, commit the product change only. Leave the bundle outside that commit unless fixtures were requested. Name the exact test command in the pull request body.

cd repo
git add pkg/slug.py tests/test_slug.py
git status --short
git diff --cached --stat
git commit -m 'Fix slugify trailing separator collapse'

A plain pull request body can stay short and concrete. It points at local evidence without claiming a model sign-off. The sample body below is a template, not a required format.

cat > contrib-bundle/pr-body.md <<'EOF'
## Local evidence

Repro command: ./contrib-bundle/repro.sh
Test command: python -m unittest tests/test_slug.py
Checker result: bundle-ok
Model notes, if any, stayed advisory and were not an approval.
EOF
gh pr create --title 'Fix slugify trailing separator collapse' --body-file ../contrib-bundle/pr-body.md

Do not write that a model approved the merge. Say the tests ran locally and name the command. Attach bundle paths as evidence notes, not as correctness proof.

Readers can use MonkeyCode's free model access for this review step. The free server option can host that pass away from the suite. Local tests and the maintainer still gate the merge.

Record an OSS Replay Packet Before Model Dissent

Taylor Wang — Thu, 08 Oct 2026 08:28:58 +0000

A model review is useful only after a frozen replay packet exists. That packet stores the pinned commit, the failing command, and the patch. The model may write dissent notes, but the local oracle decides the merge.

Open-source fixes fail when review context lives only in a chat window. A later reader cannot replay the bug, the diff, or the exact prompt. A review packet turns that context into files with a recorded result.

Start from a pinned failure

Contribution quality drops when the patch and the bug drift apart. The author pins the upstream commit before any edit or model call. The manifest records the clone URL, the commit, and the test command.

This step is not a fixture shrink and not a path allowlist. Those earlier bounds limit the files under edit. This packet bounds the evidence a later reviewer can replay.

What belongs in the packet

The packet stays small enough to copy onto a clean server. It stores inputs, commands, and the expected local result. Chat transcripts and screenshots stay out of the tree.

review-packet/
  manifest.json
  repro.sh
  fix.patch
  fail.out
  fail.err
  fail.exit
  notes/dissent.md
  notes/scored.json
  oracle/checklist.txt
  score.py

The manifest names the project, the commit, and the test command. It does not name a model, a quota, or a hardware size. Those fields change, and a packet should not freeze unverified claims.

Run seven packet steps

The author follows these steps in order on a clean worktree. The workflow stops when the first required artifact is missing. A skipped step makes the later model note unverifiable.

1. Pin the upstream commit

The author clones the repository and checks out a known commit. The workflow refuses to continue when the worktree is dirty. A dirty tree hides unrelated edits inside the review.

git clone "$UPSTREAM_URL" repo
cd repo
git fetch origin "$PINNED_SHA"
git checkout --detach "$PINNED_SHA"
test -z "$(git status --porcelain)"

The status test must succeed before the packet proceeds. A shallow clone can save time on a large history. Full history is fetched when blame or bisect becomes necessary.

2. Capture the failing command

The author writes one command that fails for the reported bug. The capture saves stdout, stderr, and the exit code beside the manifest. The author does not patch the tree during this capture step.

The author places the failing command in repro.sh before any edit. The script below is a proposal and was not executed here. The script points at one reported failure, not the whole suite.

#!/usr/bin/env bash
set -euo pipefail
pytest tests/test_parser.py::test_empty -q

The author saves streams after that script runs on the pinned commit. The exit code stays in a separate file for later comparison. The capture sequence below is proposed and was not executed here.

set -o pipefail
bash repro.sh > fail.out 2> fail.err
echo $? > fail.exit
test "$(cat fail.exit)" -ne 0

A zero exit means the bug is not reproduced yet. The author stops and fixes the command before any model call. A model cannot validate a failure the packet never shows.

3. Keep the human patch minimal

The author applies the smallest change that targets the captured failure. The author commits it locally with a message that names the bug. Generated suggestions stay in a side file, not in the diff.

git add -p
git commit -m "Fix report parser on empty input"
git format-patch -1 --stdout > fix.patch
git apply --check fix.patch

The patch file is the only code a reviewer should read first. Extra refactors belong in a later commit with a separate packet. This keeps blame, revert, and review inside one bounded change.

4. Write the manifest

The manifest records what the author claims and what the reviewer can rerun. The author fills every field before a model prompt is sent. The JSON below is a proposal with placeholders, not a measured run.

{
  "project": "example-cli",
  "upstream": "https://example.invalid/org/example-cli.git",
  "commit": "PINNED_SHA",
  "fail_command": "pytest tests/test_parser.py::test_empty -q",
  "pass_command": "pytest tests/test_parser.py -q",
  "patch": "fix.patch",
  "oracle": "oracle/checklist.txt"
}

A real upstream URL belongs only after an operator verifies it. The example host above is intentionally not a live project. The author replaces every placeholder before the packet leaves the machine.

5. Replay on a clean server

A clean server matters when the laptop cannot host the toolchain. MonkeyCode, per the operator, offers free model access for the later dissent pass. The same source reports a free server option for this replay.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. Current quotas, model names, and duration are not stated here. The operator confirms those terms on the product page before a long run.

The packet should still run on any clean host with the same commands. Those commands, not the vendor, define the replay. The manifest stays free of quota claims and hardware guesses.

The author copies the packet, not the chat history, onto that server. The server runs the fail command on the pinned commit before the patch. The patch is applied only after the saved failure code matches.

python3 score.py check-fail manifest.json
git apply --check fix.patch
git apply fix.patch
python3 score.py check-pass manifest.json

A pass after the patch is necessary, but it is not sufficient. Hidden regressions still need the broader command named in the manifest. Both exit codes are recorded in the packet before any model call.

6. Ask only for dissent

The author sends the manifest, the patch, and the oracle checklist. The prompt asks for disagreements, missing tests, and risky assumptions. The prompt does not request a rewrite or a merge approval.

A bounded prompt keeps the review inside the packet. The prompt below is proposed text and was not executed for this draft. The author saves the reply as notes/dissent.md with a UTC timestamp.

The reviewer has no merge rights.
Read manifest.json, fix.patch, and oracle/checklist.txt.
List dissent notes only.
For each note, cite a file and a reason.
Mark confidence as low, medium, or high.
Do not propose a replacement patch.
Do not claim a test ran unless the packet shows an exit code.

The author never pastes the reply over the patch without a human diff. The dissent file is evidence, not an instruction to edit. A confident tone does not raise the note above the oracle.

7. Score notes against the oracle

The oracle is a local checklist the author already trusts. The author writes that checklist from the bug report and failing command. A checklist written after the model reply is not an oracle.

empty input returns a typed error, not a stack trace
parser does not read paths outside the fixture directory
fail command exits non-zero on the pinned commit
patch does not touch lockfiles or generated assets

Each dissent note becomes confirmed, rejected, or untested. Untested notes block the pull request until a command exists. The sample score file below is a proposal, not a recorded review.

{
  "notes": [
    {
      "id": "D1",
      "file": "src/parser.py",
      "reason": "empty input still falls through to the generic handler",
      "confidence": "medium",
      "status": "untested"
    }
  ]
}

The scorer below is a proposal, not a benchmark. It checks status labels and does not rank models. It also does not measure speed or product uptime.

STATUSES = {"confirmed", "rejected", "untested"}

def gate(notes):
    unknown = [n for n in notes if n.get("status") not in STATUSES]
    if unknown:
        raise SystemExit("unknown status")
    pending = [n for n in notes if n["status"] == "untested"]
    if pending:
        raise SystemExit("untested dissent remains")
    return "ready-for-human-review"

A green gate means the packet is reviewable, not that the patch is correct. A wrong human label can still pass this script. The method catches a missing review, not a mistaken judgment.

Apply one decision row

The author chooses one row before opening a pull request. The table is a rule set, not a measured study. Empty dissent never counts as model or maintainer approval.

Local result	Model note	Next action
Failure not reproduced	Any note	Repair the packet first
Failure reproduced, patch still fails	Any note	Repair the patch locally
Patch passes and note is confirmed	Cites a real gap	Add a test or narrow the claim
Patch passes and note is rejected	Contradicts the oracle	Record the rejection reason
Patch passes and note is untested	No command yet	Hold the pull request
Patch passes and dissent is empty	No notes	Human review remains required

A silent model can miss a race, a permission check, or a bad default. Human review stays mandatory after the table selects a row. The pull request should link the row that was chosen.

Limitations to state in the request

The packet cannot see private maintainer context or unpublished issues. A free server may lack services, secrets, or hardware the bug needs. The author does not copy credentials into the packet or the model prompt.

Free model access can change in quota, model choice, or availability. This article states no token count, no uptime, and no speed claim. The operator verifies the live terms before relying on a free run.

Generated dissent can look precise while citing a stale line. The author compares every citation to the patch before trusting it. The author discards notes that name files absent from the packet.

Who should skip this method

This method is a poor fit for a one-line typo with an obvious test. The packet overhead is larger than that change. A direct pull request is enough in that narrow case.

This method is a poor fit when the bug needs secrets or private logs. A free server and a model prompt are the wrong place for those inputs. The author redacts first or keeps the work on a trusted machine.

This method is a poor fit when no local oracle exists for the bug. A model note without a checklist becomes an unreviewed opinion. The author builds the checklist before spending a model call.

Close the loop in the pull request

The pull request links the packet, the dissent file, and the score output. The body states which notes were confirmed and which were rejected. Maintainers can replay the commands without reading a chat log.

The body names the pinned commit in the pull request text. The body attaches the fail exit code and the pass exit code. That record outlives the chat session that produced the dissent.

When a clean host is missing, the operator checks current MonkeyCode terms. The author runs this same packet once under the free server option. That run counts as a replay, not as universal proof.

The local oracle and the human reviewer still own the merge. A free run does not replace the pinned failure or the dissent score. The author publishes the packet with the pull request for later replay.

Shrink an OSS Bug to One Fixture Before Patching

Taylor Wang — Wed, 23 Sep 2026 17:22:31 +0000

A patch without a tiny fixture wastes maintainer time. Upstream reviews stall when failure still spans five modules. Shrink the reported bug to one fixture first.

The fixture then becomes the only allowed proof. Production code then changes against that proof alone. Model comments arrive later as review notes only.

Capture the report, not the guess

Issue text often mixes symptoms, theories, and stack noise. A contributor copies the raw command and output. Dates, host names, and secrets get stripped first.

mkdir -p .repro
printf '%s\n' "$ISSUE_URL" > .repro/source.txt
# Replace make test-one with the upstream single-test command.
script -q -c 'make test-one TEST=failing_case' .repro/raw.log
grep -E 'FAIL|Error|Exception' .repro/raw.log > .repro/errors.txt

That log is evidence, not a patch plan. No production file is edited in this step. Theories from the issue stay out of .repro.

Shrink until one fixture fails

A full suite failure hides the real unit. The next job is strict input reduction. Keep deleting inputs until one fixture still fails.

Copy the failing input into tests/fixtures/shrink/case.json.
Run only the project's documented single-test command.
Drop unused fields while the same assertion still fails.
Stop when one more deletion changes the error class.
Record the surviving command in .repro/repro.lock.

# Example only: swap in the repository's real runner.
npm test -- tests/shrink.test.js
# or
pytest -k test_shrink_case -vv

Label every runner as project-specific before sharing it. Do not invent a framework the repo lacks. A fixture that still needs three services is not small.

Reduce network, clock, and filesystem seams next. Prefer in-process fakes the project already ships. Live cloud calls do not belong in the shrink loop.

Stop shrinking with a decision table

Reduction without a stop rule deletes the bug. Use the table below as the exit gate. Each row is a hard stop, not a preference.

Observation	Action
Next deletion changes the error class	Keep the previous fixture and stop
Fixture still needs live network	Replace bytes with a recorded stub
Two unrelated assertions fail	Split fixtures; patch only one
Result depends on test order	Stop; isolate the test first
Clock skew changes the assertion	Freeze time with the project's helper
Fixture spans two packages	Cut imports until one package remains

Copy the chosen row into .repro/stop.txt. Reviewers then see why shrinking ended. A missing stop row means the fixture is still too large.

# .repro/repro.lock  (example layout, not a real hash)
test_cmd: pytest -k test_shrink_case -vv
fixture: tests/fixtures/shrink/case.json
stop_row: Next deletion changes the error class
delta: .repro/delta.md

Pin the command text, not a remembered alias. Aliases rot across contributor shells. The lock file must replay on a clean clone.

Record a behavior delta table

Maintainers read tables faster than long essays. The fixture needs a before row and after row. Expected values come from public docs or the failing assertion.

Probe	Command or call	Before (failing)	After (required)
Status	`cli parse case.json`	exit 2	exit 0
Message	stderr line 1	`invalid token`	(empty)
Return	`parse()`	throws `ParseError`	object with `ok: true`

This table is the only in-scope contract. Any extra behavior change needs a new row. Silent extras stay out of the pull request.

# .repro/delta.md
probe: parse(case.json)
before: ParseError("invalid token")
after: { ok: true, value: 42 }
out_of_scope: logging format, metrics counters

Keep the out-of-scope list short and explicit. Reviewers then ignore unrelated diffs with cause. Changelog noise is not a fixture win.

Patch only the path the fixture touches

Open the stack from the fixture's failing assertion. Change the smallest function that makes the after column true. Leave nearby refactors for a later pull request.

// Example fixture driver. Adapt names to the upstream tree.
const { parse } = require('../src/parse')
const caseJson = require('./fixtures/shrink/case.json')

test('shrunk fixture reaches documented success', () => {
  const result = parse(caseJson)
  expect(result).toEqual({ ok: true, value: 42 })
})

Run the single test until it passes. Then run the nearest package suite. A green fixture with a red neighbor package is incomplete.

git diff --stat
git diff -- src tests/fixtures/shrink tests/shrink.test.js
python - <<'PY'
# Example: fail the workflow if extra paths appear.
import subprocess, sys
allowed = {"src/parse.py", "tests/fixtures/shrink/case.json", "tests/shrink.test.js"}
out = subprocess.check_output(["git", "diff", "--name-only"], text=True)
files = {line for line in out.splitlines() if line}
extra = sorted(files - allowed)
if extra:
    print("extra paths:", extra)
    sys.exit(1)
print("diff stays inside the fixture graph")
PY

If the stat list grows past the fixture path, stop. Split the extra files out before review. Import-graph creep is a common silent expansion.

# Example import-graph check for a Python package.
python -c "import ast, pathlib, sys
root = pathlib.Path('src')
# Proposal only: replace with the project's module walker.
print('review import edges from the fixture next')"

Treat that walker as unlabeled proposal code. Real graphs come from the upstream package layout. Do not paste a generic graph into a foreign repo.

Re-read the diff without writing new code

A second pass should hunt extra behavior, not new features. Check license headers, changelog rules, and the commit template. Match the repository's existing style, not a personal one.

git log -5 --oneline -- src
git diff --check
# Example: reuse the project's own lint entry if it exists.
[ -f Makefile ] && grep -n '^lint' Makefile || true

Free model access can help that second pass. A contributor pastes the fixture, the delta table, and the diff. MonkeyCode provides free model access and a free server option for that review chat.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The model does not own the fixture. Review notes that lack a table row get discarded. Suggested rewrites stay out of the tree until a human replays the test.

Review prompt (example, not executed):
1. Read tests/fixtures/shrink/case.json
2. Read .repro/delta.md and .repro/stop.txt
3. Read the git diff only
4. List claims that lack a delta row
5. List files outside the fixture import graph

Keep the prompt numbered and boring on purpose. Creative refactors from a model are noise here. The optional review chat can use that free server.

Publish the reproduction with the pull request

Maintainers should replay the fixture without a meeting. Put the shrink command in the pull request body. Link the issue URL already stored in .repro/source.txt.

## Reproduction
1. `git checkout <this-branch>`
2. Run the locked single-test command in `.repro/repro.lock`
3. Confirm the after-column in `.repro/delta.md`

## Out of scope
- logging format
- metrics counters

Do not attach secrets or machine-specific paths. The fixture must run on a clean clone. Host-local caches are not reproduction.

git status --porcelain
test -z "$(git status --porcelain)" && echo 'tree clean before push'

A dirty tree hides fixture files from reviewers. Commit the lock, delta, stop row, and test together. Skip generated caches and editor swap files.

Limitations

This workflow assumes a test runner already exists. Greenfield repos without assertions need tests first. Flaky fixtures also fail the method, because shrink cannot split bugs from races.

Large performance issues rarely collapse to one JSON file. Multi-service outages need a different harness. Generated code and vendored trees also resist tiny fixtures.

Model review is optional and advisory only. It does not replace CI, owners, or maintainer judgment. Free access and a free server do not imply named models, quotas, hardware, or uptime here.

Who should skip this

Do not use fixture shrink for speculative cleanups. Do not use it when the issue already names a one-line typo. Embargoed security reports need the project's private channel, not a public fixture.

Contributors who cannot run the project tests locally should stop. A model chat is not a substitute for that run. The fixture and the delta table remain the patch.

Run a Fail-Then-Pass Replay Before an OSS Fix

Taylor Wang — Tue, 22 Sep 2026 15:38:27 +0000

An OSS fix without a replayable failure remains a guess.
Reviewers cannot verify a story that never failed locally.

This workflow pins one issue to one command.
That command must fail before any production edit.
The same command must pass after the patch.

The artifact is a fail-then-pass replay gate.
The gate stores needles from real test logs.
It refuses a patch when either needle is missing.

Replay beats a narrative

Issue threads often describe symptoms more than commands.
A contributor then ships a purely speculative diff.
The maintainer reruns folklore instead of a gate.

A fail-then-pass replay removes most of that folklore.
The failing log becomes the local contract.
The passing log remains the only proof.

The method fits libraries with already deterministic tests.
It also fits command-line tools with golden output.
It does not fit flaky network-only incidents.

Declare the gate

Keep the gate file beside the patch notes.
Use boring machine-readable fields for every run.
Treat this file as an example, not policy.

# example only: review/issue-replay.toml
issue_url = "https://github.com/example/lib/issues/1842"
upstream_ref = "v2.4.1"
test_cmd = "python -m pytest tests/test_parse.py::test_trailing_comma_issue_1842 -q"
fail_needle = "AssertionError: trailing comma must not emit Token.COMMA"
pass_needle = "1 passed"
log_dir = "review/logs"

Needles are literal substrings from local runs.
They are not clever regular expressions at all.
The contributor copies them from the first failure.

1. Isolate a worktree

Do not patch the daily working clone.
Add a detached worktree on the issue tag.
Record the resolved SHA beside the captured logs.

# example commands, not a live upstream
git fetch origin tag v2.4.1
git worktree add --detach ../lib-issue-1842 v2.4.1
mkdir -p ../lib-issue-1842/review/logs
git -C ../lib-issue-1842 rev-parse HEAD > ../lib-issue-1842/review/logs/HEAD.sha

The worktree keeps the main tree clean.
The SHA file stops silent tag movement.
Later logs must cite that exact SHA.

Use the version named in the issue, not folklore.
A tag is a stated reproduction target.
It is not a search across random history.

2. Write one regression assertion

The first edit belongs in the test file.
The test name should carry the issue number.
The assertion should encode the requested behavior.

# example sketch, not production code
def test_trailing_comma_issue_1842():
    tokens = parse("item,")
    kinds = [token.kind for token in tokens]
    assert kinds[-1] != "COMMA", (
        "trailing comma must not emit Token.COMMA"
    )

Run that test before touching library code.
It should fail with the same fail needle.
A pass here means the reproduction is wrong.

3. Prove the failure first

Install test extras only in that worktree.
Run the command stored in the gate file.
Capture stdout and stderr as one log.

# example
cd ../lib-issue-1842
python -m venv .venv
. .venv/bin/activate
pip install -e ".[test]"
python -m pytest tests/test_parse.py::test_trailing_comma_issue_1842 -q \
  > review/logs/fail.log 2>&1 || true
grep -F "trailing comma must not emit Token.COMMA" review/logs/fail.log

The captured log must contain the fail needle.
Absence of that needle aborts the whole workflow.
The contributor then tightens the failing reproduction.

A missing failure remains a blocked local state.
No production file should change before failure.
The test selection is the first bug to fix.

4. Patch the production surface

Change the smallest function that feeds the test.
Leave formatting and comments for a later commit.
Keep the patch readable in ordinary diff output.

# example sketch, not production code
def parse(source: str) -> list[Token]:
    raw = tokenize(source)
    if raw and raw[-1].kind == "COMMA":
        raw = raw[:-1]
    return raw

This parser sketch stays deliberately tiny and local.
Real upstream code will look quite different.
The rule is still one behavior per patch.

5. Require the pass log

Re-run the identical command after the production edit.
Save a sibling pass log next to the SHA.
Require the pass needle and forbid the fail needle.

# example
python -m pytest tests/test_parse.py::test_trailing_comma_issue_1842 -q \
  > review/logs/pass.log 2>&1
grep -F "1 passed" review/logs/pass.log
! grep -F "trailing comma must not emit Token.COMMA" review/logs/pass.log

A pass without the old needle is necessary.
It is not sufficient for an upstream merge.
Maintainers still own design and compatibility choices.

6. Automate both gates

Manual log reading still drifts across long sessions.
A small script should enforce both log needles.
The script below is a labeled example harness.

# example: tools/replay_gate.py
from __future__ import annotations

import argparse
import subprocess
import sys
import tomllib
from pathlib import Path


def load_gate(path: Path) -> dict:
    data = tomllib.loads(path.read_text(encoding="utf-8"))
    required = (
        "test_cmd",
        "fail_needle",
        "pass_needle",
        "log_dir",
    )
    missing = [key for key in required if key not in data]
    if missing:
        raise SystemExit(f"missing gate keys: {missing}")
    return data


def run_cmd(cmd: str, log_path: Path) -> str:
    log_path.parent.mkdir(parents=True, exist_ok=True)
    proc = subprocess.run(
        cmd,
        shell=True,
        text=True,
        capture_output=True,
    )
    body = proc.stdout + proc.stderr
    log_path.write_text(body, encoding="utf-8")
    return body


def require(body: str, needle: str, label: str, present: bool) -> None:
    found = needle in body
    if present and not found:
        raise SystemExit(f"{label}: missing needle: {needle!r}")
    if not present and found:
        raise SystemExit(f"{label}: unexpected needle: {needle!r}")


def main() -> int:
    parser = argparse.ArgumentParser()
    parser.add_argument("--gate", default="review/issue-replay.toml")
    parser.add_argument("--phase", choices=("fail", "pass"), required=True)
    args = parser.parse_args()

    gate = load_gate(Path(args.gate))
    log_dir = Path(gate["log_dir"])
    log_path = log_dir / f"{args.phase}.log"
    body = run_cmd(gate["test_cmd"], log_path)

    if args.phase == "fail":
        require(body, gate["fail_needle"], "fail", True)
        require(body, gate["pass_needle"], "fail", False)
    else:
        require(body, gate["pass_needle"], "pass", True)
        require(body, gate["fail_needle"], "pass", False)

    sha = Path("review/logs/HEAD.sha")
    if not sha.exists():
        raise SystemExit("missing review/logs/HEAD.sha")
    print(f"{args.phase} gate ok sha={sha.read_text().strip()}")
    return 0


if __name__ == "__main__":
    sys.exit(main())

The example uses shell mode only for local brevity.
A stricter harness should pass an argument list.
The harness ignores process exit codes on purpose.

Needles decide the gate, not pytest status.
A green pytest run can still miss the pass needle.
A red pytest run can still miss the fail needle.

# example sequence, run inside the worktree
python tools/replay_gate.py --phase fail
# edit tests, then library code
python tools/replay_gate.py --phase pass
git add tests/test_parse.py src/lib/parse.py review/issue-replay.toml

Run the fail phase before any production edits.
Run the pass phase after those production edits.
Commit the gate file with the patch notes.

The harness does not upload any local files.
It does not call a language model.
It only compares local logs to declared needles.

7. Second reader on the log delta

A local replay can still miss weak assertions.
A second reader can inspect the log delta only.
The source tree itself should stay off that path.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.
MonkeyCode offers free model access and a free server option.
Those availability notes come from the account operator.
They are not measured quotas, hardware claims, or benchmarks.

The prompt should receive three artifacts only.
Send the gate file, the fail log, and the pass log.
Do not send secrets, credentials, or vendor dumps.

# example review prompt, unexecuted
Read issue-replay.toml, fail.log, and pass.log.
List assertions that still look too weak.
List log lines that changed for the wrong reason.
Do not rewrite the patch.
Do not invent files that are not in the logs.

The model output is a checklist, not a merge vote.
The contributor still runs the local replay harness.
The maintainer still owns the final merge.

Contributors who already hold both logs can continue.
They can try that free model path on the delta alone.

Limitations

The gate trusts deterministic tests above all else.
Flaky suites will flap both declared needles.
Pin time, locale, and working directory first.

The gate cannot replay GUI clicks at all.
It cannot replay production-only race conditions.
Those bugs need other capture and tracing tools.

Needle matching is literal and somewhat brittle.
A wording change can fake a passing gate.
Re-copy needles after each test message edit.

Model review of logs stays inherently shallow.
It cannot see unlogged process state at all.
It cannot replace a human maintainer on the merge.

Free model access may change without prior notice.
A free server option is not a lab promise.
Do not build release process around either one.

Who should skip this workflow

Skip it when the project has no automated tests.
Skip it when the bug is a leaked secret.
Skip it when the patch is a docs typo.

Binary-only vendors cannot run this local harness.
Emergency hotfixes may not wait for extra worktrees.
Those cases need a different local discipline.

The core rule stays small and local.
Fail first on the issue's stated tag.
Pass later on the same recorded command.

A patch that cannot replay both states remains a guess.

Build an OSS Patch Allowlist From One Failing Test

Taylor Wang — Mon, 21 Sep 2026 13:58:50 +0000

An OSS patch is ready only after one named test flips. That named failing test is the merge oracle. The issue thread is only background noise here.

This workflow turns one test into a file allowlist. Coverage names every path the test actually executes. Files outside that map stay out of the commit.

Why issue threads fail as oracles

Bug reports mix symptoms, guesses, and local environment detail. Contributors then edit the modules that look related. The resulting pull request often touches idle code.

CI can still pass after that extra contact. Distant callers then break on the next release. A coverage allowlist stops that drift before review starts.

Artifact overview

The artifact is a small local review gate. It records one failing test and writes a coverage allowlist. It then rejects extra git paths before commit.

The Python snippets below are an unexecuted proposal. Projects should swap in their real test runner. Do not add a new framework for a single bug.

Preconditions

Match the issue against the default remote branch. The tree must be clean before coverage runs. A dirty worktree will poison the coverage allowlist.

git fetch origin
git switch --detach origin/main
test -z "$(git status --porcelain)"

The last command must print nothing at all. Stash or commit leftover files before continuing.

The project must already have a test command. This method does not install a new stack.

Step 1. Encode the bug as one test

Name the test after the issue number. Keep exactly one hard assertion in the test. Avoid snapshots of an entire production module.

# tests/test_issue_8841_retry_budget.py
from pkg.client import RetryBudget


def test_issue_8841_retry_budget_stops_after_two_failures():
    budget = RetryBudget(max_attempts=2)
    budget.record_failure()
    budget.record_failure()
    assert budget.can_retry() is False

Run only that test file for now. Confirm the failure matches the original report.

pytest -q tests/test_issue_8841_retry_budget.py -vv --tb=short

A passing result means the oracle is wrong. Rewrite the assertion before any production code edit.

Step 2. Collect coverage from that test only

Full-suite coverage is too wide for a patch allowlist. Collect data from the named test alone.

pytest tests/test_issue_8841_retry_budget.py \
  --cov=pkg \
  --cov-report=json:issue_8841_cov.json \
  --cov-report=term-missing

Turn covered files into a plain allowlist. Print that list before any edit starts.

# tools/cov_allowlist.py  (proposal)
import json
from pathlib import Path

data = json.loads(Path("issue_8841_cov.json").read_text())
files = sorted(
    path
    for path, row in data.get("files", {}).items()
    if row.get("summary", {}).get("covered_lines", 0) > 0
)
out = Path("issue_8841_allow.txt")
out.write_text("\n".join(files) + "\n")
print("\n".join(files))

python tools/cov_allowlist.py

Add the new test path to the allowlist by hand. Production files must come only from coverage output.

Step 3. Reject git paths outside the allowlist

Do not stage any changed files yet. Diff the working tree against HEAD first. Fail the gate when extra paths appear.

# tools/enforce_allowlist.py  (proposal)
from pathlib import Path
import subprocess
import sys

allow = {
    line.strip()
    for line in Path("issue_8841_allow.txt").read_text().splitlines()
    if line.strip()
}
allow.add("tests/test_issue_8841_retry_budget.py")
raw = subprocess.check_output(
    ["git", "diff", "--name-only", "HEAD"],
    text=True,
)
changed = {line for line in raw.splitlines() if line}
extra = sorted(changed - allow)
if extra:
    sys.stderr.write("paths outside coverage allowlist:\n")
    sys.stderr.write("\n".join(extra) + "\n")
    raise SystemExit(1)
print("allowlist ok:", ", ".join(sorted(changed)))

python tools/enforce_allowlist.py

A nonzero exit means the patch is too wide. Shrink the diff until the gate passes. Do not grow the allowlist to match it.

Step 4. Edit only executed files

Change production code after the gate exists. Stay strictly inside the coverage allowlist paths. Re-run the named test after every edit.

pytest -q tests/test_issue_8841_retry_budget.py --tb=short

The test must keep failing until the real fix lands. A stub that forces a pass is a false oracle.

Then run tests that import the same module. Package regressions hide beside a green single test.

pytest -q pkg/client tests/test_issue_8841_retry_budget.py

Public function signatures stay frozen unless a new test says otherwise. Extra helpers belong in a later separate change.

Step 5. Write a replay log for reviewers

Reviewers should not reconstruct the local ritual. Dump the test name, HEAD, allowlist, and output. Attach that file to the pull request.

{
  echo "TEST=tests/test_issue_8841_retry_budget.py"
  echo "HEAD=$(git rev-parse HEAD)"
  echo "--- ALLOWLIST ---"
  cat issue_8841_allow.txt
  echo "--- DIFFSTAT ---"
  git diff --stat
  echo "--- TEST ---"
  pytest -q tests/test_issue_8841_retry_budget.py --tb=short
} > issue_8841_replay.txt

The replay log is the primary review surface. The raw diff is supporting evidence only here.

Maintainer checks on the pull request

The maintainer re-runs one command from the replay log. The named test must fail on origin/main. It must pass on the proposed contributor branch.

git switch --detach origin/main
pytest -q tests/test_issue_8841_retry_budget.py --tb=short
# expect fail
git switch --detach FETCH_HEAD
pytest -q tests/test_issue_8841_retry_budget.py --tb=short
# expect pass
python tools/enforce_allowlist.py

A test that never failed on main is not an oracle. Reject that pull request without any further review.

Decision table

Use this table before the pull request opens. Each missed row should block the merge.

Signal	Action
Named test never failed	Rewrite the test. Do not patch.
Coverage JSON lists no files	The test never imported production code.
Diff includes files off the map	Split the work or add another named test.
Single test passes, package tests fail	Keep the patch. Fix the regression first.
Public API shape changed	Add a dedicated compatibility test.
Only comments or lockfiles changed	Stop. The oracle did not move.
Allowlist contains generated files	Exclude those paths. Do not hand-edit them.

Each table row is a hard stop condition. Green CI without a named oracle is not enough.

Optional second pass on the replay log

A language model can read the replay log and the allowlisted diff. It still cannot replace the failing named test. Treat those notes as extra maintainer questions only.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode provides free model access and a free server option. Those two availability facts are the only product claims in this article.

A contributor without a local GPU can run the replay-log review there. Keep the model prompt inside the allowlist. Send the log, the diff, and the test output only.

Review only files listed in issue_8841_allow.txt.
Ignore every path outside that list.
The oracle is tests/test_issue_8841_retry_budget.py.
List contradictions between the diff and the replay log.
Do not suggest new modules.
Do not delete or weaken the failing test.

Discard any note that widens the change set. Discard any note that removes the oracle. Humans still own the final merge decision.

Limits

Coverage is not a proof of full correctness. It only records lines the named test executed. Untested branches in the same file can still fail.

The coverage allowlist remains a tool-specific artifact. coverage.py maps Python poorly onto C extension code. Go, Rust, and Java need their own cover commands.

Cross-cutting refactors do not fit this gate. Wide migrations need a different review contract. Do not force one test onto a wide rename.

Flaky tests also break this merge oracle. Pin time, network, and random seeds first. A flake is not a bug reproduction.

Who should not use this

Authors of large public API redesigns should skip this method. Contributors without any runnable harness should skip it. People editing generated or vendored trees should skip it.

Security fixes that must touch many files need another process. A coverage allowlist can hide required extra paths. Follow the project security advisory flow instead.

Closing

The named failing test is the merge oracle. Coverage from that test is the file allowlist. The replay log is what reviewers actually read.

A model may comment on that replay log. That model does not get a merge vote. The test still has to fail, then pass, on the contributor machine.

Require a Local Review Contract Before an OSS Pull Request

Taylor Wang — Sun, 20 Sep 2026 11:54:03 +0000

An OSS patch should not leave a fork until four artifacts exist. Those artifacts are reproduction notes, a bounded patch, tests, and a review contract.

A model may fill fields in that contract. A human maintainer still owns the merge decision.

This workflow keeps model output out of git history. It also keeps contributor judgment in one signed file. The contract is the only place a model is allowed to speak.

Why the contract exists

Maintainers do not merge confidence scores or vibes. They merge commands, paths, and observed test output. A raw chat transcript is not review evidence.

The contract records answers maintainers already expect in review. It does not invent extra narrative or scope. It stores commands, paths, and residual risk.

The next sections propose a local convention. These files are not a published industry standard. Teams should adapt field names to their project.

Current AI review chatter often hides missing tests. A filled chat window is easy to fake. A signed contract with rerun commands is harder to fake.

Artifact layout

Keep review files off the upstream default branch. Use a local directory that never ships by default.

.review/
  reproduce.md
  test-record.md
  review-contract.yml
  model-fill.yml
  series.diff
  SIGNED_BY.txt

The patch itself lives only in git commits. The review contract lives beside those git commits. Do not add .review/ unless the project asks.

Step 1: Reproduce the bug on a frozen tree

Clone the project at the reporter commit SHA. Do not patch the tree before reproduction. Write the failing command into reproduce.md first.

git clone https://github.com/example/project.git
cd project
git checkout --detach "$REPORTER_SHA"
git switch -c review/issue-1234

Record the environment in two short lines. Record the exact command that fails. Record the observed output, truncated if noisy.

# .review/reproduce.md
- sha: 9f3c1aa
- os: linux
- command: cargo test queue::drop_on_cancel -- --exact
- observed: assertion failed: ticket.cancelled
- notes: fails with RUST_TEST_THREADS=1 only

Stop if the failure does not reproduce locally. A remote model cannot review a ghost bug. Leave a comment on the issue instead.

This example is a proposed layout, not a run log. Replace the cargo command with the project runner. Do not invent timings or pass rates.

Step 2: Bound the patch to one behavior change

Create commits that a maintainer can read in order. One commit should add or fix tests. One commit should change production code.

Optional third commit updates nearby docs only. Reject mixed commits at this stage. A later model reviews each commit alone.

git add src/queue.rs tests/queue.rs
# split the index if both files landed together
git reset HEAD
git add tests/queue.rs
git commit -m "test(queue): cover cancel during drop"

git add src/queue.rs
git commit -m "fix(queue): honor cancel on drop"

git add README.md
git commit -m "docs(queue): note cancel on drop"

The oneline log from main must stay short. List two or three commits after the split. Keep the subject lines boring and specific.

git log --oneline origin/main..HEAD
git diff --stat origin/main...HEAD

If the stat output spans unrelated packages, stop. Split the work before any model pass. Do not ask a model to justify a mixed diff.

A stacked series also helps revert. Maintainers can drop the docs commit. They can keep the test commit during bisect later.

Step 3: Record a test result a stranger can rerun

Write the test record by hand. Paste command output without paraphrase. Keep the original failing command in the same file.

mkdir -p .review
{
  echo "date: $(date -u +%Y-%m-%dT%H:%M:%SZ)"
  echo "sha: $(git rev-parse HEAD)"
  echo "fail_cmd: cargo test queue::drop_on_cancel -- --exact"
  echo "pass_cmd: cargo test queue::drop_on_cancel -- --exact"
  RUST_TEST_THREADS=1 cargo test queue::drop_on_cancel -- --exact
} | tee .review/test-record.md

The record must include the failing command from step one. It must include the passing command after the fix. Both commands should share the same runner flags.

Label this block as unexecuted on a public repo. Replace cargo with the real project runner. Do not invent pass rates or machine specs.

If CI is required, capture one job log excerpt. Store only the failing test names. Do not dump secrets from the log.

gh run view --log --job test | tail -n 80 > .review/ci-excerpt.txt

That gh command is optional and proposed. Skip it when the issue is local only. Never paste tokens into the excerpt file.

Step 4: Freeze a review contract schema

The contract is YAML with empty string defaults. Empty fields block the pull request locally. A small script enforces the required keys.

# .review/review-contract.yml
issue: "https://github.com/example/project/issues/1234"
reporter_sha: "9f3c1aa"
patch_shas: []
scope:
  allowed_paths:
    - "src/queue.rs"
    - "tests/queue.rs"
    - "README.md"
  forbidden_paths:
    - "vendor/"
    - ".github/workflows/"
repro_command: ""
fail_output_excerpt: ""
pass_command: ""
pass_output_excerpt: ""
behavior_change: ""
public_api_change: "none"
residual_risk: ""
maintainer_answers:
  why_this_fix: ""
  why_not_larger_refactor: ""
  how_to_revert: ""
model_role: "fill_blanks_only"
contributor_attestation: ""

This schema is the original artifact for this workflow. It is intentionally boring and explicit. Boring fields are easier for humans to verify.

Copy the reporter SHA from step one. Leave patch_shas empty until commits exist. Fill patch_shas with git rev-parse after the split.

git rev-parse origin/main..HEAD > /tmp/patch_shas.txt

Step 5: Let a model fill blanks, not commits

Copy the contract to model-fill.yml before any model run. Send only that file plus the commit diffs. Do not paste the entire repository into the prompt.

cp .review/review-contract.yml .review/model-fill.yml
git diff origin/main...HEAD > .review/series.diff
git log -p origin/main..HEAD > .review/series-commits.diff

A proposed prompt, not a production prompt, follows. It forbids new files and invented commands. It also forbids commit rewrites.

Fill empty string fields in model-fill.yml.
Use only series.diff, reproduce.md, and test-record.md.
Do not invent commands that were not run.
Leave unknown fields as UNKNOWN.
Do not add paths outside allowed_paths.
Do not rewrite commit messages.
Return YAML only.

The contributor then diffs the fill against the original. Keep every human-run command in place. Replace model-invented commands with UNKNOWN.

diff -u .review/review-contract.yml .review/model-fill.yml

Reject any fill that adds new paths. Reject any fill that claims the model ran tests. The model did not run those tests.

Merge accepted fields by hand. Do not overwrite human command strings. The signed file later attests to that edit.

Where a hosted model fits

Local review still needs a place to run the fill step. Some contributors lack a spare local GPU. Some forks keep diffs and prompts on a small server.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode offers free model access and a free server option. That option can host this YAML fill step.

The contract, tests, and commits stay under contributor control. The model still does not own the merge.

Use a hosted fill only after steps one through four exist. Do not start the issue by chatting with a model.

Step 6: Validate the contract before opening the PR

The validator is a proposed shell script. It does not contact the network. It fails closed on empty required fields.

#!/usr/bin/env bash
# validate-review-contract.sh — proposed local gate, not production YAML parsing
set -euo pipefail
file=".review/review-contract.yml"
need=(
  issue reporter_sha repro_command fail_output_excerpt
  pass_command pass_output_excerpt behavior_change
  residual_risk contributor_attestation
)
for key in "${need[@]}"; do
  val="$(awk -F': ' -v k="$key" '$1==k{print $2; exit}' "$file" | tr -d '"')"
  if [[ -z "$val" || "$val" == "UNKNOWN" ]]; then
    echo "missing $key" >&2
    exit 1
  fi
done
while IFS= read -r p; do
  case "$p" in
    src/queue.rs|tests/queue.rs|README.md) ;;
    *) echo "path outside scope: $p" >&2; exit 1 ;;
  esac
done < <(git diff --name-only origin/main...HEAD)
echo "review contract ok"

Mark the script as a proposal only. Real projects should parse YAML with a real parser. The awk shortcut is for teaching.

Run the script on the filled contract. Sign the attestation after it passes. Only then open the pull request.

chmod +x validate-review-contract.sh
./validate-review-contract.sh
printf '%s\n' \
  'I ran the commands in test-record.md.' \
  'I read the model fill and discarded inventions.' \
  > .review/SIGNED_BY.txt

The PR body can quote three contract fields. It should not paste the model YAML. Maintainers should not have to read the fill file.

Step 7: Write the maintainer-facing PR body from the contract

Copy facts, not generated style. Keep the body shorter than the contract. Use the same commands as the test record.

Fixes #1234

Reproduction: `cargo test queue::drop_on_cancel -- --exact`
Change: honor cancel on drop in src/queue.rs
Tests: same command passes on the patch SHA
API: none
Revert: revert the fix commit; keep the test commit

This body is boring on purpose. Maintainers can rerun the listed command. They do not need a model summary.

If the tracker wants a checklist, quote contract keys. Do not paste residual model prose. Residual prose belongs in .review/model-fill.yml only.

Decision table

Situation	Action
Bug does not reproduce	Stop. Comment on the issue.
Diff crosses forbidden paths	Split or abandon the patch.
Model fill invents a command	Replace with UNKNOWN and rerun.
Tests pass only on one laptop	Do not open the PR.
Maintainer asks for a squash	Squash after review, not before.
Embargoed security patch	Skip hosted models entirely.

Limitations

This workflow does not prove correctness. It only proves that required fields were filled. UNKNOWN handling still depends on contributor honesty.

YAML parsed by awk is brittle. Nested lists need a real parser. The path check is a demo for three files.

A model can still hallucinate causal language. The contract reduces that risk. It does not delete it.

Hosted models also see the supplied diff. Secret patches do not belong in that prompt. Embargoed security issues stay offline.

Availability of a hosted fill is not a quality benchmark. Duration and hardware are out of scope. Do not treat a free fill host as proof.

The stacked commit rule can annoy squash-only projects. Follow the project guide after review. Do not fight a stated squash policy.

Who should not use this

Do not use this flow for security embargoes. Do not upload embargo diffs to any hosted model. Do not use this for drive-by refactors.

Skip it when the project already has a review schema. Do not run two parallel contracts. Skip it when the contributor cannot reproduce the bug.

New contributors who cannot run the test command should not sign. Signing without a local test run is the failure mode. The attestation is a statement of fact.

Skip it for generated code dumps with no tests. A contract cannot rescue an untested tree. Write the failing test first.

Closing

The useful order is reproduce, patch, test, then contract. Models enter at the contract step. They do not enter at clone time.

A pull request that cannot cite commands is not ready. A model comment that cannot cite those commands is noise.

Keep residual model noise in the YAML file. Keep the commit history only in git.