DEV Community: Blake Yang

Ship an OSS Patch With a Reviewer Question Bank, Not Just a Diff

Blake Yang — Mon, 21 Sep 2026 12:46:43 +0000

A maintainer opened the crash-fix pull request after a long triage shift and found a three-line diff with no reproduction notes. The commit message claimed the parser no longer panicked, yet the issue still lacked a failing command for a clean checkout. Review time vanished into guesswork about callers, error paths, and whether the public API still returned the same sentinel values. The tab closed, and the contributor waited three days for a review that never started.

That stall is rarely about missing brilliance in the patch, and it is almost never about git hygiene alone. Maintainers ration attention across many repositories, and an unexplained diff is expensive even when the change is correct. Contributors who used a coding model to draft the hunks often ship fluent code that still fails the social contract of review. The missing artifact is not more commentary in the PR template; it is a question bank the contributor already answered.

Unexplained diffs stall because review is interrogation

Open-source review is a time-boxed interrogation, not a rubber stamp on a green CI badge. A careful reviewer still needs the reproduction, the moved public symbols, and the error paths that must keep failing. Those demands appear whether or not the contributor writes them down in advance of the request. Writing them first turns the pull request into evidence instead of a riddle wrapped in a unified diff.

Most contribution templates ask for a summary, a test plan, and a checklist of boxes. Those boxes age into ritual, and they do not change when the diff touches an exported error type. A question bank is generated from this branch's paths, headers, and exit codes, so the questions track the patch. The template in CONTRIBUTING.md can stay generic while the bank stays specific to the branch.

The artifact: REVIEWER_QUESTIONS.md

The proposed artifact is a single Markdown file named REVIEWER_QUESTIONS.md that lives beside the patch branch. Each entry is a maintainer question, a required evidence pointer, and a short answer that cites a file and line. Unanswered items remain marked TODO so a local gate can refuse to open the pull request. The file is not a blog post and should stay short enough to read in about twelve minutes.

# Reviewer Question Bank

Project: example-parser
Issue: https://github.com/example/parser/issues/1842
Branch: fix/issue-1842
Base: origin/main

## Q1. Reproduction
- Question: What single command fails on a clean checkout of main and passes on this branch?
- Evidence: TODO
- Answer: TODO

## Q2. Public surface
- Question: Which exported symbols changed name, type, or error contract?
- Evidence: TODO
- Answer: TODO

## Q3. Negative path
- Question: Which input must still fail, and what exit code is preserved?
- Evidence: TODO
- Answer: TODO

## Q4. Regression test
- Question: Where is the test that would have failed before the patch?
- Evidence: TODO
- Answer: TODO

## Q5. Commit shape
- Question: Does each commit encode one logical change a revert could isolate?
- Evidence: TODO
- Answer: TODO

## Q6. Docs and changelog
- Question: What user-visible behavior needs a changelog note, or why is none required?
- Evidence: TODO
- Answer: TODO

Six steps from a failing command to a fail-closed gate

The following workflow is a proposed local checklist, and the commands are illustrative rather than a recorded run against a public crate. Contributors should swap the binary names, fixture paths, and test runners for the project under patch. The point of the checklist is the review packet rather than any particular language or CI toolchain.

1. Freeze the failing command

Before any edit, record one command that fails on a clean checkout of the upstream default branch. Store that command in the question bank under reproduction so later answers cannot drift into a private setup. If the crash needs a fixture, check the fixture into the branch rather than describing it in prose. Reviewers should be able to paste the command without guessing at environment variables or hidden data files.

git fetch origin
git switch -c fix/issue-1842 origin/main
# Proposed: capture the failing invocation, not a narrative.
./target/debug/tool parse tests/fixtures/issue-1842.json
echo "expected non-zero on main"

2. Keep every hunk load-bearing

Edit only the symbols required to make that command pass, and resist drive-by cleanups that enlarge the review surface. A question bank cannot rescue a diff that also reformats a neighboring module or renames an unrelated helper. Split formatting and refactors into separate pull requests if the project will accept them at all. The bank should list every touched path so a maintainer can see the blast without running extra git archaeology.

git diff --name-only origin/main...HEAD
git diff --stat origin/main...HEAD
git log --oneline origin/main..HEAD

3. Collect exit codes instead of vibe screenshots

Run the original failing command, the nearest unit tests, and one negative path that should still error after the fix. Capture the exit codes and a one-line summary rather than pasting entire logs into the pull request body. The question bank needs those exit codes as answers, not as decorative terminal screenshots that hide the command. Label any unexecuted matrix as proposed so a maintainer does not treat a wish as a result.

# Proposed local evidence, not a claimed CI run.
set +e
./target/debug/tool parse tests/fixtures/issue-1842.json
echo "repro_exit=$?"
cargo test parse_issue_1842 -- --nocapture
echo "unit_exit=$?"
./target/debug/tool parse tests/fixtures/truncated.json
echo "negative_exit=$?"

4. Pack the diff and draft questions

A coding model can draft questions from the unified diff, but it cannot invent evidence the tests never produced. Feed the model the diff, the test commands, and the project's CONTRIBUTING snippet, then keep only questions a human can answer with citations.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode's free model access can draft the bank from a unified diff, and the free server option can run the recorded test commands.

# Proposed helper: pack a small context bundle for a coding model.
mkdir -p .review-packet
git diff origin/main...HEAD > .review-packet/patch.diff
git log --oneline origin/main..HEAD > .review-packet/commits.txt
git diff --name-only origin/main...HEAD > .review-packet/paths.txt
python3 scripts/list_fn_headers.py < .review-packet/patch.diff \
  > .review-packet/headers.md

#!/usr/bin/env python3
"""Proposed helper: list added and removed fn headers from a unified diff."""
import sys


def main() -> None:
    added, removed = [], []
    for raw in sys.stdin:
        if raw.startswith("+++") or raw.startswith("---"):
            continue
        if raw.startswith("+") and "fn " in raw:
            added.append(raw[1:].rstrip())
        elif raw.startswith("-") and "fn " in raw:
            removed.append(raw[1:].rstrip())
    print("## Removed headers")
    for row in removed:
        print(f"- `{row}`")
    print("## Added headers")
    for row in added:
        print(f"- `{row}`")


if __name__ == "__main__":
    main()

Proposed prompt for the drafting pass, to be edited by a human before any answer is trusted:

You are drafting reviewer questions for an OSS patch, not writing the patch.
Read patch.diff, headers.md, and commits.txt.
Propose at most eight questions a maintainer would ask before merging.
Reject style nits. Prefer reproduction, public surface, negative paths,
tests, commit shape, and changelog impact.
Do not invent test results. Leave every Answer and Evidence as TODO.

5. Fill answers with file:line citations

Each answer should be one or two sentences and must point at a path a reviewer can open. Saying tests pass is not an answer, because it hides which command and which assertion moved. If the model drafted a question the tests cannot answer, delete the question instead of fabricating a citation. A short honest bank beats a long bank that launders missing evidence behind confident, fluent prose.

## Q1. Reproduction
- Question: What single command fails on a clean checkout of main and passes on this branch?
- Evidence: tests/fixtures/issue-1842.json, src/parse.rs:214
- Answer: `./target/debug/tool parse tests/fixtures/issue-1842.json` exited 101 on main and 0 on HEAD.

## Q2. Public surface
- Question: Which exported symbols changed name, type, or error contract?
- Evidence: src/lib.rs:88 (still `pub fn parse`)
- Answer: No exported name changed; `parse` still returns `Result<Doc, ParseError>`.

## Q3. Negative path
- Question: Which input must still fail, and what exit code is preserved?
- Evidence: tests/fixtures/truncated.json, src/parse.rs:240
- Answer: Truncated input still exits 1 with `ParseError::UnexpectedEof`.

6. Refuse to open the pull request while TODO remains

A local gate keeps the social contract from depending on memory when the pull request is finally opened. The script below is proposed, and it only searches for the TODO markers the template uses. Contributors can wire it as a pre-push hook or as a make target named review-packet. Opening the pull request then becomes a copy of the bank into the PR body, not a second writing task.

#!/usr/bin/env bash
# Proposed gate: refuse to open a PR while the bank still has TODO answers.
set -euo pipefail
BANK="REVIEWER_QUESTIONS.md"
if [[ ! -f "$BANK" ]]; then
  echo "missing $BANK" >&2
  exit 1
fi
if grep -E '^- (Answer|Evidence): TODO' "$BANK"; then
  echo "question bank is incomplete" >&2
  exit 1
fi
echo "question bank is complete"
# Proposed: only run after the gate passes.
# gh pr create --fill --body-file REVIEWER_QUESTIONS.md

Pass rules for each required question

Question	Required evidence	Pass rule
Reproduction	command plus exit codes	Fails on `origin/main`, passes on `HEAD`
Public surface	exported names from the diff	No undocumented rename or signature change
Negative path	one input that must still fail	Non-zero exit and original error type preserved
Regression test	test path and assertion line	Test failed before the patch and passes after
Commit shape	`git log --oneline`	One logical change per commit, revertible
Docs	changelog, rustdoc, or skip reason	Skip only for internal helpers with no user behavior

Limitations

The bank does not replace a maintainer who knows the crate's history, and it will not catch semantic bugs the tests never expressed. Models often over-generate questions about style and under-generate questions about concurrency, ABI, and platform quirks. A free server run is only as good as the command the contributor recorded in the first step. Teams that already require a formal design review should treat this file as a packet, not as a substitute for that review.

Generated questions also drift when the diff includes vendored code, snapshots, or generated protobuf bindings. In those cases the header lister will emit noise, and a human should delete those rows before the drafting pass. The gate only proves that answers exist as text; it does not prove that the cited lines implement the claim. Lying in the bank is still possible, so maintainers should sample at least one citation against the working tree.

Who should skip this packet

Tiny typo pull requests do not need a twelve-minute packet, and the process would look like ceremony. Security issues that cannot discuss the exploit path in public should stay in the project's private disclosure channel. Contributors who cannot run the failing command at all should not invent passing answers for a server they never invoked. Projects that forbid model-assisted patches in CONTRIBUTING.md should skip the drafting step and fill the bank by hand, or skip the bank.

Typo-only or comment-only pull requests
Private security disclosures that cannot describe the failure mode
Patches the contributor cannot execute on a clean checkout
Repositories that ban model-assisted contributions

The next time a crash fix looks ready, the contributor can spend twenty minutes answering the questions a tired maintainer will ask anyway. The resulting packet does not make the patch correct, but it makes incorrectness cheaper to see. A filled bank also gives a coding model a narrower job than rewriting a module until continuous integration turns green. That narrower job is the difference between a reviewable patch and a fluent diff that still stalls for days.

A One-Command Reproducer Kit That Survives a Clean OSS Checkout

Blake Yang — Sun, 20 Sep 2026 10:57:53 +0000

A contributor opened a pull request against a popular HTTP client with a three-line timeout fix and a screenshot. Maintainers could not reproduce the hang on CI, so the branch sat idle across two review cycles. The hidden cause was a leftover HTTP_PROXY export in a shell profile plus a vendored snapshot that no longer matched main. The patch itself was reasonable; the missing artifact was a reproduction kit that survived a clean clone.

Industry chatter in mid-September 2026 keeps returning to the same failure mode: generated diffs look complete while the engineering work around them is skipped. Open-source review is where that gap becomes public. A maintainer cannot merge a story about a laptop. A maintainer can merge a command that fails the same way on a fresh tree.

Dirty trees hide the actual bug

Local developer machines accumulate exports, toolchains, cached wheels, and half-applied patches that never exist on a maintainer laptop. A failure that depends on that residue will vanish the moment someone clones the repository into an empty directory. Maintainers then spend review time debugging the reporter's environment instead of the library. A clean-clone gate makes that class of confusion expensive for the contributor and cheap for the project.

Screenshots, truncated logs, and "it works on my machine after I source .envrc" notes do not travel. They also do not bisect. The kit described here is the first deliverable of an OSS bugfix, not an afterthought pasted under the diff. The patch comes only after the kit fails for the same reason in two isolated trees.

What the kit must contain

A useful kit is not a paragraph of remembered steps inside the issue form. It is a script, a pinned revision, an expected failure, and a short environment dump that another person can run without extra narrative. The kit should fail for the same observable reason on two machines before any production patch is written. The following items are the minimum set for most application-level bugs in published libraries:

pinned upstream remote plus a commit SHA or release tag
one command that exits non-zero while the bug is present
captured tool versions for the language, package manager, compiler, and OS
explicit unset of proxy, credential, and vendor-cache variables
a recorded last-known-good tag when the reporter already has that information
a short expected-versus-actual block that a stranger can compare by eye

The kit is not a substitute for the project's own test suite. It is a passport that lets a stranger reach the failing assertion without inheriting the reporter's shell.

A one-command template

The script below is a labeled template, not a captured run from a specific repository. Operators should replace the remote, SHA, language toolchain, and failing invocation with values from the issue they are actually working. The important property is isolation: a temporary directory, a fresh clone, and an environment that starts empty of proxy and virtualenv hints.

#!/usr/bin/env bash
# repro_kit.sh — labeled template for a clean-clone OSS reproducer
set -euo pipefail

REMOTE="${REMOTE:-https://github.com/example/http-client.git}"
SHA="${SHA:-a1b2c3d4e5f60718293a4b5c6d7e8f9012345678}"
WORKDIR="$(mktemp -d /tmp/oss-repro.XXXXXX)"
trap 'rm -rf "$WORKDIR"' EXIT

# Strip common local residue that maintainers will not have.
unset HTTP_PROXY HTTPS_PROXY ALL_PROXY http_proxy https_proxy all_proxy || true
unset VIRTUAL_ENV PYTHONPATH NODE_PATH GOPATH GOFLAGS CARGO_HOME || true
export PATH="/usr/bin:/bin:/usr/local/bin"

echo "workdir=$WORKDIR"
echo "uname=$(uname -a)"
command -v python3 >/dev/null && python3 --version || true
command -v git >/dev/null && git --version

git clone --quiet "$REMOTE" "$WORKDIR/src"
cd "$WORKDIR/src"
git checkout --quiet "$SHA"
echo "HEAD=$(git rev-parse HEAD)"
echo "describe=$(git describe --always --dirty)"

# Project-specific install. Keep it deterministic; avoid user-site packages.
python3 -m venv .venv
# shellcheck disable=SC1091
source .venv/bin/activate
python -m pip install -e . -q

# The single failing command. Replace with the issue's minimal trigger.
set +e
python - <<'PY'
from client import Session

s = Session(timeout=0.05)
try:
    s.get("https://example.invalid/slow")
    raise SystemExit("expected timeout, got success")
except Exception as exc:
    name = type(exc).__name__
    if name != "Timeout":
        print(f"actual_exception={name}: {exc}")
        raise SystemExit(2)
    print(f"reproduced:{name}")
    raise SystemExit(1)
PY
status=$?
set -e
echo "repro_exit=$status"
exit "$status"

A passing mental check for this script is simple. A stranger with network access and the language toolchain should reach the same non-zero exit without reading the pull request. If the script needs a paragraph of preamble, the kit is still a blog post and not yet a kit.

Expected versus actual, in the repo

Issue threads lose formatting and lose files. Checking a tiny sidecar into the branch keeps the contract next to the script. The file below is also a template; the strings must come from a real run, not from memory of the original crash.

# repro_kit.expect
remote: https://github.com/example/http-client.git
sha: a1b2c3d4e5f60718293a4b5c6d7e8f9012345678
command: python repro_min.py
expected_exit: 1
expected_stderr_contains: reproduced:Timeout
forbidden_stderr: HTTP_PROXY
last_known_good_tag: v2.4.1

Store the script and the expect file in a directory that reviewers can delete after merge, such as repro/issue-8412/. Do not hide them inside a personal gist that will rot. The pull request body should link to the path and paste the last local run, including HEAD and repro_exit.

Decision table: when the kit is honest

Use the table before writing production code. Each row is a gate. A "no" in the remote column means the bug is still a local myth, even if the laptop demo looks perfect.

Observation after `repro_kit.sh`	Local dirty tree	Fresh `/tmp` clone	Remote clean machine	Action
Same non-zero exit and same exception name	yes	yes	yes	Write the patch; kit is load-bearing
Fails locally, passes on fresh clone	yes	no	no	Stop; inspect exports, caches, extra remotes
Fails on fresh clone, passes on remote	yes	yes	no	Record OS, CPU, and filesystem; do not send a Linux-only guess
Passes everywhere, including the reported SHA	no	no	no	The issue is closed or mis-filed; do not invent a fix
Fails with different exceptions across machines	yes	yes	yes	Narrow the trigger; the kit is still too wide
Requires secrets, production data, or a private registry	yes	maybe	maybe	Do not publish the kit; use a redacted fixture or a private maintainer channel

The interesting row is the second one. That is the screenshot pull request. The clean clone is the cheapest way to discover it before a maintainer does.

Run the kit where the laptop cannot help

A second machine is not a luxury for this workflow. It is the control group. Continuous integration on the fork is one control group, provided the workflow file does not reuse caches that hide the bug. A disposable remote workspace is another, provided it starts without the reporter's direnv, keychain, or Docker layer cache.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode offers free model access and a free server option that can host a throwaway clone for this control-group run. The models are useful later for reviewing the kit text; the server is useful now because it does not inherit the laptop's residue. No quota, model catalog, or hardware profile is claimed here, because those details change and do not belong in a frozen article.

A practical remote pass looks like the following sequence. Clone into a new directory, copy only repro_kit.sh and repro_kit.expect, and refuse to copy .env, venv, or node_modules. Run the script once, capture stdout and stderr to a file, and attach that file to the issue. If the remote exit code disagrees with the laptop, the patch waits.

# Labeled example: capture a remote control-group run.
chmod +x repro_kit.sh
./repro_kit.sh > repro_remote.log 2>&1
echo "remote_exit=$?" | tee -a repro_remote.log
grep -E '^(HEAD|describe|repro_exit|reproduced:|actual_exception=)' repro_remote.log

Free-model review of the kit, not the vibes

Generated patches often read as confident while the reproduction story remains sloppy. The safer use of a free coding model in this workflow is editorial, and it happens before the production diff exists. Paste the script, the expect file, and the draft issue comment. Ask for a maintainer-facing review of clarity, not for a speculative fix.

A prompt that stays inside that boundary looks like the block below. It is labeled, because it is a proposal rather than a log from a particular model run.

You are reviewing an OSS reproduction kit, not writing a patch.
Point out missing pins, leftover environment coupling, and claims
that a stranger cannot verify with one command. List concrete edits
to repro_kit.sh, repro_kit.expect, and the issue comment. Do not
propose production code. Do not invent versions I did not supply.

Treat the model output as a diff against the kit. Keep every edit that removes hidden state, vague verbs, or un-pinned installs. Discard advice that adds a rewrite of the library. If the model wants to "just catch Timeout and retry," the workflow has already failed, because the task was documentation of the bug, not product design.

After the kit is tightened, a second short pass can check the pull request body for maintainer time costs. Useful flags include missing SHA, missing exit code, commands that assume a particular absolute path, and logs that still contain home-directory fragments. Those are review comments the reporter can fix without waiting for a human round trip.

Only then write the patch

Once local fresh-clone and remote clean-machine runs agree, the patch has a job: make repro_kit.sh exit zero for the right reason. Keep the kit on the branch until reviewers say otherwise. A green kit plus a green project test suite is a stronger story than a green suite alone, because the suite may never have covered the reported path.

Commit shape still matters. One commit should add the kit and show the failure on the parent SHA if the project's CI will allow a known-failing commit. The next commit should contain the fix and the permanent regression test that will remain after repro/ is deleted. Do not squash away that history if the project likes bisectable series. The kit is evidence; the regression test is the long-term contract.

# Labeled local sequence after both environments agree on the failure.
git checkout -b issue-8412-timeout-repro
git add repro/issue-8412/repro_kit.sh repro/issue-8412/repro_kit.expect
git commit -m "repro: add clean-clone kit for issue 8412"

# ... implement the library fix and a permanent test ...
git add src tests
git commit -m "fix: surface Timeout when the deadline is already expired"

If the permanent test cannot be written without the kit's private fixture, the fixture needs to be reduced until it can live under tests/. A kit that cannot be turned into a project test is still valuable for triage, but it should not be the only safety net after merge.

Limitations

This workflow assumes the bug is deterministic enough to fail a script twice. Flaky timing bugs, GPU-only faults, and races that need hours of load will not honor a one-command kit. Hardware-specific failures need a documented machine profile that a free generic server will not provide. The clean-clone gate also assumes the project can be built by a stranger from a tag; monorepos that require internal Bazel seeds or private wheels need a reduced public fixture first.

The workflow is the wrong tool for embargoed security reports. A public reproducer kit that demonstrates a practical exploit does not belong on a fork. Those issues follow the project's security policy, which often means a private channel and a delayed disclosure, not a repro/ directory on a personal branch. License and CLA constraints are also out of scope for the script; they still block the pull request even when the kit is perfect.

Free model access will not know whether the exception name is stable across minor tags. Free remote machines will not magically match an affected user's older distro. Both are controls against local dirt, not oracles. When the table's remote column cannot be filled, say so in the issue instead of implying a complete matrix.

Who should skip this approach

Drive-by typo fixes, documentation-only pull requests, and issues already failing on upstream CI do not need a personal reproducer kit. Maintainers who already pasted a failing job URL have done the isolation work. Contributors without permission to clone the code, or without a legal right to redistribute a fixture, should stop at a redacted description and wait for maintainer instructions.

Teams that punish extra files on a branch may prefer a gist or a CI job, but the isolation rules stay the same. The method is for reporters who are about to change shared behavior and who need a stranger to believe the bug exists. Readers who want a disposable machine for the control-group step can use MonkeyCode's free server option with its free model access, then keep the kit in the pull request regardless of where it was run.

Classify the Public Surface Before an OSS Crash Fix Hits Review

Blake Yang — Sat, 19 Sep 2026 09:15:48 +0000

Consider a mid-size TypeScript CLI that received a three-line pull request stopping a null crash in flag parsing. The reporter attached a passing unit test, a screenshot of the local run, and a short stack note. Maintainers still closed the request after one pass because an exported helper changed return values for empty argv. Downstream packages had treated that helper as a stable contract, so the crash fix was a breaking change wearing patch-sized clothing.

Correctness is not the only gate on an upstream pull request for libraries that already have downstream dependents in late 2026. Maintainers also protect exported names, documented flags, config keys, and the promise implied by the next version tag. A contributor who only proves the bug is gone still leaves contract classification sitting on the reviewer. The packet below keeps a human in charge of reproduce, patch, and test, then records a Semver Impact Card before anyone files the diff.

Why small crash fixes still stall on versioning

Issue threads reward a green test and a short, readable diff against the reported file. Versioning rewards a boring, named claim about what downstream code may keep calling after the next tag. Those two rewards pull in opposite directions when a crash lives inside a helper that already ships in __all__, export, or a public header. The helper can start returning a documented empty object instead of throwing, which is kinder to interactive users and still a meaning change for callers that caught the exception.

Review time then migrates from the hunk to an argument about frozen intent. Maintainers compare README examples, type stubs, changelog footnotes, and the last tagged tarball. Contributors repeat that the crash was never written down as a feature. The Semver Impact Card moves that argument into a table before the pull request exists, so the first human review starts on a classified claim rather than on informal confidence.

Four stages before `gh pr create`

The working sequence is reproduce, patch, test, and classify. Each stage writes a file under .patch-packet/ so the pull request can link evidence instead of retelling the narrative. Model output appears only after the first three stages are green on a clean tag, because a model that reviews a still-failing tree invents compatibility stories.

Reproduce the reported failure on the issue tag, not on a dirty local main.
Patch the owning module without drive-by refactors or formatting sweeps.
Run the project's test runner plus one new assertion that binds the contract.
Fill the Semver Impact Card and invite a model to falsify the proposed bump.

Stage 1: Reproduce on the reported tag

Check out the tag named in the issue, not a local branch that already contains extra commits. A dirty tree hides whether upstream already shipped a fix, which makes any compatibility claim dishonest. Record the command, expected output, and observed output in repro.md beside a tiny shell wrapper that exits non-zero on the broken tag.

git fetch --tags origin
git switch --detach v4.2.1
python -m venv .venv && source .venv/bin/activate
pip install -e ".[dev]"
mkdir -p .patch-packet

cat > .patch-packet/repro.sh <<'EOF'
#!/usr/bin/env bash
set -euo pipefail
# Documented contract: empty argv yields an empty object and exit 0.
# Observed on v4.2.1: TypeError from parse_argv([]).
python -m samplecli parse --argv '' --format json
EOF
chmod +x .patch-packet/repro.sh
./.patch-packet/repro.sh; echo "exit=$?"

The wrapper must fail on the tagged release and later pass on the patched tree. If it cannot fail on that tag, the issue is not isolated enough for a compatibility claim, and the card should not be filled yet.

Stage 2: Patch the owning module only

Keep the diff inside the file that owns the crash path. Cross-cutting cleanups belong in a follow-up issue, not in a crash fix that also claims a version bump. After the edit, store a unified diff and a name-only file list next to the card so later review does not depend on a working tree.

git diff --stat
git diff -U3 -- src/samplecli/parse.py > .patch-packet/patch.diff
git diff --name-only > .patch-packet/files.txt

A crash fix in this shape is usually a null guard plus a default that already appears in the docs. If the hunk set grows past the failing function, split the work before filling the card, because extra files inflate the public-surface table without extra evidence.

Stage 3: Bind the new behavior with the project runner

Call the same runner the CI workflow already uses. Add one assertion that encodes the old crash input and the intended contract, then keep neighboring tests unmodified unless they encoded the crash as expected behavior. Do not introduce a second framework for a single upstream patch, because reviewers will not install it.

# tests/test_parse_empty_argv.py
from samplecli.parse import parse_argv


def test_empty_argv_returns_empty_object_without_raising():
    result = parse_argv([])
    assert result == {"flags": {}, "positionals": []}
    assert "help" not in result["flags"]

pytest -q tests/test_parse_empty_argv.py tests/test_parse.py
git add tests/test_parse_empty_argv.py src/samplecli/parse.py

Green tests still do not prove that exported meaning stayed compatible for downstream imports. That proof is the card in the next stage, not another assertion about private helpers or log lines.

Stage 4: Classify public surface before the pull request

Copy the template into .patch-packet/semver-impact.md and complete every field on the same day as the diff. Blank cells are a stop for the contributor, not a reminder for the maintainer to finish the taxonomy. The proposed bump must cite the project's own versioning document, even when that document is only a short README section.

Artifact: the Semver Impact Card

# Semver Impact Card

- Project / issue: samplecli#1842
- Base tag: v4.2.1
- Proposed bump: PATCH | MINOR | MAJOR
- Versioning source: VERSIONING.md (commit abc1234) or README "Versioning"

## Public surface touched
| Kind | Name | Before | After | Visible to downstream? |
| ---- | ---- | ------ | ----- | ---------------------- |
| function | parse_argv | raises TypeError on [] | returns empty object | yes, listed in __all__ |
| CLI flag | --format | unchanged | unchanged | yes |
| config key | (none) | | | |
| file format | (none) | | | |

## Classification rules used
- MAJOR: remove or rename an export, change return meaning, or reject previously valid input
- MINOR: add an export or optional flag whose default preserves old calls
- PATCH: fix a crash or wrong result without changing documented meaning

## Claim
PATCH. `parse_argv([])` now matches the documented empty-object example in README.md.
The TypeError was never listed as a contract in VERSIONING.md or the type stubs.

## Explicit non-claims
- Does not add --strict
- Does not change JSON key order
- Does not backport to the 3.x line

## Tests that bind the claim
- tests/test_parse_empty_argv.py (new)
- tests/test_parse.py (unchanged, still green)

## Items for maintainer confirmation
- Confirm parse_argv is public because __all__ includes it
- Confirm the README example, not the TypeError, is the frozen contract

The table is the artifact a maintainer can scan without opening every hunk. The non-claims section matters as much as the bump, because it blocks extra behavior from sneaking into the pull request title. Store the filled card in git even if the project will not merge the .patch-packet/ directory, then paste a summary into the pull request body.

Extract candidate exports from the unified diff

Hand-filled tables miss aliases, re-exports, and definitions added only as +export. A small illustrative script greps added definition lines, then checks __all__ on the current file. The output is a candidate list for the card, not a legal API boundary and not a substitute for reading VERSIONING.md.

#!/usr/bin/env python3
"""semver_scan.py — list definition lines in a unified diff (illustrative)."""
from __future__ import annotations

import re
import sys
from pathlib import Path

DEF_RE = re.compile(
    r"^\+\s*(?:export\s+)?(?:async\s+)?(?:def|function|class|const|exports\.)\s*([A-Za-z_][\w.]*)"
)
ALL_RE = re.compile(r"__all__\s*=\s*\[([^\]]*)\]", re.S)


def definitions(diff: str) -> list[str]:
    found: list[str] = []
    for line in diff.splitlines():
        match = DEF_RE.match(line)
        if match:
            found.append(match.group(1))
    return found


def public_all(py_source: str) -> set[str]:
    match = ALL_RE.search(py_source)
    if not match:
        return set()
    return {tok.strip(" \"'") for tok in match.group(1).split(",") if tok.strip()}


def main() -> None:
    diff = Path(sys.argv[1]).read_text(encoding="utf-8")
    names = definitions(diff)
    print("definitions_added_or_modified:")
    for name in names:
        print(f"  - {name}")
    if len(sys.argv) > 2:
        src = Path(sys.argv[2]).read_text(encoding="utf-8")
        pub = public_all(src)
        print("in__all__:")
        for name in names:
            print(f"  - {name}: {name in pub}")


if __name__ == "__main__":
    main()

python .patch-packet/semver_scan.py .patch-packet/patch.diff src/samplecli/parse.py

Label the script illustrative, because languages disagree about the word public. Go uses capitalization, Rust uses pub and pub use, C uses installed headers, and TypeScript uses export. The card must accept or reject each candidate by name, including names the scanner never saw because they live only in documentation examples.

Falsify the bump with a free model after tests pass

A language model is a weak author of production hunks and a useful adversary for a versioning claim. The human still reproduces the crash, writes the change, and runs the project test runner. The model receives only the card, the unified diff, and VERSIONING.md, then tries to show that the bump is too low, too high, or unsupported by the files.

Contributors without a private GPU can run that second pass on a free model endpoint. MonkeyCode currently offers free model access and a free server option for this kind of review host. Disclosure: This article was prepared as part of MonkeyCode's product outreach. Teams that want a hosted loop for the review prompt can use the free server option as one of several possible hosts. The product is not required; any local endpoint that accepts a text prompt can execute the same checklist against the packet files.

Save the prompt as .patch-packet/review-prompt.md and paste the card plus git diff --name-only under a clear delimiter.

You are reviewing a Semver Impact Card for an upstream pull request.
Do not rewrite the patch. Do not invent symbols that are absent from the diff.

Inputs:
1) semver-impact.md
2) git diff --name-only and the unified diff
3) VERSIONING.md or the README versioning section

Tasks:
- List every added, removed, or retyped name that looks exported.
- State whether the proposed bump is too low, too high, or consistent.
- Quote the rule from VERSIONING.md that supports that judgment.
- Flag any line in Explicit non-claims that the diff already implements.
- Return at most 12 lines with sections: missed_exports, bump_judgment, rule_quote, overclaims.

If evidence is missing, write insufficient evidence instead of guessing.

Run the prompt only after the wrapper fails on the base tag and passes on HEAD. Store the output in .patch-packet/model-review.txt and treat disagreements as contributor work, not as automatic edits to parse.py. A model that rewrites the hunk at this stage undoes the whole point of classifying a human patch.

Decision table for the proposed bump

Observation on the tagged tree	Typical bump	Stop filling the card if
Crash on input the docs already allow	PATCH	docs and tests disagree about the input
New optional flag with a preserving default	MINOR	the default changes old output
Return type or error class changes for valid input	MAJOR	the issue asked only for a crash fix
Rename of an exported helper	MAJOR	no alias remains for one release
Internal function only, absent from docs and `__all__`	PATCH or no release	the name appears in a public example

The table is a starting filter for contributors who do not yet know the project's customs. The project's VERSIONING.md wins when it conflicts with this chart, and the card should quote the winning sentence. When VERSIONING.md is missing, the card should say so in the versioning-source field and propose PATCH only for undocumented crash behavior that existing tests already allowed.

Sample pull request body

## Summary
Crash on empty argv in parse_argv. Packet claims PATCH against v4.2.1.

## Evidence
- Repro: .patch-packet/repro.sh fails on v4.2.1, passes on this branch
- Test: tests/test_parse_empty_argv.py
- Surface: .patch-packet/semver-impact.md
- Adversarial pass: .patch-packet/model-review.txt (informational only)

## Maintainer confirmation requested
Please confirm parse_argv is public and that README empty-object example is the contract.

Keep the model report informational. Maintainers did not ask for a generated review persona, and a contributor should not present model prose as CI.

Limitations

The scanner does not understand re-exports, decorator-wrapped APIs, or documentation-only contracts. A README sample can freeze behavior that never appears in __all__, and plugin entry points can expose names that live only in packaging metadata. Generated bindings need a human pass over the installed artifact, not merely over the source diff. Model review can miss domain meaning, such as a numeric default that looks compatible but breaks a wire protocol or a sort order that tests never pinned.

Free model access and a free server option do not make the review authoritative. They do not replace maintainer judgment, signed tags, or the project's CI matrix. The card also fails for embargoed security fixes that must not describe public surface in a public packet. Those patches follow the project's security policy rather than this template, and they should not be sent to a third-party endpoint.

Time-sensitive product quotas, model catalogs, and hardware lists change often and are omitted here on purpose. Operators should confirm current availability on the surface they actually use, rather than copying capacity claims from a blog post.

Who should skip this packet

Skip the card for typo-only documentation edits and for private forks with no downstream consumers. Skip it for contributors who have not reproduced the failure on a clean tag, because there is no honest base for a bump. Skip model review when the repository forbids sending source to third-party endpoints, including patches that contain secrets, customer data, or undisclosed vulnerabilities. Skip the whole packet when the maintainer template already requires a changelog with the same fields, because duplicate paperwork helps nobody and slows the merge.

What the pull request should link

A pull request that links the repro wrapper, the new test, semver-impact.md, and model-review.txt gives reviewers a contract claim they can accept or correct in one pass. The patch can still be wrong on performance or style, but the argument is about a named export and a named bump. Engineering work stays in reproduce, patch, and test; the model only pressure-tests the written claim after those stages are already green.

Declare the Behavior Delta Before Maintainers Open an OSS Bugfix

Blake Yang — Fri, 18 Sep 2026 07:11:15 +0000

A contributor cloned a popular CLI, reproduced a parser crash, and typed a confident patch in one sitting. The pull request touched a helper, two tests, and a changelog line that promised a complete fix. Maintainers opened the diff and still could not tell which captured outputs were supposed to change. Screenshots arrived later, then a second patch, then a request for logs the contributor had already discarded.

That stall is common when a bugfix is treated as a narrative instead of a measurable delta. Reviewers need the command, the before output, the after output, and a short allowlist of intended change. Without those four pieces, a green local run is only a private memory of one machine. The workflow below turns that memory into files a distant maintainer can replay without extra archaeology.

Treat the patch as a declared behavior delta

An OSS bugfix is not finished when the process exits zero on one laptop. The finished unit is a declared delta: old behavior, new behavior, and the exact invocation that produces both. Code review then checks whether the git hunks are necessary for that delta, rather than guessing intent from renamed helpers. This framing also limits accidental drive-by refactors that hide inside a supposedly small fix.

A useful delta statement is boring, local, and specific enough to fail in public. It names the binary or test target, the fixture, and the lines that must differ after the patch. It also names the outputs that must stay byte-identical so collateral churn cannot hide. Maintainers can reject a patch that cannot produce that statement in a few quiet minutes.

The oracle packet

Keep the packet beside the worktree, not inside a chat log that will rot. A simple directory is enough for most CLI tools and library bugs that fail in user space.

oracle/
  README.md
  invoke.sh
  before.txt
  after.txt
  before.exit
  after.exit
  allowlist.txt
  observed.diff
  check_allowlist.py
  check_oracle.sh

Each file has one job, and mixing those jobs is how packets become essays. invoke.sh is the only entry point a stranger should need to execute. before.txt and after.txt are raw captured bytes, not edited summaries of what the contributor remembers. allowlist.txt lists the fragments that are allowed to differ, and check_oracle.sh fails when the live diff escapes that list.

README.md as the human index

# Oracle packet: parse crash on empty --config

- Upstream issue: #4182
- Failing commit used for the before capture: 9f3c1aa
- Invocation: ./oracle/invoke.sh
- Intended delta: stderr gains a one-line usage error; process status becomes 2
- Forbidden delta: stdout of `--help` and the JSON fixture under tests/data/

The README is an index so a maintainer can skip archaeology, not a second design document. Link the issue, pin the commit, and state the delta in one plain sentence. Leave motivation and design opinions in the pull request body, where discussion belongs.

Step 1: freeze the failing invocation

Do not start in the editor. Write the invocation that currently fails, including working directory and environment variables that the tool actually reads. If the bug needs a fixture, generate that fixture in the same script so the packet stays closed.

#!/usr/bin/env bash
# oracle/invoke.sh
set -euo pipefail
ROOT="$(cd "$(dirname "$0")/.." && pwd)"
cd "$ROOT"

export PYTHONUNBUFFERED=1
# Status is recorded below; do not abort the script on the expected failure.
set +e
python -m tools.parser --config tests/data/empty.toml \
  >oracle/stdout.txt 2>oracle/stderr.txt
echo "$?" > oracle/exit_code.txt
set -e

Record status, stdout, and stderr as separate streams when those streams matter to the bug. Mixing them in a terminal screenshot is how reviews lose the only signal they can replay. If a test runner can target one case, prefer that invocation over a manual CLI session with hidden shell history.

Step 2: capture the before side without editing it

Run the frozen invocation on the parent commit that still contains the bug. Save the streams before any local patch exists in that tree. Resist the urge to trim timestamps or absolute paths by hand, because silent edits make the after file incomparable.

git switch --detach 9f3c1aa
./oracle/invoke.sh
cp oracle/stderr.txt oracle/before.txt
cp oracle/exit_code.txt oracle/before.exit

If logs embed clocks, hostnames, or random request ids, freeze those sources in invoke.sh before treating the files as oracles. A packet that cannot be replayed tomorrow is a diary entry, not evidence. Characterization of the failure belongs in the captured bytes, not in a paraphrase of a stack trace.

Step 3: apply the smallest patch that should move the delta

Copy before.txt and before.exit to a safe path if the working tree will rerun the same script. Then restore a writable branch and change only what the issue asked to change. Do not fold formatter noise, comment rewrites, or unrelated helper cleanups into the same commit, because those extra hunks make the allowlist dishonest even when the crash is gone.

git switch --create fix/empty-config 9f3c1aa
# edit tools/parser.py and the focused test only
git diff --stat

A short self-check before capture keeps the later allowlist honest:

Every edited path is named in the issue or required by the failing invocation.
No lockfiles, generated docs, or IDE metadata slipped into git status.
The changelog line, if required by the project, matches the declared delta and nothing else.

Step 4: capture the after side with the same invocation

Use the identical invoke.sh. If the command line changes, the packet is invalid and the README must be rewritten first. Copy the new streams into after.txt and after.exit, then produce a unified diff the checker can read.

./oracle/invoke.sh
cp oracle/stderr.txt oracle/after.txt
cp oracle/exit_code.txt oracle/after.exit
diff -u oracle/before.txt oracle/after.txt > oracle/observed.diff || true

Read observed.diff as a product artifact, not as debug scrap from a local terminal. If the diff is empty, the patch did not change the failing behavior and review is premature. If the diff is huge, the patch changed more than the issue asked for and the allowlist will not save it.

Step 5: publish an allowlist, not a vibe

The allowlist is the original technical artifact of this workflow. It states which observed changes are load-bearing for the issue. Everything else is a defect in the packet, the invocation, or the patch itself.

# oracle/allowlist.txt
# One fragment per line. Lines starting with # are comments.
# EXIT: lines record an intended status change.
+error: empty --config is not a valid document
-Traceback (most recent call last):
EXIT: 1 -> 2

The checker below is a proposal for local use, not a harvested suite from production CI. It treats uncovered unified-diff lines as a failed packet, which is stricter than a glance at git diff.

#!/usr/bin/env python3
"""Fail if oracle/observed.diff contains lines not covered by allowlist.txt."""
from pathlib import Path
import sys

root = Path(__file__).resolve().parent
observed = (root / "observed.diff").read_text(encoding="utf-8", errors="replace")
allow = []
exit_rule = None
for raw_line in (root / "allowlist.txt").read_text(encoding="utf-8").splitlines():
    line = raw_line.strip()
    if not line or line.startswith("#"):
        continue
    if line.startswith("EXIT:"):
        exit_rule = line
        continue
    allow.append(line)

uncovered = []
for raw in observed.splitlines():
    if not raw.startswith(("+", "-")) or raw.startswith(("+++", "---")):
        continue
    text = raw[1:]
    if not any(pat in text for pat in allow):
        uncovered.append(raw)

before_exit = (root / "before.exit").read_text(encoding="utf-8").strip()
after_exit = (root / "after.exit").read_text(encoding="utf-8").strip()
if exit_rule:
    expected = exit_rule.split(":", 1)[1].strip()  # e.g. "1 -> 2"
    actual = f"{before_exit} -> {after_exit}"
    if expected != actual:
        print(f"exit allowlist expected {expected!r}, got {actual!r}")
        sys.exit(1)
elif before_exit != after_exit:
    print("exit code changed but allowlist.txt has no EXIT: rule")
    sys.exit(1)

if uncovered:
    print("oracle allowlist missed these diff lines:")
    print("\n".join(uncovered))
    sys.exit(1)
print("oracle allowlist covered the observed diff")

Wrap the checker so a maintainer runs one script after cloning the branch.

#!/usr/bin/env bash
# oracle/check_oracle.sh
set -euo pipefail
cd "$(dirname "$0")"
test -s before.txt
test -s after.txt
python3 check_allowlist.py

Adjust the exit-code rule to the issue rather than copying the sample. Some fixes must keep the same status and only change a message, and the allowlist should say so in words the checker can enforce.

Step 6: review the packet against the git hunks

Once the files exist, a second reader can compare three artifacts that humans often keep in separate windows. The first is observed.diff. The second is allowlist.txt. The third is git diff for the actual source change. Contradictions among those three are the review, not whether the patch looks tidy in isolation.

A narrow prompt keeps that second reader on the packet. The block below is a template, not a transcript of a run performed for this article.

You are reviewing an OSS bugfix packet, not writing new code.
Inputs: issue excerpt, git diff, oracle/before.txt, oracle/after.txt,
oracle/allowlist.txt, oracle/observed.diff, oracle/invoke.sh.
Report only:
1. Allowlist claims that do not appear in observed.diff
2. Observed.diff lines that the allowlist does not mention
3. Git hunks that cannot be tied to the declared delta
4. Steps in invoke.sh that look non-deterministic (time, net, random)
Do not suggest extra features. Do not rewrite the patch.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. Free model access can run that constrained pass over the packet, and the free server option can replay invoke.sh plus check_oracle.sh on a machine that is not already loaded with the contributor's daemons. The packet remains the useful artifact if both of those options are skipped.

Local caches, language shims, and half-installed extras are a frequent source of unreproducible fixes that only exist on one laptop. A clean clone that reproduces before.txt and then after.txt is part of the proof. If a shared server is used for that clone, scrub host paths, tokens, and customer fixtures from the oracle files before they leave the machine.

Decision table for the packet

Signal in the packet	Meaning	Action before opening a PR
`observed.diff` is empty	The patch did not move the failing behavior	Keep debugging; do not request review
Diff is large, allowlist is short	Hidden refactors or noisy logs	Split the commit or quiet the invocation
Allowlist mentions lines absent from the diff	The declaration is aspirational	Rewrite the allowlist against real bytes
Exit code unchanged when the issue needs a status change	Incomplete fix	Add an `EXIT:` rule and recapture
Invocation embeds timestamps or absolute home paths	Replay will rot on another host	Fix `invoke.sh` before any model pass

Limitations

This workflow assumes a deterministic invocation. Flaky tests, live network calls, and clocks inside log lines will poison before.txt and after.txt on the second run. Binary fixtures and huge snapshot dumps also make the packet hard to review, so a focused assertion beats megabytes of pretty-printed JSON.

The allowlist checker is string oriented on purpose. It can miss semantic changes that keep the same text, and it can flag harmless path reordering that still confuses a reviewer. Contributors still need a human pass on the git hunks, especially around public API and error-code stability. A model will not see uncommitted files, private credentials in environment dumps, or project policy on breaking changes.

Do not paste secrets into a shared server or a model prompt. Oracle files often contain host paths, tokens from failed auth, or fixtures that should never be public. If the bug only reproduces with proprietary data, a public before-and-after packet is the wrong shape for the proof.

Who should skip this approach

Maintainers of one-line typo fixes do not need an oracle directory sitting in the review. Security patches that should not advertise exploit details in before and after logs should use a private disclosure channel instead of this packet. Contributors without a failing invocation, including design debates and API proposals, should write a decision record rather than invent a fake output pair.

Teams that already have a golden-test harness covering the same invocation can point the pull request at that harness and skip the extra files. The principle remains even then: declare the delta, prove it on both sides of the patch, and keep the proof next to the diff. Chat summaries disappear; replayable bytes do not.

Pin the Crash in a Characterization Test Before You Patch Upstream

Blake Yang — Thu, 17 Sep 2026 05:19:50 +0000

A first-time contributor cloned a popular command-line tool and reproduced a parsing crash within minutes of setup. The stack trace pointed at a single helper, so the contributor inserted a defensive return and watched the process exit cleanly. The pull request still stalled because maintainers could not replay the old failure against the new tree. They needed a test that encoded the crash, not a narrative that the path now succeeded.

This article describes a characterization-first workflow for open-source bug patches inside libraries and command-line tools. The method records buggy behavior as an executable oracle before any production source file is changed. Free coding models may review that oracle for weak assertions after the crash is pinned locally. The procedure below is proposed sample material rather than a measured result from any one repository.

Happy-path diffs still bounce in review

Open-source maintainers read stranger diffs under time pressure and with incomplete local context around the crash. A patch that only proves a new success path leaves the original failure as folklore inside the issue thread. Characterization tests freeze observed behavior so later production edits cannot drift without a failing automated suite.

Michael Feathers described characterization tests for untested code that teams must change under real maintenance risk. The same idea helps first-time contributors who lack a mental model of every upstream caller. An oracle that fails on unpatched main and passes after the fix gives reviewers a replay button.

Common holes appear in first patches for crashing CLIs and libraries:

The reproduction lives in a screenshot or comment instead of tests/.
The new test asserts the desired API, so it cannot fail on main.
The fix changes return values that other callers still depend on.
Continuous integration stays green because the suite never sent the crashing input.

A useful oracle fails on the unpatched tree and passes on the patched commit for one named input. Maintainers can then replay the crash without reconstructing the original issue from screenshots and comment threads. Contributors should refuse to open the pull request until that replay exists in the test tree. Folklore in the issue tracker is not an acceptable substitute for that recorded executable failure.

Capture the crash before production edits

The contributor should stop after reproduction and write a test that documents current behavior, including the crash. That test may expect an exception, a nonzero process exit, or a known-bad stdout fixture on disk. Production files stay untouched until the oracle lives on a local branch that still matches upstream main. Only then does a later source change have a failing baseline that reviewers can move with confidence.

The following Python example is a proposed template for a CLI parser that dies on an empty flag value. It is unexecuted sample code and not a harvested result from any named upstream project. Reviewers should adapt file paths and exception names to the real tree before running any command. Inline comments mark the file as characterization work rather than as a finished contract test.

# tests/test_characterize_empty_flag.py
# Proposed characterization oracle. Unexecuted example.

import subprocess
import sys
from pathlib import Path

CLI = Path(__file__).resolve().parents[1] / "src" / "cli.py"


def run_cli(args):
    return subprocess.run(
        [sys.executable, str(CLI), *args],
        capture_output=True,
        text=True,
        check=False,
    )


def test_empty_flag_currently_crashes():
    """Pin the bug: empty --out should not be a silent success on main."""
    result = run_cli(["--out", "", "sample.txt"])
    # Characterization: record what main actually does today.
    assert result.returncode != 0
    assert "Traceback" in result.stderr or "ValueError" in result.stderr

On the unpatched tree this characterization test should pass because it still asserts the original crash. After the production fix, the same file must be rewritten to assert a controlled error message and a stable exit code. Both assertion blocks belong in the pull request notes so reviewers can inspect the behavioral flip. A short shell helper then makes that flip visible without rereading the entire test suite.

# proposed_oracle_flip.sh
# Unexecuted sample: prove the oracle on main, then after the assertion rewrite.
set -euo pipefail

git switch --detach main
python -m pytest tests/test_characterize_empty_flag.py -q
# Expect PASS on main because the crash still exists.

git switch --detach HEAD@{1}  # patched tree; adjust the ref as needed
python -m pytest tests/test_characterize_empty_flag.py -q
# After rewriting assertions, expect PASS for the intended contract.

Label the two assertion blocks in the pull request body as oracle-before and oracle-after for later readers. Maintainers can replay the failure story without guessing which exception used to appear on standard error. Contributors should attach the helper output as a gist or a PR comment rather than a screenshot. Those labels keep the discussion tied to files instead of vague adjectives like fixed and broken.

A three-ref behavior grid

A single passing test on the feature branch remains a weak signal for busy upstream maintainers. The same crashing input should run against the last release tag, current main, and the patched commit. The grid shows whether the crash is a regression, a long-standing hole, or an environmental false alarm. Contributors should read that comparison table before they touch production parsers or shared helpers.

The sample script assumes a clean throwaway clone because detached HEAD checkouts will disrupt a dirty working tree. Copy the repository into a scratch directory before looping across refs. Record fingerprints of stderr instead of full traces that include local paths. The script below is unexecuted sample material and will need real tag names.

#!/usr/bin/env bash
# proposed_behavior_grid.sh — unexecuted sample
set -euo pipefail
INPUT_ARGS=(--out "" sample.txt)
REFS=("v1.4.0" "main" "HEAD")

echo "ref,exit,stderr_fingerprint"
for ref in "${REFS[@]}"; do
  if ! git switch --detach "$ref" >/dev/null 2>&1; then
    printf '%s,unbuildable,\n' "$ref"
    continue
  fi
  set +e
  python src/cli.py "${INPUT_ARGS[@]}" >out.txt 2>err.txt
  code=$?
  set -e
  fp="$(head -n 3 err.txt | tr '\n' ' ' | cut -c1-80)"
  printf '%s,%s,%s\n' "$ref" "$code" "$fp"
done

Read the grid before writing the production patch and keep the CSV in the request body. If the last release already crashes, the hole is old and the test belongs in the suite even without a fix. If only main crashes, a bisect remains a separate investigation and should not mix into this oracle work. Unbuildable refs should be recorded as unbuildable in the CSV rather than as invented exit codes.

Let a free model review the oracle, not the patch

Human reviewers still miss vacuous assertions, copied fixtures, and tests that never call the changed function. A coding model can critique the oracle file before the contributor edits any product source code. The prompt should ask for weak assertions, missing inputs, and accidental coupling to log wording. It should refuse any request to invent a production patch from the issue title alone.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

MonkeyCode offers free model access and a free server option for this kind of oracle review. Contributors can paste the oracle test, the public issue excerpt, and the three-ref CSV into that workspace. The model should list holes in the characterization rather than generate a speculative production fix. No model name, quota, or hardware claim is made here beyond that operator-supplied free access option. Contributors who already pin crashes locally can use that free server option when they want a second reader on the oracle file.

A proposed review prompt, labeled as unexecuted sample text, follows in the fenced block below. The instructions ban product-code edits and ask only for defects inside the characterization oracle. Contributors should treat every bullet from the model as a hypothesis to prove against main. A mutated assertion that still passes on main means the check never encoded the crash.

You are reviewing a characterization test, not writing a patch.
Issue excerpt:
<paste the frozen acceptance lines>

Oracle file:
<paste tests/test_characterize_empty_flag.py>

Behavior grid CSV:
<paste ref,exit,stderr_fingerprint>

List:
1. Assertions that would pass even if the crash disappeared.
2. Inputs from the issue that the oracle never sends.
3. Log phrases that will drift across versions.
4. A minimal assertion rewrite for the post-fix contract.
Do not suggest product-code edits.

Treat the chat transcript as commentary rather than as evidence attached to the pull request. If the model flags a weak assertion, mutate that assertion on main and watch whether the suite still passes. That mutation check is the load-bearing step in this characterization workflow for reviewers. Maintainers should ignore a model "LGTM" that is not backed by the grid and the oracle flip.

Decision table for the oracle loop

The following table is a proposed decision aid, not a measured policy from a maintainer team. It keeps free-model review inside a narrow job that starts after the crash is pinned locally. Models help find vacuous asserts and missing inputs copied poorly from the public issue. They are a poor source of the original crash record because they were not present when the process died.

Signal on main	Oracle shape	Allowed next step	Stop if
Crash with nonzero exit	Expect exception or exit != 0	Write a production guard	Test still passes after crash is removed
Wrong output with exit 0	Golden stdout file	Change the formatter only	Golden file includes timestamps or paths
Flake only under load	Do not characterize yet	Capture a deterministic repro first	Model invents a sleep-based test
Missing feature, not a crash	Skip characterization	Write a spec test that fails on main	Oracle encodes the desired API too early

Limitations

Characterization tests freeze whatever the machine did today, including accidental environment details that will not travel. A traceback assertion that mentions a local virtualenv path will fail on CI and teach nothing about the parser. Golden files that capture full help text will churn on unrelated copy edits across later releases. Contributors should fingerprint a stable error substring rather than the entire stderr buffer from the process.

The three-ref grid assumes tags are buildable with the same toolchain as HEAD on the contributor laptop. Old releases may need different language versions, native extensions, or lockfiles that this sample script ignores. When a ref cannot run, the CSV should say unbuildable instead of fabricating a numeric exit. That honest CSV cell still helps maintainers decide whether the crash is actually new.

Model review cannot certify correctness of a patch or the absence of a regression in callers. A free server is the wrong place for secrets, unreleased customer data, or embargoed security issues. Prompts should contain only public issue text plus the contributor's own characterization oracle file. This workflow also adds latency that tiny documentation and typo fixes do not deserve.

Maintainers who already requested a specific unit test in the issue should follow that written request. This template is a fallback for sparse suites, not a replacement for a named test in the ticket. Extra ceremony on a one-character patch wastes reviewer attention and delays an otherwise obvious merge. Skip the three-ref grid when the issue already pins a file, a function, and a crashing input.

Who should not use this approach

Security patches that disclose an exploit path should not be developed as public characterization tests on a shared server. Contributors without a failing local repro should not ask a model to imagine the crash from the issue title. Teams that lack permission to push extra test files should not treat this article as license to expand scope. Those constraints matter more than any convenience that comes from free model access during review.

The method is aimed at functional bugs in libraries and CLIs where subprocess or unit tests can pin a crash. It is a poor fit for graphical timing issues, distributed race conditions, and hardware-dependent failures in drivers. Those classes need specialized harnesses that this sample workflow does not attempt to provide. Using an oracle template in those domains creates false confidence and noisy continuous integration.

Contributors who already maintain a strong, input-driven suite can skip the characterization rename step entirely. They can add a normal failing test on main and then implement the production change against that test. The oracle pattern exists for sparse suites that cannot yet name the bug in existing tests. Extra renaming on a mature suite is ceremony without new information for the reviewing maintainers.

A maintainer can replay a crash that lives in tests/ without trusting a chat log or a screenshot from the contributor. The original contributor still owes a production fix that flips the oracle for a documented reason. Optional model commentary on that oracle is useful only after the failure is executable in tests. Recorded failures outlast the issue thread that described them only in prose and screenshots.

Trace Callers Before the Diff: A Blast-Radius Card for Shared OSS Helpers

Blake Yang — Wed, 16 Sep 2026 03:38:43 +0000

The following labeled scenario opens this walkthrough and does not name a public repository or vendor team. A contributor proposed a timeout fix against a shared helper that twelve command modules imported through thin wrappers. The unit file beside that helper stayed green, so the change looked complete during a fast skim. A later release candidate failed because a worker still called the previous signature through another package.

Maintainers then rebuilt the call graph by hand, which delayed the tag and exhausted scarce review time. The missing artifact was not extra commentary pasted into the pull request description or chat thread. Reviewers needed a caller inventory that anyone could replay from the branch without extra tribal knowledge.

Why a green neighbor test is not blast-radius proof

File-local tests answer a narrow question about the hunk under the cursor and its closest fixtures. Shared helpers in open source libraries usually sit behind several import paths, generated shims, and optional extras. A green test in the same directory does not prove those paths still compile, still type-check, or still honor timeouts.

Vibe-shaped patches make this gap worse because the author narrates intent instead of listing dependents. Engineering a contribution means publishing the blast radius before the diff, then refusing hunks that fall outside that card. The rest of this article is a replayable workflow for that card, including harvest commands, a contract table, and a frozen review prompt.

The blast-radius card maintainers can replay

Keep one markdown file in the branch, named BLAST_RADIUS.md, and treat it as a merge gate rather than decoration. The card must be complete enough that a reviewer who never joined the issue thread can still reconstruct the risk. Label the following template as a proposal that should be filled from the working tree, not from memory.

# Blast-radius card

- Issue: #
- Symbol(s) under change:
- Public signature before:
- Public signature after:
- Intended behavior change (one sentence):
- Forbidden behavior change:
- Call sites harvested (path:line):
- Call sites excluded, with reason:
- Tests that must change:
- Tests that must stay green without edits:
- New dependency allowed: yes/no
- Rollback note:

Three rules keep the card honest during review. Every listed call site must come from a command that a stranger can rerun. Every excluded path must carry a reason that cites an import boundary, a generated file, or an unused extra. Every intended behavior change must fit one sentence, because a second sentence usually hides a second patch.

Harvest call sites from the tree, not from chat

Start from a clean branch that tracks the issue and avoids mixed chores. Record the symbol names before editing, including aliases and re-exports that wrappers may use. The commands below are labeled examples for a Python-like tree; adapt the patterns to the language under review.

git status --short
git rev-parse --abbrev-ref HEAD

# Example harvest for a helper named apply_timeout
rg -n --hidden -g '!vendor/**' -g '!dist/**' \
  'apply_timeout|from \.timing import apply_timeout|timing\.apply_timeout'

# Confirm the definition sites before classifying callers
rg -n '^def apply_timeout|^async def apply_timeout'

Paste every hit into the card with path and line, then classify each hit as a direct caller, a re-export, a test double, or noise. Delete noise only when the file cannot import the symbol at runtime, and write that reason beside the exclusion. A second pass with git grep catches generated files that some ignore rules hide from rg.

git grep -n 'apply_timeout'
git log -S 'apply_timeout' --oneline -- max-count=20

The history search is not a substitute for the inventory. It only shows whether the symbol recently moved, which often explains stale wrappers that tests never import. When the log shows a rename, add the old name to the harvest patterns and extend the card before any production hunk is written.

A small classifier script for the harvest

The following Python sketch is an unexecuted example that turns ripgrep-style lines into a table. Run it only after redirecting harvest output to a text file, and treat its labels as hints rather than proof.

# unlabeled proposal: classify harvested lines into a blast-radius table
from pathlib import Path

KINDS = (
    ("test", ("/tests/", "_test.py", "test_")),
    ("reexport", ("__init__.py",)),
    ("wrapper", ("compat.py", "shims/")),
)

def classify(path: str) -> str:
    for kind, needles in KINDS:
        if any(n in path for n in needles):
            return kind
    return "caller"

lines = Path("harvest.txt").read_text().splitlines()
for raw in lines:
    if ":" not in raw:
        continue
    path, rest = raw.split(":", 1)
    print(f"{classify(path):9} {path}:{rest}")

Commit the harvest file beside BLAST_RADIUS.md so reviewers can rerun classification without trusting a screenshot. If the classifier and the card disagree, stop and repair the card before writing product code. Disagreement at this stage is cheaper than a revert after a release candidate.

Freeze a contract table before the first production hunk

Turn the intended change into a table that later tests can fail against. Columns stay boring on purpose: input, old observable, new observable, and owners of that path. Rows that cannot name an owner do not belong in the patch.

Call site	Input that matters	Old observable	New observable	Owner test
`cmd/sync.py:88`	HTTP client timeout	30s default	5s default	`tests/cmd/test_sync.py`
`worker/retry.py:41`	retry wrapper timeout	inherits helper	must stay 30s	`tests/worker/test_retry.py`
`compat/old_api.py:12`	re-export	same signature	same signature	`tests/compat/test_export.py`

The middle row is the reason this workflow exists. Shared helpers often have one caller that must change and another caller that must not change, and file-local tests cannot see the second caller. If the table needs more than five rows, split the work into two issues rather than smuggling a migration through a timeout fix.

Patch against the card, then move the tests that own the rows

Edit production code only after the card and table are committed. Keep the diff inside the listed symbols, and reject drive-by formatting in files that are not call sites. When a wrapper needs a new argument, add it as an explicit keyword with a default that preserves the forbidden behavior change.

# labeled example: preserve the worker path while tightening the CLI path
def apply_timeout(client, seconds=30, *, inherit=True):
    if inherit:
        return client
    client.timeout = seconds
    return client

Run the owner tests named in the table before the full suite, because those tests are the contract. A patch that leaves every owner test untouched has not demonstrated the new observable. A patch that rewrites unrelated assertions has expanded the blast radius after the card was frozen, which should fail review.

pytest -q tests/cmd/test_sync.py tests/worker/test_retry.py tests/compat/test_export.py
pytest -q
git diff --stat

--stat is part of the gate. If the file list contains modules absent from the card, restore those files or amend the card with a new harvest. Reviewers should not have to discover extra files by scrolling a surprise diff.

Frozen-prompt review after the card is complete

After the blast-radius card, the contract table, and the tests exist, a second pass can interrogate those files without extra narrative. Disclosure: This article was prepared as part of MonkeyCode's product outreach. Contributors without a spare checkout can keep that frozen packet on MonkeyCode's free server option and run the same prompt through free model access, so the model sees only the card, the harvest, and the diff.

The prompt below is a labeled template. Paste the files; do not paste issue chatter, because chatter reintroduces assumptions the card was meant to kill.

You are reviewing an OSS patch packet, not writing new code.
Read BLAST_RADIUS.md, harvest.txt, the contract table, and the diff.
Report only:
1) call sites in the diff that the card does not list
2) call sites on the card that the diff does not touch, with risk
3) owner tests that did not move for a changed observable
4) signature changes that violate the forbidden behavior line
Refuse to suggest extra features. If the packet is incomplete, say incomplete.

Treat the model output as a checklist against the working tree, not as a merge vote. Every finding must be confirmed with rg or a failing test before the author changes code. Findings that cannot be replayed from the packet are discarded, because they are vibes wearing a reviewer's coat.

Decision table for this workflow

Situation	Action	Merge stance
Harvest and card disagree	Stop and repair the card	Block
Owner test did not move	Add or fail the observable	Block
Diff file absent from the card	Restore or re-harvest	Block
Model flags a missing caller, `rg` confirms	Extend tests, then patch	Block until green
Model flags a missing caller, `rg` is clean	Discard the finding	Continue
Typo in a comment only	Skip the card	Allow
Security embargo or private fixture	Do not upload the packet	Use a private review path

The table is the engineering part of the loop. Tools can harvest and nag, but they cannot decide which callers are load-bearing. Humans still own that classification, and the card makes the classification visible.

Limitations and who should skip this card

This workflow assumes the repository has searchable call sites and tests that can name an observable. Generated parsers, huge monorepos without import conventions, and binary plugins will produce noisy harvests that look complete while missing real callers. In those trees, the card still helps, but it cannot be the only gate.

Authors should skip the full card for single-line comment fixes, changelog-only pull requests, and lockfile refreshes that do not touch runtime symbols. They should also skip uploading a packet that contains secrets, private customer fixtures, or embargoed vulnerability detail to any hosted model or shared server. Maintainers should reject the method when a contributor uses the model output as a substitute for running the owner tests.

The approach also fails when the intended change cannot fit one sentence on the card. That failure is useful, because it usually means the issue is a migration and needs a staged plan. Splitting the work is slower on the first day and cheaper than a revert after a green-looking helper change lands.

A blast-radius card does not make a patch small, and it does not make a model authoritative. It makes the dependents visible before the diff, which is the part most vibe-shaped OSS reviews still skip. Contributors who already freeze that card can run the frozen review pass with free model access when another local checkout is the wrong place for the packet.

The Hunk Necessity Test: Prove Every Line of an OSS Patch Is Load-Bearing

Blake Yang — Mon, 14 Sep 2026 21:29:59 +0000

A contributor fixes a three-line null check in a parser, then ships the same commit with a reformatted switch statement and two renamed locals. Six days later the pull request is still open, and the maintainer asks the question that stalls most external patches: which parts of this diff are actually required? CI is green, but a green branch only proves the suite still passes with the patch applied. It says nothing about whether every changed line earns its place in the review.

This article describes a small harness that answers that question hunk by hunk. For each hunk in a patch, it reverse-applies only that hunk and reruns the pinned failing test, then records whether the test breaks again. The output is a three-value verdict per hunk, produced by the test runner rather than by opinion, and it turns a subjective "this diff feels large" comment into a table a maintainer can act on.

Why per-hunk evidence beats a narrated patch

Maintainers triage far more patches than they can read closely, so they optimize for review cost. A diff that mixes a fix with unrelated cleanup forces them to do three jobs at once: find the actual fix, verify it, and audit everything else for hidden behavior change.

Three failure modes show up again and again in review threads.

Dead weight. A hunk that no test depends on, usually a drive-by refactor, gets reverted during review or blocks the merge entirely.
False confidence. A patch passes because the suite is weak, not because the change is correct, and nobody can tell which lines carried the fix.
Split requests. A maintainer asks for the cleanup to move to a separate pull request, which costs the contributor another full review cycle.

A hunk necessity run front-loads that work. The contributor discovers the dead hunks before a human has to point them out, and the final patch arrives with machine-checkable evidence attached.

Gate zero: the pinned test must fail before the patch

The harness assumes a single test command that reproduces the reported bug and exits non-zero on the unfixed revision. Without that gate, the whole run is noise: if the test already passes on HEAD, then every hunk will look "dead" and the table will be meaningless.

Two preconditions therefore run before any hunk is touched. The patch must make the test pass, and reverting the patch must make it fail again. A contributor who cannot satisfy both conditions has a test-quality problem to solve first, not a patch-splitting problem.

# gate 0, abbreviated
 git apply "$PATCH"      && eval "$TEST_CMD"   # must succeed
 git apply -R "$PATCH"   && eval "$TEST_CMD"   # must fail

The harness: split, reverse-apply, rerun

The script below runs inside a detached worktree so the reset loop cannot touch uncommitted work in the main checkout. It splits a unified diff into one file per hunk, keeps the file headers attached to each piece, then evaluates every hunk independently.

#!/usr/bin/env bash
# hunk-necessity.sh <patch-file> "<test-command>"
# Requires: git >= 2.30, a clean worktree, a test command that exits non-zero on the pinned bug.
set -euo pipefail

PATCH="$(realpath "${1:?patch file required}")"
TEST_CMD="${2:?test command required}"

ROOT="$(git rev-parse --show-toplevel)"
TMP="$(mktemp -d)"
HUNKS="$TMP/hunks"
RESULTS="$TMP/results.tsv"
mkdir -p "$HUNKS"
trap 'git -C "$ROOT" worktree remove --force "$TMP/wt" 2>/dev/null || true; rm -rf "$TMP"' EXIT

git -C "$ROOT" worktree add -q --detach "$TMP/wt" HEAD
cd "$TMP/wt"

# --- gate 0 ---
git apply "$PATCH"
eval "$TEST_CMD" >/dev/null 2>&1 || { echo "gate0: patch does not fix the pinned test" >&2; exit 2; }
git apply -R "$PATCH"
if eval "$TEST_CMD" >/dev/null 2>&1; then
  echo "gate0: pinned test already passes on HEAD; it does not pin the bug" >&2; exit 3
fi

# --- split the diff into one patch per hunk, headers included ---
awk -v out="$HUNKS" '
  /^diff --git / { if (hunk != "") close(hunk); hunk = ""; header = $0 "\n"; next }
  /^(index |--- |\+\+\+ |new file|deleted file|similarity|rename|old mode|new mode)/ {
                    header = header $0 "\n"; next }
  /^@@/           { if (hunk != "") close(hunk);
                    hunk = sprintf("%s/hunk-%03d.patch", out, ++n);
                    printf "%s", header > hunk }
  hunk != ""      { print >> hunk }
' "$PATCH"

printf 'hunk\tverdict\n' > "$RESULTS"
for piece in "$HUNKS"/hunk-*.patch; do
  git checkout -q -- .
  git apply "$PATCH"
  id="$(basename "$piece" .patch)"
  if ! git apply -R "$piece" 2>/dev/null; then
    printf '%s\tentangled\n' "$id" >> "$RESULTS"
  elif eval "$TEST_CMD" >/dev/null 2>&1; then
    printf '%s\tdead\n' "$id" >> "$RESULTS"
  else
    printf '%s\tload-bearing\n' "$id" >> "$RESULTS"
  fi
done

column -t "$RESULTS"

The harness is a starting point rather than a released tool, so treat the reset logic as reviewable code and run it on a scratch clone first. Every verdict comes from an exit code, which keeps the classification reproducible on any machine that can run the suite.

Reading the verdict table

Verdict	How it is produced	What to do next
`load-bearing`	Reverse-applying this hunk alone makes the pinned test fail	Keep it in the fix commit, one hunk at a time
`dead`	The test still passes with the hunk removed	Move it to a separate cleanup pull request, or drop it
`entangled`	The hunk cannot be reverse-applied on its own	Inspect manually; often a rename, a move, or half of a two-part fix

A patch whose hunks are mostly dead is not wrong, it is simply over-scoped, and the table gives the contributor a defensible reason to split it before review starts.

Where a free model actually helps, and where it must not decide

Once the table exists, the remaining work is interpretation: explaining why a hunk came back entangled, grouping dead hunks into a coherent follow-up commit, and drafting split-commit messages that a maintainer can read in one pass. That is language work over structured evidence, which is a reasonable fit for a hosted model in a browser session.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

MonkeyCode is relevant here for one narrow reason: it advertises free model access and a free server option, both operator-supplied claims, which means a contributor can run the interpretation pass and the harness itself without paying for a subscription or provisioning a machine. The free plan page advertises a ten-million-token allowance at the time of writing, and quotas, model names, and machine specifications change often, so verify the current terms before depending on any specific number.

The division of authority matters more than the tooling. The model proposes a hypothesis about an entangled hunk or a commit split; the test runner remains the only thing allowed to issue a verdict. Anything the model claims about a hunk's necessity is unverified until the harness reruns and reproduces it.

Running the loop on a free server

A long suite multiplies awkwardly: with n hunks, the harness performs roughly n + 2 full test runs. A ten-minute suite and twelve hunks is about two hours of wall-clock time, which is a poor fit for a laptop needed for other work. The free server option moves that loop off the contributor's machine and keeps the local checkout free for editing, which is the practical case for it here rather than any claim about speed.

Limitations and who should not use this

The method has real boundaries, and a contributor who ignores them will produce a confident table that is wrong.

Interacting hunks. Two halves of one fix can each look dead in isolation; a pairwise or grouped pass is required before deleting anything.
Skipped changes. Header-only and mode-only changes, binary files, and pure renames fall outside the splitter and never appear in the table.
Flaky tests. A flaky suite converts dead into load-bearing at random, so rerun borderline verdicts or quarantine the unstable test first.
Cost. Runtime scales linearly with hunk count, which makes the approach expensive for a fifty-hunk diff and cheap for a five-hunk one.
Not a correctness proof. A load-bearing verdict means the suite noticed the hunk, not that the fix is right or complete.

Skipping the harness is reasonable for a one-file, one-hunk patch, for repositories whose tests need privileged infrastructure the contributor cannot reproduce, and for projects whose contribution guidelines require a single squashed commit regardless of scope.

A pre-push checklist

Confirm the pinned test fails on HEAD and passes with the full patch.
Split the diff into hunks and record a verdict for each one.
Move every dead hunk into a separate pull request, or delete it.
Inspect every entangled hunk by hand and note the reason in the patch description.
Attach the verdict table to the pull request so the reviewer starts from evidence.

None of this replaces a maintainer's judgment about design, and it is not meant to. It simply removes the cheapest objection from the review thread before anyone has to type it, which is often the difference between a patch merged this week and one reopened next month. If the loop above fits how you already work, MonkeyCode's free tier is one place to run it without a paid subscription.

Freeze the Issue Thread: Numbered Criteria Before an OSS Patch

Blake Yang — Sun, 13 Sep 2026 19:40:35 +0000

A first-time contributor opened a three-year-old GitHub issue with forty-seven comments and three competing workarounds. The original report described a timeout, while later comments blamed a retry helper that landed in a different module. The contributor shipped a twelve-file refactor that matched the loudest comment and missed the reproduction steps in comment eighteen. Maintainers closed the pull request as out of scope, even though the local test suite stayed green.

Open-source issues rarely stay frozen after the first report, because later users paste new stacks and partial fixes. Comment order then starts to look like a specification, even though nobody voted on which workaround became canonical. A green test run on a laptop cannot prove that the change matches the issue the maintainers still intend to close. The missing artifact is a short contract that restates the thread as numbered, testable acceptance criteria.

Implicit specs collapse during review

Maintainers read a pull request against the issue title, the latest comment, and the code they remember shipping. Contributors often read the same thread as a pile of anecdotes and then patch the symptom that matched their local setup. Coding assistants amplify that mismatch when they treat the entire comment cloud as equally authoritative input. The workflow below treats the issue thread as raw material for a contract, not as a prompt dump.

This pattern sits next to current debates about assistants writing entire diffs without a frozen problem statement. The failure is not that models cannot emit patches; the failure is that the issue never named the observable result. A surgical open-source patch still needs an explicit MUST list before reproduction, implementation, or review begins.

Failure modes the packet is meant to catch

A workaround in a late comment silently replaces the original reproduction without maintainer confirmation.
A drive-by rename lands in the same commit as the behavioral fix and hides the contract change.
A test asserts an implementation detail that no comment required, so later refactors look like regressions.
A model-authored summary restates the loudest comment and drops environment pins from the original report.

Build the criteria packet before any source edit

The packet lives beside the clone, usually as ISSUE_CONTRACT.md, and it is committed only if the project already stores review notes. Otherwise the file stays in the contributor's notes directory and is pasted into the pull request body as a checklist. Every MUST line must name an observable result, an environment pin, and a source comment or issue section. SHOULD and WONTFIX lines exist so the patch can refuse popular requests that maintainers already rejected.

The following template is an illustrative packet, not a scraped issue from a live repository.

# ISSUE_CONTRACT.md
Issue: https://github.com/example/libretry/issues/1841
Head: main @ 9f3c1aa
Packet date: 2026-09-14

## MUST
- M1. A request that times out after 200ms returns `Retry-Exhausted` without calling the deprecated `sleep_loop` helper (issue body, env: Python 3.12, Linux).
- M2. `tests/test_retry.py` fails on current main with fixture `timeout_200ms.json` before the patch (comment 18).
- M3. Public module `libretry.api` keeps the same exported names; `git diff --stat` stays inside `libretry/retry.py` and `tests/test_retry.py` (CONTRIBUTING.md, small diffs).

## SHOULD
- S1. Log line on exhaustion includes the attempt count already used by `libretry.metrics` (comment 22, not confirmed).

## WONTFIX for this PR
- W1. Rewrite the backoff to jittered exponential across all clients (comment 31, maintainer: out of scope).
- W2. Rename `Retry-Exhausted` to `TimeoutError` (comment 9, closed as breaking).

Numbering rules that keep the packet honest

Bind each MUST to a comment number, issue body section, or CONTRIBUTING rule so the source stays auditable.
Record the toolchain pin in the same line as the behavior, because a laptop default is not an observable.
Put popular but rejected ideas under WONTFIX so a later model review cannot revive them as helpful extras.
Freeze Head to a commit SHA before reproduction, so later main movement cannot silently rewrite the contract.

Reproduce against MUST lines only

Reproduction is the first implementation of the contract, and it should fail in the way M2 describes. Clone a clean tree, check out the frozen SHA, and install the project's documented toolchain instead of the contributor's global defaults. The commands below are a labeled example for a Python library; they are not claimed as a run from a production incident.

git fetch origin main
git checkout --detach 9f3c1aa
python -m venv .venv && . .venv/bin/activate
pip install -e ".[test]"
pytest tests/test_retry.py::test_timeout_200ms -q
# Expected on main: FAIL, matching M2

If the named test does not exist yet, add a failing test that encodes M1 before any production edit. Keep that commit separate from the fix so reviewers can read failure, then repair, as two logical steps. Do not ask a model to make tests pass until M2 is visible on the frozen SHA. A passing suite at this stage means the contract is wrong, not that the project is healthy.

Constrain the diff to contract paths

Most rejected OSS patches fail review because the diff teaches the project a second lesson the issue never asked for. After M2 fails on main, create a branch whose only job is M1, and refuse files that M3 did not name. The shell snippet below is a local gate the contributor can run before every git push.

git checkout -b fix/1841-retry-exhausted
# edit only libretry/retry.py after the failing test exists
ALLOWED='^(libretry/retry.py|tests/test_retry.py|ISSUE_CONTRACT.md)$'
bad=$(git diff --name-only origin/main...HEAD | grep -Ev "$ALLOWED" || true)
if [ -n "$bad" ]; then
  printf 'Paths outside M3:\n%s\n' "$bad"
  exit 1
fi
git diff origin/main...HEAD --stat

A second commit may add S1 if the metrics log is cheap and already patterned in the code. W1 and W2 stay out of the branch even when an assistant proposes them as cleanup. Surgical diffs remain easier to revert, bisect, and backport than the comment-driven refactors that usually follow a noisy thread.

Bind tests to criterion identifiers

Reviewers should not hunt through the issue thread for the sentence that a test claims to cover. Name the test after the MUST id, and restate the observable in the assertion message. The example below is pseudocode for illustration, not a patch against a real library.

# tests/test_retry.py
def test_m1_timeout_returns_retry_exhausted_without_sleep_loop():
    """ISSUE_CONTRACT M1: 200ms timeout -> Retry-Exhausted, no sleep_loop."""
    client = RetryClient(timeout_ms=200)
    with pytest.raises(RetryExhausted):
        client.request("https://example.invalid/slow")
    assert "sleep_loop" not in client.helpers_called

Criterion	Observable	Test id	Allowed paths	Status on frozen main
M1	`Retry-Exhausted`, no `sleep_loop`	`test_m1_timeout_returns_retry_exhausted_without_sleep_loop`	`libretry/retry.py`	not implemented
M2	named test fails before the fix	same test on `9f3c1aa`	`tests/test_retry.py`	FAIL required
M3	export surface unchanged	`check_issue_contract.py`	those two files only	gate
S1	attempt count in log	optional	`libretry/retry.py`	skip if noisy
W1–W2	no extra diff	`git diff` review	none	reject extras

The small checker below walks the contract file and the merge diff. It is a local gate, not a claim about any project's CI. Save it as check_issue_contract.py and treat it as a labeled example rather than a published OSS tool.

#!/usr/bin/env python3
from __future__ import annotations

import re
import subprocess
import sys
from pathlib import Path

CONTRACT = Path("ISSUE_CONTRACT.md")
MUST_RE = re.compile(r"^- (M\d+)\. ", re.M)
PATH_RE = re.compile(r"`([^`]+)`")


def must_ids(text: str) -> list[str]:
    return MUST_RE.findall(text)


def allowed_paths(text: str) -> set[str]:
    block = text.split("## MUST", 1)[-1].split("## SHOULD", 1)[0]
    return {p for p in PATH_RE.findall(block) if "/" in p or p.endswith(".py")}


def diff_names() -> list[str]:
    out = subprocess.check_output(
        ["git", "diff", "--name-only", "origin/main...HEAD"], text=True
    )
    return [line for line in out.splitlines() if line]


def main() -> int:
    text = CONTRACT.read_text(encoding="utf-8")
    ids = must_ids(text)
    if not ids:
        print("No MUST ids found", file=sys.stderr)
        return 2
    pr_path = Path("PR_BODY.md")
    pr_body = pr_path.read_text(encoding="utf-8") if pr_path.exists() else text
    missing = [i for i in ids if i not in pr_body]
    if missing:
        print("PR body missing MUST ids:", ", ".join(missing))
        return 1
    allowed = allowed_paths(text) | {"ISSUE_CONTRACT.md", "PR_BODY.md"}
    bad = [p for p in diff_names() if p not in allowed]
    if bad:
        print("Paths outside contract:")
        print("\n".join(bad))
        return 1
    print("Contract ids:", ", ".join(ids))
    print("Diff paths inside M3.")
    return 0


if __name__ == "__main__":
    raise SystemExit(main())

Run python check_issue_contract.py after filling PR_BODY.md with the same MUST list that will appear on GitHub. The script fails closed when a drive-by file appears, which is the usual review killer. It does not prove semantic correctness; it only proves the packet and the diff still talk about the same files and ids.

Score the diff against the packet, not against vibes

Once M2 fails on the frozen SHA and the fix plus tests pass, the remaining risk is semantic drift. A second human pass remains the standard of record, especially for security-sensitive or cryptography-adjacent code. A model pass is useful when scored against ISSUE_CONTRACT.md and the merge diff, rather than a request to improve the patch.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode's free model access can host that contract-aware review without pasting the issue thread into an ad-hoc chat window. The free server option is relevant when the review command and the project's test extras need a longer-lived workspace than a laptop session. No model name, quota, or benchmark is claimed; the workflow only needs a model that can read a diff and a packet.

The review prompt below is a template for operators to adapt. It is not an executed evaluation and it is not a ranking of products.

You are scoring a patch against ISSUE_CONTRACT.md.
Read the contract first, then the git diff.
For each MUST id, answer: satisfied / violated / not evidenced.
For each WONTFIX id, answer: absent / revived.
List files in the diff that the contract did not allow.
Do not suggest extra refactors, renames, or dependency bumps.
If the diff includes files outside M3, fail the review.

Keep that prompt next to the packet so the same words gate both the human description and the model pass. Contributors can run the same prompt against MonkeyCode's free model access when they want a second reader that cannot invent new scope. The packet still works if that second reader is a teammate instead of a hosted model.

Decision table for common issue shapes

Issue shape	Contract emphasis	Patch allowed to touch	Model review question
Timeout or retry bug	environment pin plus exact exception	one module and one test	Did the deprecated helper return in traces?
Docs versus behavior mismatch	quote the documented example	docs file or code, not both unless M1 says so	Which side is canonical in CONTRIBUTING?
Closed WONTFIX revived by users	WONTFIX list copied from maintainer comment	no files, or an issue comment only	Did the diff revive W1?
CI-only failure	job name, image tag, and seed	CI config plus the failing test	Was the local command the same as CI?

The table is a planning tool for triage, not a complete taxonomy of every issue shape. If an issue mixes a public API break with a private helper change, split it into two packets and two pull requests. Mixing those concerns is how comment eighteen disappears under a rename that no MUST line requested.

Limitations

This workflow assumes the issue thread contains at least one reproduction hint and one maintainer signal about scope. It will not invent a root cause for a crash that nobody on the thread can trigger. The path gate cannot see semantic breakage inside an allowed file, so M1 still needs a real assertion. Automated model review will still miss domain invariants that the written contract forgot to capture in MUST form.

The checker also trusts origin/main...HEAD, which is wrong for stacked branches or for repositories whose default branch is not main. Teams that squash every commit still need the failing test visible in review, even if history becomes one commit later. None of the examples above are performance claims, and none should be copied into a security advisory without a separate process.

Who should skip this workflow

Drivers who already have a maintainer-written design doc should follow that doc rather than re-deriving MUST lines from noisy comments.
Emergency incident patches that need to land in minutes cannot afford a packet ceremony and should use a revert or a feature flag.
Contributors who cannot run the project's tests at all should not use a model review as a substitute for reproduction.
People looking for a fully automated issue-to-merge agent will not find that here, because the contract is a human freeze.

What maintainers see

A pull request that pastes the packet and shows M2 failing on the named SHA is easier to trust than a restated comment. The follow-up commit should show M1 passing on the branch with the same identifiers in the test names. The diffstat matches M3, WONTFIX items stay absent, and the citations still point at issue comments. That packet is the original artifact; the optional model score is only a second reader with the same sheet.

Ship a One-Command Repro Script Before Maintainers Read an OSS Diff

Blake Yang — Sat, 12 Sep 2026 17:35:58 +0000

A contributor pasted a twelve-file diff under an issue that described a CLI exiting zero on invalid UTF-8 input. The maintainer cloned the fork on a clean runner and could not trigger the failure with the commands in the description. Three review comments later, the conversation still concerned local virtualenvs, Python minor versions, and a fixture that never reached the repository. The patch may have been correct, yet the project had no command a stranger could run to watch the bug happen.

Open-source review stalls when the reproduction lives only in the author's shell history. Maintainers need a script that fails on the reported revision, passes after the patch, and names every environment assumption in plain text. This article records that packet as a review artifact, then shows how a free-tier model pass can check issue-to-command traceability without rewriting the patch.

Treat the Issue as a Contract, Not as Flavor Text

Issue templates already collect version strings, operating systems, install steps, and expected versus actual output. Contributors often skip that structure and jump into a refactor that also happens to mention the bug. The safer order is to freeze a reproduction contract before any production file changes, then refuse to expand the diff until that contract fails on current HEAD.

The contract is a short table, not a narrative. Each row must be something a maintainer can execute without guessing paths or secrets. Rows that cannot be turned into a command do not belong in the first patch.

Contract field	Example freeze	Reject if missing
Upstream revision	`git rev-parse HEAD` at clone time	"latest main" with no SHA
Runtime	`python3.12` from the project's documented range	Author's private conda env
Input	`printf '\xff' > /tmp/bad.bin`	"a weird file I have locally"
Command	`./cli parse /tmp/bad.bin`	IDE click-path only
Actual now	exit `0` and empty stderr	screenshot without exit code
Expected after patch	exit `2` and `invalid utf-8` on stderr	"should feel more correct"

Copy that table into REPRO.md beside the future patch. Do not describe hoped-for architecture in this file. Architecture notes wait until the command fails for a stranger.

Build a Maintainer-Shaped Script Before Touching Library Code

A reproduction script should pin the revision, create an isolated directory, install only documented extras, and exit non-zero while the bug still exists. The script is the first commit on the contribution branch, even when the production fix is still unknown. Maintainers can then run one command on the author's branch and on main without reading the diff.

#!/usr/bin/env bash
# repro.sh — must fail on the buggy revision, pass after the fix.
set -euo pipefail

ROOT="$(cd "$(dirname "$0")" && pwd)"
REV="${REV:-$(git -C "$ROOT" rev-parse HEAD)}"
WORKDIR="$(mktemp -d /tmp/oss-repro.XXXXXX)"
trap 'rm -rf "$WORKDIR"' EXIT

git -C "$ROOT" archive "$REV" | tar -x -C "$WORKDIR"
cd "$WORKDIR"

python3.12 -m venv .venv
# shellcheck disable=SC1091
source .venv/bin/activate
python -m pip install -e ".[dev]" >/tmp/repro-install.log

printf '\xff' > /tmp/bad.bin
set +e
./.venv/bin/cli parse /tmp/bad.bin >/tmp/repro-out.txt 2>/tmp/repro-err.txt
STATUS=$?
set -e

echo "rev=$REV exit=$STATUS"
if [[ "$STATUS" -eq 0 ]]; then
  echo "BUG STILL PRESENT: invalid input was accepted" >&2
  exit 1
fi
grep -q 'invalid utf-8' /tmp/repro-err.txt
echo "repro satisfied"

Mark the script executable and run it against the issue revision before writing a fix. The first run should print BUG STILL PRESENT and exit 1. If it prints repro satisfied on main, the issue is already fixed or the contract is wrong, and no library patch should ship.

chmod +x repro.sh
git switch -c fix/invalid-utf8-exit-code
git add repro.sh REPRO.md
git commit -m "test: add maintainer repro for invalid UTF-8 exit code"
REV=$(git rev-parse HEAD) ./repro.sh; echo "first-run exit=$?"

Keep generated virtualenvs and /tmp fixtures out of git. The script creates those paths on each run so reviewers do not inherit the author's machine. If the project needs a compiler flag or a locale, put that export in the script rather than in a blog comment.

Fail the Same Assertion in the Project Test Runner

A shell repro is necessary for maintainers who will not open an IDE. It is not a substitute for the suite that CI already knows how to execute. After the script fails, add one test in the project's normal layout that encodes the same input, command, and exit contract.

# tests/test_invalid_utf8_exit.py
from pathlib import Path
import subprocess
import sys

def test_parse_rejects_invalid_utf8(tmp_path: Path) -> None:
    payload = tmp_path / "bad.bin"
    payload.write_bytes(b"\xff")
    proc = subprocess.run(
        [sys.executable, "-m", "cli", "parse", str(payload)],
        capture_output=True,
        text=True,
        check=False,
    )
    assert proc.returncode == 2
    assert "invalid utf-8" in proc.stderr.lower()

Run that file alone until it fails for the documented reason, not for an import error. Import failures mean the repro still depends on unpublished helpers. Only then is a production edit in scope.

python -m pytest tests/test_invalid_utf8_exit.py -q
# expected on main: 1 failed, assertion on returncode or stderr

Patch Against the Contract, Then Re-run Both Gates

Limit the production change to the code path the repro actually exercises. Wide cleanups belong on a later branch after the bug is closed. After the edit, the same repro.sh must invert its meaning: the bug-present branch of the script should no longer trigger, and the pytest file should pass in isolation and with the project's default target.

# after the smallest library edit
./repro.sh && python -m pytest tests/test_invalid_utf8_exit.py -q
git add src/cli/parse.py tests/test_invalid_utf8_exit.py
git commit -m "fix: reject invalid UTF-8 with exit 2"

If repro.sh still exits 1, the patch did not implement the contract, even if unit tests elsewhere turned green. If pytest passes while repro.sh fails, the suite is mocking away the install path maintainers will use. Both results block the pull request until they agree.

Map Each Issue Sentence to a Hunk Before Asking for Review

Maintainers reject AI-shaped patches when the diff argues with the issue. A mapping file prevents that drift. Write REVIEW_MAP.md with one bullet per issue claim, the command that witnesses it, and the file hunk that implements it. Claims without a command stay out of the pull request body.

# REVIEW_MAP.md
- Claim: invalid byte sequence must not exit 0
  Witness: `./repro.sh` on this branch exits 0 after the fix commit
  Hunk: `src/cli/parse.py` decoder error path
- Claim: stderr names the encoding failure
  Witness: `grep -q 'invalid utf-8' /tmp/repro-err.txt`
  Hunk: `src/cli/parse.py` error formatter
- Non-claim: no public CLI flag renamed in this patch
  Witness: `git diff main -- src/cli/flags.py` is empty

A second reader can check that mapping without touching the working tree. Disclosure: This article was prepared as part of MonkeyCode's product outreach. Contributors who already use MonkeyCode's free model access and free server option can paste the issue text, repro.sh, and the unified diff into that workspace and ask only whether each claim has a failing-then-passing command. The workflow does not depend on that product, and this article does not add model names, quotas, or hardware claims beyond those two operator-supplied options.

Keep the model prompt narrow so it cannot invent extra refactors. The useful question is whether the packet is internally consistent, not whether the code looks stylish.

You are checking a maintainer reproduction packet.
Do not propose new features or drive-by cleanups.
Issue text, REPRO.md, repro.sh, REVIEW_MAP.md, and git diff follow.
List every issue claim that lacks a command or a hunk.
List every diff hunk that matches no issue claim.
If the script would pass on the buggy revision, say so explicitly.

Treat the model output as a checklist against REVIEW_MAP.md. If it reports an unmatched hunk, delete that hunk or move it to a follow-up issue. If it reports a missing witness, extend repro.sh instead of arguing in prose.

Decision Table for Stopping Work

Observation	Action	Do not
`repro.sh` passes on `main`	Close as already fixed or rewrite the contract	Ship a speculative cleanup
Script fails on missing compiler	Record the toolchain in `REPRO.md` and the script	Ask maintainers to "use my laptop"
Pytest green, script red	Drop mocks that hide install layout	Trust local IDE runners
Model flags an extra hunk	Revert that hunk before review	Explain the extra hunk in the PR essay
Contract needs network or secrets	Split into a follow-up with documented credentials	Put tokens in the repro script

Limitations and Who Should Skip This Packet

The one-command packet assumes the bug is deterministic, local, and expressible as process input plus exit status or files. Timing races, GPU kernels, and hardware-specific faults need a different harness, often with logging and multiple iterations. This workflow also assumes the project already documents an install path that a clean checkout can follow.

Contributors should not use this packet as a replacement for the project's required CI jobs. A green repro.sh on one Python minor version does not prove the matrix the maintainers actually ship. People working on security issues with non-public fixtures should keep those fixtures out of a public repro.sh and follow the project's private disclosure process instead.

Teams that already have a hermetic nix develop or container entrypoint can wrap that entrypoint rather than inventing a second installer. The artifact that matters is still a single command whose exit code flips with the patch. Free model review of the mapping is optional and can be skipped when the packet is shorter than a screen.

The pull request body then stays short: link the issue, paste the one command, and attach REVIEW_MAP.md. Maintainers can reproduce before they argue about style. Contributors who already keep a free coding workspace can run the same packet there as optional verification, then still wait for the project's own runner before claiming the bug is gone.

Worktree Isolation for OSS Bugs: A Review Packet Maintainers Can Trust

Blake Yang — Fri, 11 Sep 2026 15:22:50 +0000

On a typical issue tracker, a contributor clones a small HTTP client and treats a timeout as a one-line constant change. The pull request description cites a single local test run on the newest interpreter the laptop already had installed. Continuous integration then fails on an older runtime the contributor never installed, and the thread spends days reconstructing a reproduction. Maintainers are not rejecting the idea of a fix; they are rejecting a change that never isolated the failing command.

Public discussion around AI coding tools often frames models as ready to emit complete patches from a full repository dump. That framing skips the part of open source work that still fails in practice: proving the bug, constraining the diff, and showing one command before and after. A model that reads an entire tree will invent helpers, ignore matrix cells, and rewrite style the project does not use. A smaller packet built inside an isolated git worktree gives both humans and models the same evidence.

Whole-repo pastes hide the failure

Open source reviews fail for mechanical reasons that have little to do with model quality or contributor intent. The reviewer cannot see which command actually failed, which files were incidental, or which contributing rules the author actually read. A paste of the repository also hides generated lockfiles, vendored snapshots, and local configuration that should never leave the laptop.

A useful packet answers four points with files rather than adjectives:

Which worktree and commit reproduced the bug in isolation
Which exact command failed, including arguments and exit status
Which files the patch actually touches, shown as a unified diff
Which project rules constrain the change, quoted from tests or CONTRIBUTING

The rest of this article proposes a workflow. Commands are examples for a typical Python library and should be adapted to the target project's documented test runner.

Isolate the bug in a throwaway worktree

Cloning a second full copy is slower than git worktree, and a second worktree keeps failed experiments off the main checkout. The contributor should start from the revision named in the issue, not from a personal branch that already contains unrelated formatting. A clean tree makes later diffs honest, because generated files from a previous attempt will not leak into the patch.

# Proposed workflow — adapt paths and the issue SHA to the target repository.
git fetch origin
git worktree add ../repro-issue-1842 origin/main
cd ../repro-issue-1842
git switch -c repro/issue-1842

If the issue names a tag or a commit, check that revision out before installing dependencies so the failing command matches the report. Project docs, not a model guess, should decide how extras and test tools are installed.

# Example only: pin the reported revision, then install from the project docs.
git checkout --detach 7c1a9e4
python -m venv .venv
source .venv/bin/activate
pip install -e ".[dev]"

Capture the failing command as the contract

A stack trace in an issue comment is not a contract. The contract is a command that exits non-zero in this worktree and will be re-run after the patch. Write that command into a file so later review, CI, and any model see the same bytes.

# Proposed capture: store stdout, stderr, and the exit code beside the tree.
mkdir -p .repro
printf '%s\n' 'python -m pytest tests/test_retry.py::test_timeout_default -q' > .repro/failing_cmd.sh
chmod +x .repro/failing_cmd.sh
set +e
.repro/failing_cmd.sh > .repro/before.txt 2>&1
echo $? > .repro/before.exit
set -e
cat .repro/before.exit

If the exit file does not contain a non-zero status, stop immediately. The environment does not yet reproduce the bug, and a patch would be guesswork. Common misses include the wrong extra, a missing locale, and a test that only fails under a specific language minor version.

Truncate the log before anyone pastes it into a review tool. Long traces bury the assertion, and remote reviewers should not need the full pip wheel noise.

# Proposed helper: keep the packet readable.
from pathlib import Path

LIMIT = 80
log = Path(".repro/before.txt").read_text(errors="replace").splitlines()
Path(".repro/before.tail.txt").write_text("\n".join(log[-LIMIT:]) + "\n")

Keep the patch inside the failing command's neighborhood

Once the command fails, search only the modules that command imported. A model asked to fix retries will often rewrite the public client class, add a helper the suite never calls, or change default arguments that other tests rely on. The human author should produce the smallest diff that turns the captured command green.

# Example investigation commands, not a diagnosis of any real project.
python -c 'from retry_client import Client; print(Client.__module__)'
git grep -n "timeout" -- "*.py"
git grep -n "def test_timeout_default"

After editing, re-run the same script without changing its arguments. If unrelated files appear in git diff --stat, reset them before the packet is built. Formatters that touch the whole tree belong in a separate commit, and only when the project already requires that formatter in CI.

set +e
.repro/failing_cmd.sh > .repro/after.txt 2>&1
echo $? > .repro/after.exit
set -e
test "$(cat .repro/after.exit)" = "0"
git diff --stat
git diff -- "*.py" "tests/*.py"

A second local command is worth running when the project already documents it: the unit node that failed, then the nearest file or folder that guards regressions. Expanding to the full suite too early hides the original contract inside unrelated noise.

Build a review packet instead of uploading the tree

The packet is a single markdown file plus the unified diff. It should be small enough to read in one sitting and free of secrets, absolute home paths, and vendor directories. Optionally append short quotes from CONTRIBUTING.md, the issue's acceptance comments, and the project's test matrix file.

# Proposed packet builder. Review the output before any remote upload.
{
  echo "# Review packet for issue 1842"
  echo
  echo "## Reproduction"
  echo "- commit: $(git rev-parse HEAD)"
  echo "- command: \`$(tr '\n' ' ' < .repro/failing_cmd.sh)\`"
  echo "- exit before: $(cat .repro/before.exit)"
  echo "- exit after: $(cat .repro/after.exit)"
  echo
  echo "## Failing command output (truncated)"
  echo '```

'
  tail -n 80 .repro/before.txt
  echo '

```'
  echo
  echo "## Diff"
  echo '```

diff'
  git diff -- "*.py" "tests/*.py"
  echo '

```'
} > .repro/REVIEW_PACKET.md

Do not paste entire policy documents into the packet. A model that receives twenty pages of contributing guide will ignore the one rule that matters, such as no new runtime dependencies. Quote the rule in one block, then keep the rest of the file mechanical.

Files that must stay out of the packet

.env, tokens, cookie jars, and editor settings
node_modules, .venv, vendor, and built wheels
Unrelated dirty files from a formatter or a failed rebase
The full git object store; the worktree diff is enough

Decision table: what a model is allowed to do

This table is the working artifact of the workflow. The model does not replace the test runner, and it does not replace the project's hosted CI. It reviews a bounded packet after the failing command has already turned green.

Packet section	Human owner	Model role	Reject if
Failing command	Contributor	May suggest a narrower test node id	The command was never run locally
Diff	Contributor	Review only: style, missing tests, API breakage	The model authors files not in the packet
CONTRIBUTING quotes	Contributor	Check the diff against quoted rules	The quote is paraphrased from memory
CI matrix note	Contributor	Flag untested runtimes	The model claims a matrix cell passed without a log

Treat any suggested patch from the model as a comment, not as a commit. Re-run .repro/failing_cmd.sh after every accepted edit, even when the comment looks purely stylistic. Style-only rewrites still break public signatures in libraries with downstream users.

Where a free model and a free server fit

Some contributors cannot install every CI runtime on a laptop, and some want a second pass on the packet before the pull request. Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode offers free model access and a free server option, which can re-run the captured command and return comments on REVIEW_PACKET.md rather than on a full repository archive. Contributors who already have a green local command can use that optional remote pass, then still wait for the project's hosted CI before pinging maintainers.

The useful split stays strict. The server re-runs .repro/failing_cmd.sh in a clean environment. The model receives the packet and returns a review, not a rewritten tree. If the remote command disagrees with the local exit codes, the packet is incomplete and the pull request should wait.

Limitations and who should skip this

The workflow does not claim that a free server matches every OS, libc, or language version in a project's matrix. It does not verify performance, concurrency, or network flakiness that only appears under load. Models still invent APIs that look plausible in a diff comment and still miss license headers the project requires.

Skip this approach in the following cases:

The bug is under a security embargo, or the fixture contains production data.
Reproduction needs hardware or licensed runtimes the remote environment cannot provide.
The change is a documentation typo that never needed a model or a second server.
The contributor cannot compare the model's review against the actual test log.

A review packet is extra ceremony for a one-character fix in a personal script. It earns its keep on libraries with downstream users, public APIs, and CI matrices that a laptop will not cover. It also fails closed: if the captured command never failed, there is no patch to review, and if the packet includes secrets or the entire tree, the remote step should not run.

Closing sequence

The sequence is isolate, capture, patch, packet, then optional remote review. Maintainers can re-run one command and read one diff without reconstructing the author's laptop. Models stay useful when they are downstream of evidence, not when they are asked to invent the evidence from a repository zip. Those two stops prevent the most common failure in AI-assisted open source work: a fluent diff that never reproduced the bug.

Last Green Tag First: Bisect an OSS Regression Before the Patch Review

Blake Yang — Thu, 10 Sep 2026 04:19:16 +0000

A mid-size Python CLI received an issue that empty HTTP_PROXY values were now being sent to child processes as blank strings. A well-meaning contributor asked a coding model for a patch and received a thirty-line rewrite of the process launcher. The pull request looked clean in the diffstat, yet it also changed timeout handling that had been stable since the v3.2.0 tag. Maintainers closed the PR after a reviewer traced the real regression to a one-line defaulting change in a logging helper.

That pattern is now common in open-source inboxes, because models optimize for the symptom in the issue title. They rarely search for the first commit that changed documented behavior on a tagged release. A cheaper sequence is mechanical: prove the last release green, prove HEAD red, then bisect to one commit. Only after that pin should any model be invited to propose or review a patch.

Treat the last release tag as the control group

Issue templates rarely state which release last worked for the reporter, so contributors patch HEAD as if the entire tree were guilty. Maintainers then pay for that guesswork during review, because unrelated helpers get rewritten in the name of safety. A tagged control group makes the claim falsifiable for anyone who can install the test extra. If the last tag already fails the oracle, the report is not a regression and the patch needs a different contract.

The control group also keeps a later model review inside a small packet of evidence. A coding model that never sees the blamed commit will invent architecture around the issue title. The bisect log is a short, citable artifact that can travel with the pull request. Reviewers can replay it without inheriting a dirty local worktree or a half-applied virtualenv.

Worked example: empty proxy values

The files below are a labeled worked example, not a claim about any production codebase. The documented CLI contract treats a missing or empty HTTP_PROXY as unset for child processes. After a commit that “simplified defaults,” empty strings started leaking into subprocess environments. An oracle that fails for that reason alone keeps the later patch inside one function.

# proj/envutil.py
from __future__ import annotations

import os
from typing import Mapping


def child_env(overrides: Mapping[str, str] | None = None) -> dict[str, str]:
    """Build an env dict for child processes.

    Empty values are treated as unset, matching the v3.2 CLI contract.
    """
    env = {k: v for k, v in os.environ.items() if v != ""}
    if overrides:
        for key, value in overrides.items():
            if value == "":
                env.pop(key, None)
            else:
                env[key] = value
    return env

# tests/oracle_empty_proxy.py
from __future__ import annotations

from proj.envutil import child_env


def test_empty_http_proxy_is_unset(monkeypatch):
    monkeypatch.setenv("HTTP_PROXY", "")
    env = child_env()
    assert "HTTP_PROXY" not in env


def test_missing_http_proxy_stays_missing(monkeypatch):
    monkeypatch.delenv("HTTP_PROXY", raising=False)
    env = child_env()
    assert "HTTP_PROXY" not in env

A contributor who starts from the issue text might wrap the whole launcher in extra filtering. The oracle above fails for one contract only, which is the point of the control group. Neighboring timeout helpers stay out of the diff until bisect says they belong there. Maintainers can read the oracle in a minute and know what “fixed” is supposed to mean.

A maintainer-runnable bisect script

git bisect run needs a deterministic exit code from a script that does not care about chat history. Exit 0 means the bug is absent, exit 1 means the bug is present, and exit 125 means the tree cannot be tested. Any other code aborts the search and leaves reviewers with a half-finished log. The script below is meant to live on the topic branch, or to be pasted into the pull request body.

#!/usr/bin/env bash
# scripts/bisect-repro.sh
# Worked example: replayable oracle for an empty-proxy regression.
set -euo pipefail

ROOT="$(cd "$(dirname "$0")/.." && pwd)"
cd "$ROOT"

if [[ ! -f pyproject.toml && ! -f setup.cfg && ! -f setup.py ]]; then
  echo "skip: Python project metadata missing in this commit" >&2
  exit 125
fi

python -m pip install -e ".[dev]" -q || exit 125

# Isolate the oracle from the rest of a possibly red suite.
if python -m pytest -q tests/oracle_empty_proxy.py; then
  exit 0
fi
exit 1

# Record the control group, then search toward HEAD.
git fetch --tags --prune
git checkout v3.2.0
bash scripts/bisect-repro.sh    # must exit 0; otherwise this is not a regression

git checkout main
bash scripts/bisect-repro.sh    # must exit 1; otherwise the bug is not on HEAD

git bisect start
git bisect bad HEAD
git bisect good v3.2.0
git bisect run bash scripts/bisect-repro.sh
git bisect log > bisect.log
git bisect reset

The 125 skip path matters on real repositories, where merge commits and missing extras would otherwise be labeled good. A skipped commit keeps the search honest instead of poisoning the good set with an unbuildable tree. After the pin, git show --stat on the blamed SHA and git tag --contains tell reviewers which releases already shipped the fault. Those two commands belong in the pull request body beside bisect.log.

Decision table for when a model may review

The table is the workflow gate, not a style preference for commit messages. If a row says the model stays out, the contributor keeps gathering evidence instead of drafting a rewrite. Tokens are still wasted when the blamed commit is unknown and the oracle is not replayable. Maintainers can apply the same rows without reading any chat transcript.

Observation after the oracle	What to do next	Model review allowed
Last tag fails the oracle	Not a regression; write a characterization test instead	No
`HEAD` passes the oracle	Cannot reproduce; freeze locale, cwd, and fixture paths	No
Oracle is flaky on the same commit	Fix seed, time, and network before any search	No
Bisect lands on a merge commit	Bisect the merged topic branch by parent	No
Pin is one function in one file	Draft a minimal patch against that hunk	Yes, scoped packet only
Pin touches generated code or lockfiles	Re-run codegen; do not hand-edit the artifact	No
Pin is a dependency bump	Read the upstream changelog; consider pinning	Changelog only

Most rejected AI patches fail the first three rows and still look productive in a chat window. The pull request then arrives without a replayable pin, so reviewers repeat the archaeology. The boring gate is the feature: it names the moment when a model is allowed to speak. Until that row is reached, the only artifacts that matter are the oracle and the bisect log.

Draft the patch against the blamed commit

Once bisect names a commit, the patch should restore the documented contract with the smallest hunk that makes the oracle pass. Drive-by refactors in neighboring helpers should stay out of that commit, even when a model offers them as cleanup. git range-diff helps when two drafts rewrite the same idea, because it shows whether the second draft still targets the blamed lines. The topic branch can start from main while the PR body still cites the pin.

git checkout -b fix/empty-http-proxy main

# After editing proj/envutil.py, keep the oracle green and the suite green.
python -m pytest -q tests/oracle_empty_proxy.py
python -m pytest -q

git add proj/envutil.py tests/oracle_empty_proxy.py scripts/bisect-repro.sh
git commit -m "fix: treat empty HTTP_PROXY as unset in child env

Regression introduced in <bisect-sha>.
Oracle: tests/oracle_empty_proxy.py
Replay: bash scripts/bisect-repro.sh"

# Optional: compare two drafts without losing the pin.
git range-diff main fix/empty-http-proxy-v1 fix/empty-http-proxy-v2

The commit message is part of the artifact, because maintainers should not reconstruct the bisect from a chat export. Citing the SHA, the oracle path, and the replay command is enough for a reviewer who never saw the original thread. A second draft that drops that citation is a weaker patch even if the diffstat looks smaller. Range-diff is then used to recover the pin rather than to debate formatting.

A scoped review packet, not a whole-tree dump

After the pin, a coding model is useful as a second reader, not as an archaeologist for the whole repository. The packet should contain the blamed commit, the oracle, the proposed diff, and the documented contract. It should not contain secrets, .env files, private traces, or unrelated application code. If any of those files are required to reproduce, the review stays on a private runner.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

MonkeyCode's free model access is enough for this step when the packet stays small and the three checks below are explicit. The operator-supplied free server option is relevant when a laptop cannot install the project's native extras, because bisect must run against a clean clone rather than a dirty local venv. The model is asked only whether the diff restores the oracle, whether it widens public surface, and whether the commit message cites the bisect SHA.

A review prompt that stays inside that scope looks like the following template. It is not a recorded production session, and it is allowed to answer that evidence is missing.

You are reviewing an OSS regression patch. Do not propose a rewrite.

Inputs:
1) git show --stat <bisect-sha>
2) git show <bisect-sha> -- proj/envutil.py
3) tests/oracle_empty_proxy.py
4) git diff main...HEAD

Checks:
- Does the diff make the oracle pass for empty and missing HTTP_PROXY?
- Does the diff change any function that the blamed commit did not touch?
- Does the commit message cite the bisect SHA and the replay script?
- List any remaining public-API risk in one short bullet list.

If evidence is missing, say "insufficient evidence" instead of guessing.

That last instruction is the reason bisect happens before review, because models fill gaps with plausible architecture. Maintainers then spend the review cycle undoing those invented helpers and restored “simplifications.” A packet that may answer “insufficient evidence” is cheaper than a confident patch against the wrong layer. The bisect log remains the source of truth if the model and the contributor disagree.

Limitations

This workflow assumes a deterministic oracle and annotated tags that can serve as a control group. Projects that never tag releases cannot form that group without an agreed good SHA written into the issue. Flaky suites poison git bisect run, because a random red on a good commit sends the search the wrong way. Native extensions, network fixtures, and GPU tests often need 125 skips, and those skips can leave a wide blame range instead of a single commit.

Remote review has a second constraint that no free-tier workspace removes for the contributor. Embargoed security issues, private customer traces, and credentials in fixtures should not leave the maintainer's machine. The bisect script also does not replace project CI; it only names the first bad commit before CI is asked to judge a patch. Generated code and lockfile bumps still need their own toolchain, not a model-authored edit.

Who should not use this approach

First-time contributors who have not yet run the project's test extra should not start with a model review of a guessed diff. Feature requests that add behavior, rather than restore a tagged contract, need a design thread instead of a bisect log. Documentation-only changes and typo fixes do not benefit from this ceremony and will annoy maintainers if the PR is padded with scripts. Teams under a disclosure embargo should keep both the oracle and the patch on private runners.

The durable output is a replayable script, an oracle that fails for one reason, and a commit that cites the first bad SHA. Maintainers can ignore the model entirely and still trust the pin on a clean checkout. Contributors who already reproduce locally can attach bisect.log and the replay command so the next reviewer does not repeat the search.

The Review Cited a Helper That Never Shipped: Assumption Audits Before an OSS Patch

Blake Yang — Wed, 09 Sep 2026 15:38:35 +0000

A mid-size Python library merged a cache around its parser after a polite, confident review. Continuous integration stayed green because the new helper lived only inside the incoming diff. Two days later a maintainer bisected a production traceback and found the praised name only in review notes. The model had recommended load_cached_grammar(), a function that never existed on main, and the contributor had implemented the suggestion. Tests asserted against the invented helper, public docs never mentioned it, and callers had no honest way to discover the private surface.

That failure is common when a model reviews an open-source patch against a vague issue title. Plausible names, error codes, and fixtures appear in the comment thread even though git grep cannot find them. The workflow below treats invented symbols as a merge blocker. It reproduces the bug, drafts a minimal patch, and then audits every review claim against the tree before anyone clicks merge.

Why invented helpers survive CI

Green CI does not prove that a review comment described the real codebase. A test imported from the patch can exercise a helper that the patch itself just created. Maintainers reading the model output often treat fluent advice as local knowledge. The result is a private API that no release tag, changelog, or type stub can explain.

Open-source patches also travel with incomplete context. The model may see a hunk, a stack trace, and a one-line issue title. It rarely sees neighboring modules, historical names, or the public export list. Filling those gaps with confident guesses is the default behavior, not a rare hallucination.

Pin the tree before any model reads the diff

Start from a clone at the issue’s reported revision, not from an already edited working tree. Record the remote, the default branch, and the exact commit that still fails. The commands below are a worked example against a fictional parser library named tokparse; treat paths as placeholders, not as a published project.

git clone https://example.com/oss/tokparse.git
cd tokparse
git fetch origin
git switch --detach origin/main
git rev-parse HEAD > /tmp/repro.sha
git status --porcelain

If the report names a release tag, check that tag out instead of main. Mixed trees hide whether a failure belongs to the bug or to local edits. Keep secrets, tokens, and untracked editor files out of the worktree so later review prompts cannot leak them.

Reproduce, then freeze the failing command

Write the smallest command that fails on the pinned commit and save it as a script. Do not let a model invent a test runner, extra pytest flags, or a Docker image that the repository does not already document. Read CONTRIBUTING, tox.ini, pyproject.toml, and .github/workflows first.

# example: documented in the repo's CONTRIBUTING.md
python -m venv .venv
. .venv/bin/activate
pip install -e ".[dev]"
pytest tests/test_parser.py::test_trailing_comma -q
echo $? > /tmp/repro.exit

If that test does not exist yet, add one failing assertion that uses only public APIs already imported by neighboring tests. Label the new test as a proposal until it fails for the reported reason. A passing characterization of the wrong behavior is useful; a passing test of an invented helper is not.

# proposal: tests/test_parser.py
# Uses only Parser and ParseError, both imported by existing tests.
from tokparse import Parser, ParseError

def test_trailing_comma_is_a_parse_error():
    parser = Parser()
    try:
        parser.parse("{"a": 1,}")
    except ParseError as exc:
        assert "trailing" in str(exc).lower()
        return
    raise AssertionError("expected ParseError for a trailing comma")

Run the test twice on the pinned commit. If the result flips, stop and isolate fixtures before any review model sees the tree. Flaky reproduction turns invented helpers into noise that nobody can falsify.

Draft the smallest patch that makes the frozen command pass

Edit only the module that already owns the failing path. Avoid drive-by renames, formatter-only hunks, and new files whose names the issue never required. After the frozen command passes, produce a unified diff against the recorded SHA.

git add tests/test_parser.py src/tokparse/parser.py
git diff --cached > /tmp/oss.patch
git diff --cached --stat

Keep the patch nearby as a file, not only as a chat attachment. File-backed diffs can be grepped, hashed, and reapplied. Chat-backed diffs silently pick up extra prose that later looks like source.

Artifact: an assumption log plus a symbol gate

The original artifact in this article is a two-file gate. The first file is a human-edited assumption log. The second is a small checker that extracts added names from the patch and rejects review comments that cite missing symbols.

Assumption log template

Copy the block below into assumption-log.md beside the patch. Fill every row before a model is allowed to comment. Mark unknown items as unknown instead of guessing a library convention.

# Assumption log for /tmp/oss.patch
# Pinned SHA: (paste from /tmp/repro.sha)

| ID | Claim | Evidence command | Status |
| --- | --- | --- | --- |
| A1 | Public parser API is `Parser.parse` | git grep -n "class Parser" src | verified |
| A2 | Trailing commas raise `ParseError` | git grep -n "class ParseError" src | verified |
| A3 | No cache helper exists on main | git grep -n "load_cached_grammar" . | absent |
| A4 | Tests run under pytest from tox.ini | sed -n "1,80p" tox.ini | verified |
| A5 | Issue asks for a public cache | unknown; issue body never says cache | blocked |

Status values are only verified, absent, or blocked. blocked means the patch must not add the claimed behavior. Models are not asked to resolve blocked rows; humans are.

Extract names the patch itself introduces

The checker below is a proposal script. It reads a unified diff and prints added identifiers so a later review cannot treat them as pre-existing APIs.

# proposal: tools/extract_added_symbols.py
import re
import sys

IDENT = re.compile(r"\b([A-Za-z_][A-Za-z0-9_]{2,})\b")
KEYWORDS = {
    "def", "class", "return", "import", "from", "self", "None",
    "True", "False", "with", "as", "try", "except", "raise",
}

def added_lines(diff_text: str):
    for line in diff_text.splitlines():
        if line.startswith("+++") or line.startswith("---"):
            continue
        if line.startswith("+") and not line.startswith("+++"):
            yield line[1:]

def main(path: str) -> None:
    text = open(path, encoding="utf-8").read()
    found = []
    for line in added_lines(text):
        for match in IDENT.findall(line):
            if match not in KEYWORDS:
                found.append(match)
    unique = sorted(set(found))
    for name in unique:
        print(name)

if __name__ == "__main__":
    main(sys.argv[1])

python tools/extract_added_symbols.py /tmp/oss.patch > /tmp/added.syms
sort -u /tmp/added.syms

Verify every review claim with git grep

After a model (or a human) writes review comments, save them as plain text. Then run each cited identifier against the tree at the pinned SHA, not against the index that already contains the patch, unless the comment explicitly talks about new code.

# proposal: tools/verify_review.sh
set -euo pipefail
SHA=$(cat /tmp/repro.sha)
COMMENTS=${1:-/tmp/review.comments}

# Extract backtick-quoted names from review prose.
grep -oE '`[A-Za-z_][A-Za-z0-9_]+`' "$COMMENTS" \
  | tr -d '`' | sort -u > /tmp/cited.syms

while read -r name; do
  [ -n "$name" ] || continue
  hits=$(git grep -n -F "$name" "$SHA" -- src tests || true)
  added=$(grep -Fx "$name" /tmp/added.syms || true)
  if [ -z "$hits" ] && [ -z "$added" ]; then
    echo "INVENTED: $name"
  elif [ -z "$hits" ] && [ -n "$added" ]; then
    echo "NEW_IN_PATCH: $name"
  else
    echo "PRESENT: $name"
  fi
done < /tmp/cited.syms

Any INVENTED line is a failed gate. Delete that review comment, or rewrite it until git grep can show a file and a line. NEW_IN_PATCH lines are allowed only when the assumption log already accepted that new name.

Decision table for keeping a review comment

Review comment type	Evidence required	Action
Mentions a function, class, or constant	`git grep` hit on the pinned SHA, or a `NEW_IN_PATCH` row already accepted	keep or rewrite
Mentions a test file or fixture	path exists in `git ls-tree -r --name-only $SHA`	keep or rewrite
Asks for a new helper not in the issue	assumption log row is `blocked` or `absent`	discard
Style-only advice that rewrites unrelated hunks	`git diff --stat` would grow beyond the failing module	discard
Claims CI already covers a branch	job name appears in `.github/workflows`	keep only with a file citation
Security or secret-handling advice	human maintainer review; do not run extra scanners on a shared machine	escalate, do not auto-apply

The table is the review policy. Models do not get a vote on discarded rows. Humans may still apply a discarded idea later, but only after a new assumption row is verified.

Where a free model actually helps

Disclosure: This article was prepared as part of MonkeyCode's product outreach. After the assumption log and symbol files exist, a free model can review /tmp/oss.patch against those files instead of against an empty prompt. MonkeyCode’s free model access and free server option are relevant here only as a place to run that constrained pass and to execute pytest plus verify_review.sh on a clean checkout. The model prompt should attach the patch, the assumption log, the added-symbol list, and a hard rule: cite a path or mark the comment as speculation.

You are reviewing an OSS patch. You may cite identifiers only if they
appear in assumption-log.md, added.syms, or the attached diff.
If you need a helper that is not listed, write SPECULATION and stop.
Do not invent test runner flags. Do not invent CI job names.
Return comments as: path:line: severity: text.

The server side of the loop is just a clean clone, the frozen test command, and the grep gate. It is not a substitute for the project’s own CI. If the repository contains credentials, private submodules, or production data dumps, skip shared machines and run the same scripts locally.

Limitations

Symbol presence is not semantic correctness. A helper can exist and still be the wrong layer for the bug. Grep also false-positives on comments, dead code, and similarly named tests. Binary files, generated protobufs, and vendored trees need a different inventory than the script above. The workflow does not prove license, CLA, or maintainer-tone issues. It only blocks a frequent class of fluent, wrong review comments.

The loop also costs a checkout and a full test run before any model token is spent. Tiny typo patches do not repay that cost. Neither do issues that already include a failing test, a one-line fix, and a maintainer who asked only for a rebase.

Who should not use this approach

Skip the assumption audit when the change is a spelling fix in documentation with no code hunks. Skip it when the project forbids AI-assisted review in CONTRIBUTING. Skip shared free servers when the reproducer needs customer fixtures or secrets. Skip it when the reporter already attached a bisect SHA, a failing test, and a two-line patch that git apply --check accepts. In those cases a human maintainer is faster than a gated model pass.

Contributors who already pin a SHA, freeze a failing command, and grep review comments against the tree can run the same constrained pass with MonkeyCode’s free model access on the free server option, then throw away every INVENTED line before opening the pull request.

DEV Community: Blake Yang

Ship an OSS Patch With a Reviewer Question Bank, Not Just a Diff

Unexplained diffs stall because review is interrogation

The artifact: REVIEWER_QUESTIONS.md

Six steps from a failing command to a fail-closed gate

1. Freeze the failing command

2. Keep every hunk load-bearing

3. Collect exit codes instead of vibe screenshots

4. Pack the diff and draft questions

5. Fill answers with file:line citations

6. Refuse to open the pull request while TODO remains

Pass rules for each required question

Limitations

Who should skip this packet

A One-Command Reproducer Kit That Survives a Clean OSS Checkout

Dirty trees hide the actual bug

What the kit must contain

A one-command template

Expected versus actual, in the repo

Decision table: when the kit is honest

Run the kit where the laptop cannot help

Free-model review of the kit, not the vibes

Only then write the patch

Limitations

Who should skip this approach

Classify the Public Surface Before an OSS Crash Fix Hits Review

Why small crash fixes still stall on versioning

Four stages before gh pr create

Stage 1: Reproduce on the reported tag

Stage 2: Patch the owning module only

Stage 3: Bind the new behavior with the project runner

Stage 4: Classify public surface before the pull request

Artifact: the Semver Impact Card

Extract candidate exports from the unified diff

Falsify the bump with a free model after tests pass

Decision table for the proposed bump

Sample pull request body

Limitations

Who should skip this packet

What the pull request should link

Declare the Behavior Delta Before Maintainers Open an OSS Bugfix

Treat the patch as a declared behavior delta

The oracle packet

README.md as the human index

Step 1: freeze the failing invocation

Step 2: capture the before side without editing it

Step 3: apply the smallest patch that should move the delta

Step 4: capture the after side with the same invocation

Step 5: publish an allowlist, not a vibe

Step 6: review the packet against the git hunks

Decision table for the packet

Limitations

Who should skip this approach

Pin the Crash in a Characterization Test Before You Patch Upstream

Happy-path diffs still bounce in review

Capture the crash before production edits

A three-ref behavior grid

Let a free model review the oracle, not the patch

Decision table for the oracle loop

Limitations

Who should not use this approach

Trace Callers Before the Diff: A Blast-Radius Card for Shared OSS Helpers

Why a green neighbor test is not blast-radius proof

The blast-radius card maintainers can replay

Harvest call sites from the tree, not from chat

A small classifier script for the harvest

Freeze a contract table before the first production hunk

Patch against the card, then move the tests that own the rows

Frozen-prompt review after the card is complete

Decision table for this workflow

Limitations and who should skip this card

The Hunk Necessity Test: Prove Every Line of an OSS Patch Is Load-Bearing

Why per-hunk evidence beats a narrated patch

Gate zero: the pinned test must fail before the patch

The harness: split, reverse-apply, rerun

Reading the verdict table

Where a free model actually helps, and where it must not decide

Running the loop on a free server

Limitations and who should not use this

A pre-push checklist

Four stages before `gh pr create`