Reproduce First, Then Patch: An OSS Fix Loop

Open source patches fail for missing reproduction, not missing cleverness. Maintainers spend scarce hours proving other people's bugs. A contributor who ships a failing test first earns review time.

Local model review can polish a packet after tests pass. It cannot invent a missing reproduction on its own.

Why first patches stall

First-time pull requests often mix three separate jobs. They narrate a bug, guess a fix, and skip proof. Reviewers then reproduce the issue on their own machines.

That delay burns goodwill and wastes CI minutes. AI-authored diffs make the pile larger still. Cheap text does not cheapen verification.

This loop splits the work into four gates. Each gate writes a file another person can replay. Humans own reproduction, isolation, and tests.

A local model may inspect the packet last. It stays optional and secondary. The first three gates remain mandatory.

Gate 1: Reproduce on a clean clone

Start from a throwaway directory. Do not edit sources yet. Clone at the revision named in the issue.

Record the exact command that fails. Save stdout and stderr beside the notes. A memory of the crash is not evidence.

mkdir -p /tmp/oss-repro && cd /tmp/oss-repro
git clone --depth 50 https://github.com/example/project.git src
cd src
git checkout <sha-or-tag>
# template install; swap for the project docs
python -m venv .venv
. .venv/bin/activate
pip install -e ".[dev]"
pytest tests/test_reported_bug.py -q | tee /tmp/oss-repro/repro-before.txt

Write REPRO.md before any source change. State the command, the expected failure, and the observed output. Stop if the clean clone already passes.

The report is wrong, or the issue is already fixed. A reproduction that needs production secrets is not portable. Prefer a fixture or a recorded response instead.

Maintainers cannot run a bug they cannot see. Log redaction belongs in this gate. Paste no tokens into the packet.

Steps

1. Clone shallow history for the reported tag.
2. Install only the documented development extra.
3. Run the claimed failing command twice.
4. Save stdout and stderr beside REPRO.md.
5. Halt if the failure cannot be repeated.

A labeled REPRO.md template looks like this. Fill every field before branching.

# REPRO.md — template, not a live bug report
Issue: #1234
Revision: abcdef0
Command: pytest tests/test_reported_bug.py -q
Expected: non-zero exit, assertion on empty header
Observed: file /tmp/oss-repro/repro-before.txt
Secrets required: none

Gate 2: Isolate one patch

Create a branch named after the issue number. Change one concern only. Do not reformat unrelated files in the same commit.

Do not upgrade dependencies unless the bug requires it. Extra churn hides the real fix. Reviewers will bounce the pile.

git switch -c fix/issue-1234
# add the failing test first, then the smallest production change
git add -p
git diff --cached --stat
git commit -m "fix: reject empty header for issue 1234"

Keep the commit message boring and specific. Name the behavior, not the emotion. Point at the reproduction file from the body.

Reviewers search logs. They do not search essays. Split the work if two modules move at once.

A drive-by cleanup hides the real fix. Later bisection becomes impossible after mixed commits. One issue gets one branch.

Steps

1. Branch from the reproduced revision.
2. Add a failing regression test first.
3. Change the smallest production surface.
4. Use git add -p to drop noise.
5. Commit with the issue id in the subject.

Gate 3: Test on purpose

Run the original failing command until it passes. Then run the project's default suite. Capture both logs as files, not screenshots.

A green local run that is not saved will be disputed. Chat claims are not artifacts. Files are.

pytest tests/test_reported_bug.py -q | tee /tmp/oss-repro/repro-after.txt
pytest -q | tee /tmp/oss-repro/full-suite.txt
git status --short

Add a regression test that would have failed at Gate 1. The test name should mention the issue id. Future bisect runs need that stable name.

Do not treat coverage percentage as proof of the fix. Coverage can rise while the original bug remains. The named regression test is the proof.

The next example is a labeled template. It was not executed for this article.

# tests/test_issue_1234.py — template only
def test_issue_1234_rejects_empty_header():
    from project.parser import parse_header
    try:
        parse_header("")
    except ValueError:
        return
    raise AssertionError("empty header must raise ValueError")

Steps

1. Re-run the Gate 1 command on the branch.
2. Confirm the named test now passes.
3. Run the default suite to catch fallout.
4. Store both logs under a packet directory.
5. Refuse to proceed if either command is missing.

Gate 4: Build a review packet

Maintainers review packets, not chat vibes. Pack four files into one directory. Include reproduction, diff, test logs, and intent.

mkdir -p /tmp/oss-repro/packet
cp REPRO.md /tmp/oss-repro/packet/
git diff main...HEAD > /tmp/oss-repro/packet/change.diff
cp /tmp/oss-repro/repro-after.txt /tmp/oss-repro/packet/
cat > /tmp/oss-repro/packet/INTENT.md <<'EOF'
Issue: #1234
Behavior before: parse_header("") returns a placeholder
Behavior after: parse_header("") raises ValueError
Risk: parser only; no public API rename
EOF

This packet is the working artifact of the loop. A reviewer can clone, apply the diff, and replay commands. No extra thread is required for the first pass.

Keep the packet under version control or attach it. Do not bury it in a screenshot thread. Paths beat prose.

Optional local model pass

A packet can feed a local review model after tests pass. The model should read the diff and the reproduction. It should not invent extra features or refactors.

It should flag missing tests, secret leaks, and unrelated churn. Anything else is noise. Discard noise.

Some contributor setups route that packet through MonkeyCode. Disclosure: This article was prepared as part of MonkeyCode's product outreach. Operator-supplied details include free model access and a free server option.

Those options can host a review pass without a paid API. This article does not claim model names, quotas, or hardware. It does not claim permanence or benchmark wins.

A useful prompt stays narrow. Paste the packet files only. Ask for findings, not a rewrite.

You are reviewing an open-source bugfix packet.
Read INTENT.md, REPRO.md, and change.diff.
Report only: missing tests, leaked secrets, unrelated files,
or a reproduction that cannot be replayed.
Do not suggest refactors.
Do not rewrite the patch.
If reproduction is incomplete, say so in one sentence.

Save the model output as MODEL_NOTES.md beside the packet. Treat the notes as a linter, not a merge vote. Ignore style lectures that contradict the project guide.

Keep any finding that cites a path and a line. Free local review helps when CI minutes are scarce. It does not prove correctness or domain intent.

It does not replace a maintainer who knows the module. The named test still owns the truth. The model only nits the packet.

Artifact: oss-fix-loop.sh

The script below is a labeled template. Replace the clone URL and test command. It fails closed when REPRO.md is missing.

#!/usr/bin/env bash
# oss-fix-loop.sh — contributor gates before a pull request
# labeled template; not wired to a live repository
set -euo pipefail

ROOT="${ROOT:-/tmp/oss-fix-loop}"
ISSUE="${ISSUE:?set ISSUE to the tracker number}"
CLONE_URL="${CLONE_URL:?set CLONE_URL}"
TEST_CMD="${TEST_CMD:?set TEST_CMD}"

mkdir -p "$ROOT/packet"
cd "$ROOT"

if [[ ! -d src/.git ]]; then
  git clone "$CLONE_URL" src
fi

cd src
if [[ ! -f REPRO.md ]]; then
  echo "Write REPRO.md with the failing command first." >&2
  exit 1
fi

echo "== gate 3: named test after the patch =="
bash -lc "$TEST_CMD" | tee "$ROOT/packet/test-after.txt"

echo "== packet =="
if git rev-parse --verify main >/dev/null 2>&1; then
  git diff main...HEAD > "$ROOT/packet/change.diff"
else
  git diff > "$ROOT/packet/change.diff"
fi
cp REPRO.md "$ROOT/packet/"
{
  echo "Issue: #$ISSUE"
  echo "Test command: $TEST_CMD"
  echo "HEAD: $(git rev-parse --short HEAD)"
} > "$ROOT/packet/INTENT.md"

echo "Packet written to $ROOT/packet"

Run it with environment variables, not hidden flags. The command below is a template. Swap the URL and test.

chmod +x oss-fix-loop.sh
ISSUE=1234 \
CLONE_URL=https://github.com/example/project.git \
TEST_CMD='pytest tests/test_issue_1234.py -q' \
./oss-fix-loop.sh

The script does not open a pull request on purpose. A packet that cannot be inspected should not be published. Humans still click the compare view.

Decision table

Use this table before asking for review. It blocks common false starts. It also blocks scope creep from model notes.

Signal	Action	Do not
Clean clone already passes	Update the issue and stop	Open a cosmetic PR
Repro needs production secrets	Build a fixture first	Paste credentials into the packet
Diff spans unrelated modules	Split commits	Request review on the pile
Named test still fails	Keep patching	Ping maintainers
Named test passes, suite passes	Build the packet	Rewrite the whole file
Model notes cite a leaked token	Rotate and strip	Commit the notes as-is
Model notes demand a refactor	Ignore the extra scope	Expand the pull request

The table keeps humans in charge of scope. Models vote on nits. They do not merge.

Limits

This loop assumes a test runner exists in the repo. Many docs and script repos lack one. A recorded before-and-after command is a weaker substitute.

Local models miss project history and sacred ugly code. They flag legal patterns that look noisy. They also miss logic bugs that tests already cover.

Free model access and a free server option reduce cost only. They do not settle license or privacy questions. Do not send private customer code to hosted models without a written policy.

Public issue patches are the intended input for this loop. The loop also assumes the contributor can run the project. Heavy monorepos still need a real toolchain.

A cloud editor does not fix a wrong reproduction. Missing fixtures stay missing. The packet will show that gap.

Who should not use this approach

Skip the model pass on embargoed security issues. File those through the project's security contact. Skip the whole loop for a one-line docs typo.

A reproduction file for a typo wastes everyone involved. Skip the loop when the test command cannot run locally. Asking a model to imagine a failure produces fiction.

Maintainers will still ask for logs after that fiction. Teams with signed commits and mandatory reviewers keep those rules. This packet is extra evidence, not a policy bypass.

Newcomers who have not cloned the project yet should stop at Gate 1. A generated patch without a local failure is noise. Do not skip ahead to the model.

After the packet exists

Open the pull request with the four files linked. Point at Gate 1 in the first paragraph. Point at the named test in the second paragraph.

Reviewers should replay the commands, not decode a story. A short contribution should stay short under review. The loop exists to prevent a second rewrite.

No latency numbers are claimed in this writeup. No model leaderboard is claimed either. The only local metric is a replayable packet.

Readers who already keep reproduction files can add a local review pass. The free server option can host that pass for public patches. The packet still matters if that pass is skipped.