Most failed open-source contributions fail at triage, not at coding. A patch is only as good as the reproduction standing behind it. Write the contract first, then the diff.
That order matters more this month. Post-event contribution waves bring volume, and coding agents now produce plausible patches at almost no cost. Maintainers receive diffs that look reasonable and cannot be reproduced. The bottleneck is no longer writing code. It is proving the bug exists and that the change removes it.
This article describes a triage loop: reproduce, contract, patch, verify, then ask a model to review. It assumes one shell, one repository clone, and no special tooling.
The core rule
An issue becomes work only when a command fails for a stated reason. Everything before that is a conversation.
So the loop starts with a repro contract. That is a small JSON file answering five things: which commit, which environment, which command, which expected output, which actual output. If any field is empty, the issue is not ready for a patch.
Step 1: Capture the environment fingerprint
Environments drift. Record the toolchain before touching code. The script below is a starting point, not a universal solution; extend the version checks for your stack.
#!/usr/bin/env bash
# fingerprint.sh — unexecuted example, adapt to your stack
set -euo pipefail
repo="${1:?usage: fingerprint.sh <repo-dir>}"
cd "$repo"
commit=$(git rev-parse HEAD)
dirty=$(git status --porcelain=v1 | wc -l | tr -d ' ')
os=$(uname -srm)
printf '{\n'
printf ' "commit": "%s",\n' "$commit"
printf ' "dirty_files": %s,\n' "$dirty"
printf ' "os": "%s",\n' "$os"
printf ' "node": "%s",\n' "$(node -v 2>/dev/null || echo none)"
printf ' "python": "%s",\n' "$(python3 -V 2>&1 | awk '{print $2}' || echo none)"
printf ' "captured_at": "%s"\n' "$(date -u +%Y-%m-%dT%H:%M:%SZ)"
printf '}\n'
Run it, save the output as triage/fingerprint.json, and commit nothing. Triage artifacts stay local.
Step 2: Write the contract file
Keep the contract boring and machine-readable. Reviewers and scripts both read it.
{
"issue": 4127,
"base_commit": "9f3c2ab",
"command": "npm test -- --run tests/parser.spec.ts",
"expected": "parser accepts trailing comma",
"actual": "SyntaxError at line 88",
"exit_code_base": 1,
"repro_time_seconds": 42,
"status": "confirmed"
}
Allowed status values are unconfirmed, confirmed, flaky, and needs-info. Only confirmed unlocks patching. That restriction is the whole point of the file.
Step 3: Make reproduction one command
A contract that needs six manual steps is not reproducible. Wrap it in a runner that always starts from a clean state.
#!/usr/bin/env bash
# verify.sh — unexecuted example
set -euo pipefail
repo="${1:?usage: verify.sh <repo-dir>}"
contract="$repo/triage/contract.json"
cmd=$(jq -r .command "$contract")
base=$(jq -r .base_commit "$contract")
wt="$(mktemp -d)/repro"
git -C "$repo" worktree add --detach "$wt" "$base" >/dev/null
set +e
( cd "$wt" && eval "$cmd" ) 2>&1 | tee "$repo/triage/base.log"
code=${PIPESTATUS[0]}
set -e
echo "base exit code: $code"
git -C "$repo" worktree remove --force "$wt"
If the base exit code matches the contract, the report is trustworthy. If it is zero, the reporter's environment differs and the issue goes back to needs-info.
Step 4: Decide with a budget, not with enthusiasm
Time-box triage. Pick a limit you can defend, such as 45 minutes, and let the result decide the next move.
| Signal after triage | Action |
|---|---|
| Base fails, patch passes, log stable | Open the pull request |
| Base fails intermittently | Isolate the flake first; do not patch |
| No failing command after the time box | Ask for a minimal case; park the issue |
| Fix needs API or design changes | Open a discussion, not a pull request |
| Model suggests a cause, no command confirms it | Keep it as a hypothesis |
The fourth row prevents the most common bad outcome. A correct fix in the wrong shape costs maintainers more than an unfixed bug.
Step 5: Put free model access where it is safe
Models are good at reading long threads and proposing candidate causes. They are bad at knowing whether your repository actually fails. Place them after the contract, never before.
A workable handoff sends four things only: the contract JSON, the last 80 lines of the base log, the list of files touched by the failing code path, and the project test command. Whole-repository dumps waste context and leak unrelated code.
Ask for three candidate causes, one minimal diagnostic command per cause, and likely file locations. Then record the answers in a plain ledger with explicit status.
## Hypotheses for issue 4127
- [verified] Trailing comma parser stops at token `,` before `)` — confirmed by `node -e` probe
- [rejected] Node version mismatch — base fails on both 20 and 22
- [open] Loader caches the old grammar — needs a cache-clearing run
Only [verified] lines may appear in the pull request body. Everything else stays in the ledger. This single rule keeps generated text out of public claims.
Free model access fits this step well, because triage consumes many small reads, not one large one. A clean remote machine also helps, since the contract runner needs a toolchain you may not have locally.
This is where the MonkeyCode project enters the loop. Its free model access covers the hypothesis step above, and its free server option gives the contract runner an isolated place to execute. According to the operator, the offering includes 10 million free tokens and a free server; verify the current limits and terms on the project's own documentation before depending on them, since free tiers change.
Disclosure: This article was prepared as part of MonkeyCode's product outreach.
Step 6: Verify the patch against the same contract
Reuse verify.sh with the patch branch and compare both logs. Same command, same environment, same expected output. A patch that only passes on your machine is an anecdote.
If the base log is flaky, stop. A green run on a moving baseline proves nothing, and reviewers will rightly reject it.
Limitations and who should skip this
- Security-sensitive reports need private handling. Do not paste logs or code into a third-party model.
- Projects without a test harness get less value. The contract still helps, but reproduction cost rises sharply.
- The time box misses slow-burn bugs. Deep performance issues do not fit a 45-minute gate.
- Model output is a hint. It never replaces a command with a recorded exit code.
- Free-tier terms, quotas, and server capacity are external facts. They can change without notice.
Skip this loop if you cannot run local commands, if your organization forbids sending code to external services, or if the repository has no stable entry point to execute.
What changes after a week
Triage becomes cheaper and pull requests become smaller. Maintainers argue about design, not about whether the bug exists. Contributors stop guessing which issue is worth an evening.
The contract file is the artifact that survives. It travels into the pull request, the review, and the eventual regression test. If you want to try the loop, start with one issue and one contract.json, using free model access only for the hypothesis step, and compare the result against your usual process.
Top comments (1)
The fingerprint is the field that has saved me most, specifically the version one. Twice a "bug" report turned out to be a report about a different file than the one I had open: once a stale copy on the host, once the wrong build in the folder. Before trying to reproduce anything, I open the artifact the person actually has and read its hash. If the fingerprint doesn't match, there is no bug to chase yet, only a delivery to check.