An OSS patch stays reviewable when a human owns every edit. Model output belongs in a hypothesis file, not the tree. Maintainers merge tested evidence instead of untested model suggestions.
Chat-authored diffs hide invented helpers and missing tests. Reviewers then spend cycles proving the patch is even real. The project loses time on scope that nobody requested.
This workflow freezes a review contract before any model sees the diff. The contributor authors the production change by hand. A model may only emit named, testable hypotheses.
Why chat-authored OSS patches stall
A model does not share the project's history of rejected APIs. It also does not feel the cost of a noisy pull request. It often optimizes for a complete-looking but unbounded diff.
OSS maintainers review ownership, scope, and proof together. They do not review autocomplete confidence as evidence. Ungrounded comments therefore fail at first human pass.
Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode offers free model access and a free server option. Those options can host the later review pass only.
The local contract still governs every accepted line. The method remains useful without any hosted model. Remove the product and the gate still holds.
Core rule
Treat every model comment as a named hypothesis. Require a frozen path and a testable claim. Discard any comment that invents a symbol.
The model never writes files in the working tree. The model never receives tokens or private mail. The model never proposes files outside the allowlist.
Artifact: the review contract
Keep one small contract beside the clone. Pin the public issue, commands, and allowlisted paths. The review bundle is generated from this file.
# example only: .review/review-contract.yaml
issue_url: "https://github.com/example/libparse/issues/842"
base_sha_file: ".review/base.sha"
repro_cmd: "python -m pytest tests/test_parse_empty.py -q"
smoke_cmd: "python -m pytest tests/test_public_api.py -q"
allow_paths:
- "src/libparse/scanner.py"
- "tests/test_parse_empty.py"
forbid_substrings:
- "API_KEY"
- "BEGIN PRIVATE KEY"
- "xoxb-"
hypotheses_md: ".review/hypotheses.md"
This YAML is a proposed example, not a production manifest. Replace the issue URL with a real public ticket. Keep embargoed security issues out of this file.
Numbered workflow
1. Detach at a recorded upstream SHA
Fetch the default branch and detach HEAD there. Record the SHA in .review before any local edit. Later review comments must name that recorded SHA.
git fetch origin
git checkout --detach origin/main
mkdir -p .review
git rev-parse HEAD > .review/base.sha
git status --porcelain
An empty status is required at this point. Unrelated dirty files leak into the later bundle. Clean the tree before writing the review contract.
2. Pin the public issue and the allowlist
Copy only the public issue URL into the contract. List the few paths that may still change. Add the test paths that prove the reported bug.
Do not add the entire src tree to the allowlist. Broad allowlists recreate the same noisy patch problem. Two or three paths usually bound a single bug.
3. Write the failing test on the frozen SHA
Add one test that follows the public report. Run the recorded repro command on the frozen SHA. Store the non-zero exit code beside the contract.
python -m pytest tests/test_parse_empty.py -q
echo $? | tee .review/repro.exit
test "$(cat .review/repro.exit)" != "0"
Stop when the test already passes on base. The report then lacks a local failing reproducer. Do not ask a model to invent the missing proof.
4. Author a minimal production edit by hand
Change only allowlisted production files after the red test exists. Keep public signatures stable unless the issue requires a break. Avoid new exported helpers that expand public surface.
git diff --stat -- src/libparse/scanner.py tests/test_parse_empty.py
git diff -- src/libparse/scanner.py tests/test_parse_empty.py > .review/allowlisted.diff
Inspect the stat output before any model run. Extra files mean the allowlist was ignored. Restore those files instead of explaining them later.
5. Re-run the same commands after the edit
Run the original repro command until it passes. Run the smoke command against public API tests. Record both exit codes under the .review directory.
python -m pytest tests/test_parse_empty.py -q
echo $? | tee .review/fix.exit
python -m pytest tests/test_public_api.py -q
echo $? | tee .review/smoke.exit
Both recorded files must contain a zero exit code. Non-zero smoke results block the review pass. The model must not debug a still-failing tree.
6. Build a secret-stripped bundle
Concatenate the contract, the SHA, and the allowlisted diff. Strip known credential markers before the model sees the bundle. Refuse to continue if a marker remains.
# example helper: build_review_bundle.sh
set -euo pipefail
BUNDLE=".review/bundle.md"
{
echo "# Review bundle"
echo "base_sha: $(cat .review/base.sha)"
echo
echo "## contract"
cat .review/review-contract.yaml
echo
echo "## allowlisted diff"
cat .review/allowlisted.diff
} > "$BUNDLE"
if grep -E "API_KEY|BEGIN PRIVATE KEY|xoxb-|AKIA" "$BUNDLE"; then
echo "secret marker found; refuse model review" >&2
exit 1
fi
This script is an unexecuted example helper only. Projects should extend the marker list for their stack. Never paste .env files into the bundle.
7. Ask the model for hypotheses, not patches
Use a strict review prompt against the bundle only. Demand hypothesis IDs, frozen paths, and testable claims. Ban unified diffs in the model reply.
The example prompt text follows in the next block. Label that block as a proposed prompt only. Do not treat it as a vendor template.
You are a second reviewer, not a patch author.
Comment only on files listed in allow_paths.
Each finding must use this shape:
H-001 | path | claim | test idea
If you cannot test a claim, write DISCARD.
Do not invent files, APIs, or commit messages.
Do not return a patch.
MonkeyCode free model access can run that prompt. A free server option can keep the pass off the laptop. The contributor still types every accepted edit locally.
Write the model output into .review/hypotheses.md for audit. Keep the raw reply next to the contract. Reviewers then see what the model actually said.
8. Accept, convert, or discard each hypothesis
Read the hypothesis file from top to bottom. Accept a claim only when the path is allowlisted. Convert that claim into a test or a tighter assertion.
# example: .review/hypotheses.md
H-001 | src/libparse/scanner.py | empty input still returns None | assert scanner.parse("") == []
H-002 | src/libparse/scanner.py | invented helper skip_bom exists | DISCARD
H-003 | src/libparse/compat.py | missing py2 branch | DISCARD
H-002 invents a helper the patch never added. H-003 names a file outside the allowlist. Both of those rows are discarded without further discussion.
Add a test for H-001 if the claim is true. Re-run repro and smoke commands after that test. Drop the hypothesis if the new test cannot fail on the base SHA.
9. Open the pull request with the hypothesis log
Commit only allowlisted production paths plus the new tests. Attach the contract and hypothesis log in the PR body. State the base SHA in the first paragraph.
BASE=$(cat .review/base.sha)
git add src/libparse/scanner.py tests/test_parse_empty.py
git commit -m "Fix empty input handling in scanner.parse
Base SHA: ${BASE}
Hypotheses accepted: H-001
Hypotheses discarded: H-002 H-003"
Do not commit .review bundles that contain issue text copies. Some projects treat those copies as review noise. Paste a short summary instead of the raw bundle.
Decision table
Each row maps a hypothesis shape to a required human action. The reason column blocks extra debate during triage. Apply the table before any extra model round.
| Hypothesis shape | Human action | Reason |
|---|---|---|
| Path outside the allowlist | Discard | Scope is unbounded |
| Symbol not present in the diff | Discard | Model invented an API |
| Claim with no test idea | Discard | Untestable review noise |
| Test idea fails on the base SHA | Add the test | Claim is real |
| Test idea passes on the base SHA | Discard | Claim is not a regression |
| Suggestion of a full rewrite | Discard | Authorship left the human |
| Mention of secrets or keys | Stop the review | Bundle leaked |
Use the table during the hypothesis triage pass. Do not negotiate rows that say Discard. The contract is cheaper than a long maintainer thread.
Limitations
This workflow does not replace a maintainer review. It only reduces ungrounded model text in the PR. Hidden behavioral bugs can still pass the smoke file.
The path allowlist cannot express semantic code ownership. A one-line change may still break release notes. Public API policy still needs a human decision.
Free model access and a free server option do not define correctness. They only host the optional hypothesis review pass. Latency, context limits, and model quality vary and are not measured here.
The example commands assume a Python pytest tree. Other languages need their own equivalent frozen test commands. The contract format stays the same across languages.
Who should not use this approach
Do not use this flow on embargoed vulnerability reports. Those reports must not enter a third-party model. Follow the project's private security list instead.
Do not use this flow when the license forbids AI review. Some maintainer communities ban model-assisted patches as policy. Read CONTRIBUTING.md before sending any hosted prompt.
Do not use this flow to generate the production patch. Authorship of the diff must stay with the contributor. A model-authored tree still fails this method.
Do not use this flow on huge refactors. Allowlists of two paths cannot bound a rewrite. Split the work or skip the model pass.
Closing
Keep the human as the author of every OSS edit. Freeze a contract, prove the bug, then request hypotheses. Convert only testable claims and discard the rest.
The hypothesis file is the durable review artifact. It survives after any hosted model session ends. Maintainers can audit it without sharing a chat log.
Top comments (0)