Require a Local Review Contract Before an OSS Pull Request

An OSS patch should not leave a fork until four artifacts exist. Those artifacts are reproduction notes, a bounded patch, tests, and a review contract.

A model may fill fields in that contract. A human maintainer still owns the merge decision.

This workflow keeps model output out of git history. It also keeps contributor judgment in one signed file. The contract is the only place a model is allowed to speak.

Why the contract exists

Maintainers do not merge confidence scores or vibes. They merge commands, paths, and observed test output. A raw chat transcript is not review evidence.

The contract records answers maintainers already expect in review. It does not invent extra narrative or scope. It stores commands, paths, and residual risk.

The next sections propose a local convention. These files are not a published industry standard. Teams should adapt field names to their project.

Current AI review chatter often hides missing tests. A filled chat window is easy to fake. A signed contract with rerun commands is harder to fake.

Artifact layout

Keep review files off the upstream default branch. Use a local directory that never ships by default.

.review/
  reproduce.md
  test-record.md
  review-contract.yml
  model-fill.yml
  series.diff
  SIGNED_BY.txt

The patch itself lives only in git commits. The review contract lives beside those git commits. Do not add .review/ unless the project asks.

Step 1: Reproduce the bug on a frozen tree

Clone the project at the reporter commit SHA. Do not patch the tree before reproduction. Write the failing command into reproduce.md first.

git clone https://github.com/example/project.git
cd project
git checkout --detach "$REPORTER_SHA"
git switch -c review/issue-1234

Record the environment in two short lines. Record the exact command that fails. Record the observed output, truncated if noisy.

# .review/reproduce.md
- sha: 9f3c1aa
- os: linux
- command: cargo test queue::drop_on_cancel -- --exact
- observed: assertion failed: ticket.cancelled
- notes: fails with RUST_TEST_THREADS=1 only

Stop if the failure does not reproduce locally. A remote model cannot review a ghost bug. Leave a comment on the issue instead.

This example is a proposed layout, not a run log. Replace the cargo command with the project runner. Do not invent timings or pass rates.

Step 2: Bound the patch to one behavior change

Create commits that a maintainer can read in order. One commit should add or fix tests. One commit should change production code.

Optional third commit updates nearby docs only. Reject mixed commits at this stage. A later model reviews each commit alone.

git add src/queue.rs tests/queue.rs
# split the index if both files landed together
git reset HEAD
git add tests/queue.rs
git commit -m "test(queue): cover cancel during drop"

git add src/queue.rs
git commit -m "fix(queue): honor cancel on drop"

git add README.md
git commit -m "docs(queue): note cancel on drop"

The oneline log from main must stay short. List two or three commits after the split. Keep the subject lines boring and specific.

git log --oneline origin/main..HEAD
git diff --stat origin/main...HEAD

If the stat output spans unrelated packages, stop. Split the work before any model pass. Do not ask a model to justify a mixed diff.

A stacked series also helps revert. Maintainers can drop the docs commit. They can keep the test commit during bisect later.

Step 3: Record a test result a stranger can rerun

Write the test record by hand. Paste command output without paraphrase. Keep the original failing command in the same file.

mkdir -p .review
{
  echo "date: $(date -u +%Y-%m-%dT%H:%M:%SZ)"
  echo "sha: $(git rev-parse HEAD)"
  echo "fail_cmd: cargo test queue::drop_on_cancel -- --exact"
  echo "pass_cmd: cargo test queue::drop_on_cancel -- --exact"
  RUST_TEST_THREADS=1 cargo test queue::drop_on_cancel -- --exact
} | tee .review/test-record.md

The record must include the failing command from step one. It must include the passing command after the fix. Both commands should share the same runner flags.

Label this block as unexecuted on a public repo. Replace cargo with the real project runner. Do not invent pass rates or machine specs.

If CI is required, capture one job log excerpt. Store only the failing test names. Do not dump secrets from the log.

gh run view --log --job test | tail -n 80 > .review/ci-excerpt.txt

That gh command is optional and proposed. Skip it when the issue is local only. Never paste tokens into the excerpt file.

Step 4: Freeze a review contract schema

The contract is YAML with empty string defaults. Empty fields block the pull request locally. A small script enforces the required keys.

# .review/review-contract.yml
issue: "https://github.com/example/project/issues/1234"
reporter_sha: "9f3c1aa"
patch_shas: []
scope:
  allowed_paths:
    - "src/queue.rs"
    - "tests/queue.rs"
    - "README.md"
  forbidden_paths:
    - "vendor/"
    - ".github/workflows/"
repro_command: ""
fail_output_excerpt: ""
pass_command: ""
pass_output_excerpt: ""
behavior_change: ""
public_api_change: "none"
residual_risk: ""
maintainer_answers:
  why_this_fix: ""
  why_not_larger_refactor: ""
  how_to_revert: ""
model_role: "fill_blanks_only"
contributor_attestation: ""

This schema is the original artifact for this workflow. It is intentionally boring and explicit. Boring fields are easier for humans to verify.

Copy the reporter SHA from step one. Leave patch_shas empty until commits exist. Fill patch_shas with git rev-parse after the split.

git rev-parse origin/main..HEAD > /tmp/patch_shas.txt

Step 5: Let a model fill blanks, not commits

Copy the contract to model-fill.yml before any model run. Send only that file plus the commit diffs. Do not paste the entire repository into the prompt.

cp .review/review-contract.yml .review/model-fill.yml
git diff origin/main...HEAD > .review/series.diff
git log -p origin/main..HEAD > .review/series-commits.diff

A proposed prompt, not a production prompt, follows. It forbids new files and invented commands. It also forbids commit rewrites.

Fill empty string fields in model-fill.yml.
Use only series.diff, reproduce.md, and test-record.md.
Do not invent commands that were not run.
Leave unknown fields as UNKNOWN.
Do not add paths outside allowed_paths.
Do not rewrite commit messages.
Return YAML only.

The contributor then diffs the fill against the original. Keep every human-run command in place. Replace model-invented commands with UNKNOWN.

diff -u .review/review-contract.yml .review/model-fill.yml

Reject any fill that adds new paths. Reject any fill that claims the model ran tests. The model did not run those tests.

Merge accepted fields by hand. Do not overwrite human command strings. The signed file later attests to that edit.

Where a hosted model fits

Local review still needs a place to run the fill step. Some contributors lack a spare local GPU. Some forks keep diffs and prompts on a small server.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode offers free model access and a free server option. That option can host this YAML fill step.

The contract, tests, and commits stay under contributor control. The model still does not own the merge.

Use a hosted fill only after steps one through four exist. Do not start the issue by chatting with a model.

Step 6: Validate the contract before opening the PR

The validator is a proposed shell script. It does not contact the network. It fails closed on empty required fields.

#!/usr/bin/env bash
# validate-review-contract.sh — proposed local gate, not production YAML parsing
set -euo pipefail
file=".review/review-contract.yml"
need=(
  issue reporter_sha repro_command fail_output_excerpt
  pass_command pass_output_excerpt behavior_change
  residual_risk contributor_attestation
)
for key in "${need[@]}"; do
  val="$(awk -F': ' -v k="$key" '$1==k{print $2; exit}' "$file" | tr -d '"')"
  if [[ -z "$val" || "$val" == "UNKNOWN" ]]; then
    echo "missing $key" >&2
    exit 1
  fi
done
while IFS= read -r p; do
  case "$p" in
    src/queue.rs|tests/queue.rs|README.md) ;;
    *) echo "path outside scope: $p" >&2; exit 1 ;;
  esac
done < <(git diff --name-only origin/main...HEAD)
echo "review contract ok"

Mark the script as a proposal only. Real projects should parse YAML with a real parser. The awk shortcut is for teaching.

Run the script on the filled contract. Sign the attestation after it passes. Only then open the pull request.

chmod +x validate-review-contract.sh
./validate-review-contract.sh
printf '%s\n' \
  'I ran the commands in test-record.md.' \
  'I read the model fill and discarded inventions.' \
  > .review/SIGNED_BY.txt

The PR body can quote three contract fields. It should not paste the model YAML. Maintainers should not have to read the fill file.

Step 7: Write the maintainer-facing PR body from the contract

Copy facts, not generated style. Keep the body shorter than the contract. Use the same commands as the test record.

Fixes #1234

Reproduction: `cargo test queue::drop_on_cancel -- --exact`
Change: honor cancel on drop in src/queue.rs
Tests: same command passes on the patch SHA
API: none
Revert: revert the fix commit; keep the test commit

This body is boring on purpose. Maintainers can rerun the listed command. They do not need a model summary.

If the tracker wants a checklist, quote contract keys. Do not paste residual model prose. Residual prose belongs in .review/model-fill.yml only.

Decision table

Situation	Action
Bug does not reproduce	Stop. Comment on the issue.
Diff crosses forbidden paths	Split or abandon the patch.
Model fill invents a command	Replace with UNKNOWN and rerun.
Tests pass only on one laptop	Do not open the PR.
Maintainer asks for a squash	Squash after review, not before.
Embargoed security patch	Skip hosted models entirely.

Limitations

This workflow does not prove correctness. It only proves that required fields were filled. UNKNOWN handling still depends on contributor honesty.

YAML parsed by awk is brittle. Nested lists need a real parser. The path check is a demo for three files.

A model can still hallucinate causal language. The contract reduces that risk. It does not delete it.

Hosted models also see the supplied diff. Secret patches do not belong in that prompt. Embargoed security issues stay offline.

Availability of a hosted fill is not a quality benchmark. Duration and hardware are out of scope. Do not treat a free fill host as proof.

The stacked commit rule can annoy squash-only projects. Follow the project guide after review. Do not fight a stated squash policy.

Who should not use this

Do not use this flow for security embargoes. Do not upload embargo diffs to any hosted model. Do not use this for drive-by refactors.

Skip it when the project already has a review schema. Do not run two parallel contracts. Skip it when the contributor cannot reproduce the bug.

New contributors who cannot run the test command should not sign. Signing without a local test run is the failure mode. The attestation is a statement of fact.

Skip it for generated code dumps with no tests. A contract cannot rescue an untested tree. Write the failing test first.

Closing

The useful order is reproduce, patch, test, then contract. Models enter at the contract step. They do not enter at clone time.

A pull request that cannot cite commands is not ready. A model comment that cannot cite those commands is noise.

Keep residual model noise in the YAML file. Keep the commit history only in git.