A patch without a tiny fixture wastes maintainer time. Upstream reviews stall when failure still spans five modules. Shrink the reported bug to one fixture first.
The fixture then becomes the only allowed proof. Production code then changes against that proof alone. Model comments arrive later as review notes only.
Capture the report, not the guess
Issue text often mixes symptoms, theories, and stack noise. A contributor copies the raw command and output. Dates, host names, and secrets get stripped first.
mkdir -p .repro
printf '%s\n' "$ISSUE_URL" > .repro/source.txt
# Replace make test-one with the upstream single-test command.
script -q -c 'make test-one TEST=failing_case' .repro/raw.log
grep -E 'FAIL|Error|Exception' .repro/raw.log > .repro/errors.txt
That log is evidence, not a patch plan. No production file is edited in this step. Theories from the issue stay out of .repro.
Shrink until one fixture fails
A full suite failure hides the real unit. The next job is strict input reduction. Keep deleting inputs until one fixture still fails.
- Copy the failing input into
tests/fixtures/shrink/case.json. - Run only the project's documented single-test command.
- Drop unused fields while the same assertion still fails.
- Stop when one more deletion changes the error class.
- Record the surviving command in
.repro/repro.lock.
# Example only: swap in the repository's real runner.
npm test -- tests/shrink.test.js
# or
pytest -k test_shrink_case -vv
Label every runner as project-specific before sharing it. Do not invent a framework the repo lacks. A fixture that still needs three services is not small.
Reduce network, clock, and filesystem seams next. Prefer in-process fakes the project already ships. Live cloud calls do not belong in the shrink loop.
Stop shrinking with a decision table
Reduction without a stop rule deletes the bug. Use the table below as the exit gate. Each row is a hard stop, not a preference.
| Observation | Action |
|---|---|
| Next deletion changes the error class | Keep the previous fixture and stop |
| Fixture still needs live network | Replace bytes with a recorded stub |
| Two unrelated assertions fail | Split fixtures; patch only one |
| Result depends on test order | Stop; isolate the test first |
| Clock skew changes the assertion | Freeze time with the project's helper |
| Fixture spans two packages | Cut imports until one package remains |
Copy the chosen row into .repro/stop.txt. Reviewers then see why shrinking ended. A missing stop row means the fixture is still too large.
# .repro/repro.lock (example layout, not a real hash)
test_cmd: pytest -k test_shrink_case -vv
fixture: tests/fixtures/shrink/case.json
stop_row: Next deletion changes the error class
delta: .repro/delta.md
Pin the command text, not a remembered alias. Aliases rot across contributor shells. The lock file must replay on a clean clone.
Record a behavior delta table
Maintainers read tables faster than long essays. The fixture needs a before row and after row. Expected values come from public docs or the failing assertion.
| Probe | Command or call | Before (failing) | After (required) |
|---|---|---|---|
| Status | cli parse case.json |
exit 2 | exit 0 |
| Message | stderr line 1 | invalid token |
(empty) |
| Return | parse() |
throws ParseError
|
object with ok: true
|
This table is the only in-scope contract. Any extra behavior change needs a new row. Silent extras stay out of the pull request.
# .repro/delta.md
probe: parse(case.json)
before: ParseError("invalid token")
after: { ok: true, value: 42 }
out_of_scope: logging format, metrics counters
Keep the out-of-scope list short and explicit. Reviewers then ignore unrelated diffs with cause. Changelog noise is not a fixture win.
Patch only the path the fixture touches
Open the stack from the fixture's failing assertion. Change the smallest function that makes the after column true. Leave nearby refactors for a later pull request.
// Example fixture driver. Adapt names to the upstream tree.
const { parse } = require('../src/parse')
const caseJson = require('./fixtures/shrink/case.json')
test('shrunk fixture reaches documented success', () => {
const result = parse(caseJson)
expect(result).toEqual({ ok: true, value: 42 })
})
Run the single test until it passes. Then run the nearest package suite. A green fixture with a red neighbor package is incomplete.
git diff --stat
git diff -- src tests/fixtures/shrink tests/shrink.test.js
python - <<'PY'
# Example: fail the workflow if extra paths appear.
import subprocess, sys
allowed = {"src/parse.py", "tests/fixtures/shrink/case.json", "tests/shrink.test.js"}
out = subprocess.check_output(["git", "diff", "--name-only"], text=True)
files = {line for line in out.splitlines() if line}
extra = sorted(files - allowed)
if extra:
print("extra paths:", extra)
sys.exit(1)
print("diff stays inside the fixture graph")
PY
If the stat list grows past the fixture path, stop. Split the extra files out before review. Import-graph creep is a common silent expansion.
# Example import-graph check for a Python package.
python -c "import ast, pathlib, sys
root = pathlib.Path('src')
# Proposal only: replace with the project's module walker.
print('review import edges from the fixture next')"
Treat that walker as unlabeled proposal code. Real graphs come from the upstream package layout. Do not paste a generic graph into a foreign repo.
Re-read the diff without writing new code
A second pass should hunt extra behavior, not new features. Check license headers, changelog rules, and the commit template. Match the repository's existing style, not a personal one.
git log -5 --oneline -- src
git diff --check
# Example: reuse the project's own lint entry if it exists.
[ -f Makefile ] && grep -n '^lint' Makefile || true
Free model access can help that second pass. A contributor pastes the fixture, the delta table, and the diff. MonkeyCode provides free model access and a free server option for that review chat.
Disclosure: This article was prepared as part of MonkeyCode's product outreach.
The model does not own the fixture. Review notes that lack a table row get discarded. Suggested rewrites stay out of the tree until a human replays the test.
Review prompt (example, not executed):
1. Read tests/fixtures/shrink/case.json
2. Read .repro/delta.md and .repro/stop.txt
3. Read the git diff only
4. List claims that lack a delta row
5. List files outside the fixture import graph
Keep the prompt numbered and boring on purpose. Creative refactors from a model are noise here. The optional review chat can use that free server.
Publish the reproduction with the pull request
Maintainers should replay the fixture without a meeting. Put the shrink command in the pull request body. Link the issue URL already stored in .repro/source.txt.
## Reproduction
1. `git checkout <this-branch>`
2. Run the locked single-test command in `.repro/repro.lock`
3. Confirm the after-column in `.repro/delta.md`
## Out of scope
- logging format
- metrics counters
Do not attach secrets or machine-specific paths. The fixture must run on a clean clone. Host-local caches are not reproduction.
git status --porcelain
test -z "$(git status --porcelain)" && echo 'tree clean before push'
A dirty tree hides fixture files from reviewers. Commit the lock, delta, stop row, and test together. Skip generated caches and editor swap files.
Limitations
This workflow assumes a test runner already exists. Greenfield repos without assertions need tests first. Flaky fixtures also fail the method, because shrink cannot split bugs from races.
Large performance issues rarely collapse to one JSON file. Multi-service outages need a different harness. Generated code and vendored trees also resist tiny fixtures.
Model review is optional and advisory only. It does not replace CI, owners, or maintainer judgment. Free access and a free server do not imply named models, quotas, hardware, or uptime here.
Who should skip this
Do not use fixture shrink for speculative cleanups. Do not use it when the issue already names a one-line typo. Embargoed security reports need the project's private channel, not a public fixture.
Contributors who cannot run the project tests locally should stop. A model chat is not a substitute for that run. The fixture and the delta table remain the patch.
Top comments (0)