Test-First OSS Patches: Write the Failing Fixture Before You Touch the Code

A bug report is not reproducible until a test fails on your machine. Before that moment it is just a story. This workflow converts the story into a fixture, a captured failure signature, and a two-commit patch. The first commit holds only the test. The second holds only the fix.

The loop takes about twenty minutes on a small parser bug. It saves maintainer attention, which is the scarcest resource in open source. It also makes review faster, because the reviewer sees the broken behavior encoded as code.

Why "it does not work" is not a bug report

Three patterns consume most open-source issue threads. The reporter describes an outcome without an input. The maintainer reproduces on a different version and sees nothing. The patch lands without a test, so the bug returns in six months.

A failing test removes all three. It pins the input, the version, and the expected output in one file. The test then guards the fix forever, at zero recurring cost.

The fixture contract

A usable fixture satisfies four properties. It is deterministic, so repeated runs give the same result. It is offline, so it needs no network or credentials. It is ordered, so it never depends on filesystem enumeration. It is minimal, so one assertion explains the whole failure.

If a candidate fixture breaks any of these properties, it is a flaky test in disguise. Fix the fixture before touching product code.

Step 1: freeze the reporter's input

Create a branch and a fixture directory. Copy the smallest input that still triggers the bug.

git switch -c repro/4821-trailing-comma
mkdir -p tests/regression/fixtures/issue_4821

Store the raw input exactly as the reporter supplied it. Strip secrets, absolute paths, and hostnames. Add the expected output beside it, taken from documented behavior rather than from current output.

The expected value matters. Copying current output encodes the bug as correct behavior. Derive the expectation from the spec, the docs, or a sibling parser that behaves properly.

Step 2: assert the wrong behavior on purpose

The test module below is illustrative. Adjust imports and names to your package. It was not executed in this article.

# tests/regression/test_issue_4821_trailing_comma.py
from pathlib import Path

from yourpackage.config import parse_record  # adjust import

FIXTURES = Path(__file__).parent / "fixtures" / "issue_4821"


def read(name: str) -> str:
    return (FIXTURES / name).read_text(encoding="utf-8")


def test_trailing_comma_keeps_previous_field():
    record = parse_record(read("input.conf"))
    assert record.as_text() == read("expected.txt")

One test, one assertion, one fixture pair. Name the file after the issue number so the link between code and thread stays obvious.

Run it and expect a failure:

python -m pytest -q tests/regression/test_issue_4821_trailing_comma.py \
  2>&1 | tee /tmp/4821.before.txt

If this test passes, you have not reproduced the bug. Stop. Do not write a patch against an unproven theory.

Step 3: capture the failure signature

Extract the assertion lines into a small file. This becomes the before-image of the fix.

grep -E '^(E |FAILED|AssertionError)' /tmp/4821.before.txt | head -20

The signature should mention the exact field and value that differ. A signature of "assert False" tells you the fixture is too coarse. Split it until the diff names one behavior.

Step 4: patch the smallest surface

Change the least code that makes the assertion true. Resist refactoring nearby functions in the same branch. Reviewers approve small diffs faster, and small diffs revert cleanly.

Most parser bugs need a guard clause or an index fix. If the patch grows past roughly fifteen lines, the fixture may be testing two bugs. Split it into two issues and two branches.

Step 5: verify with two commands, then commit in two parts

Run the targeted test first, then the full suite.

python -m pytest -q tests/regression/test_issue_4821_trailing_comma.py \
  2>&1 | tee /tmp/4821.after.txt
python -m pytest -q 2>&1 | tail -5

Diff the failure signatures to prove the assertion lines disappeared:

diff <(sed -n '/^E /p' /tmp/4821.before.txt) \
     <(sed -n '/^E /p' /tmp/4821.after.txt)

Then commit the test and the fix separately:

git add tests/regression
git commit -m "test: reproduce #4821 with trailing comma fixture"
git add src/
git commit -m "fix: ignore trailing comma in config parser"

Two commits let a reviewer verify the test fails before the fix and passes after it. That is the whole argument, expressed as git history.

Step 6: use model assistance where it cannot lie

Drafting fixtures is repetitive work, which is where model assistance helps most. Useful prompts ask for edge cases the reporter did not mention: empty values, duplicate keys, unicode boundaries, very long lines. A model can also summarize a long issue thread into candidate inputs.

Verification stays human. Every proposed fixture is confirmed by running the pytest command above and reading the output. A model cannot observe your failure signature, so it cannot confirm reproduction.

MonkeyCode provides free model access and a free server option, per the operator. That matters for this loop because the workflow needs only a shell, git, and Python. Free model access lowers the cost of generating candidate edge-case fixtures. The free server option lets the reproduce-patch-verify cycle run on a remote machine, keeping a local checkout clean while the fixture set grows. Free-tier limits and server terms change over time, so check the current product page instead of trusting a number in a blog post. Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Decision table: patch, ask, or close

Report state	Action	Reason
No version string or no input	Ask for both	Nothing is reproducible yet
Fixture fails on main branch	Write the patch	Reproduction is proven
Fails only on the reporter's machine	Request environment dump	Likely platform-specific
Test passes after the patch, suite green	Open the pull request	Evidence is complete
Cannot reproduce, suite green	Comment with the fixture, keep the issue open	Guessing produces bad fixes

Post the fixture in the thread even when reproduction fails. A maintainer on another platform can run it in one command.

Limitations and who should skip this

Fixtures do not fit every bug class. Flaky concurrency bugs, hardware-dependent crashes, and performance regressions need repeated runs and benchmark harnesses instead. Private data cannot be committed as a fixture, so a redacted or synthetic input is required.

Security fixes need extra care. A public failing test can describe a vulnerability before a patched release exists. In that case keep the reproduction private and coordinate disclosure with the maintainers first.

Teams with a strict contribution guide should follow it. Some projects require tests in a specific directory or forbid new fixture folders. Read CONTRIBUTING.md before creating structure.

Closing

Write the fixture, watch it fail, fix the smallest surface, watch it pass. That order is the entire method, and it scales from a two-line parser bug to a protocol change. If a reader wants a second opinion on a candidate fixture, the free model access in MonkeyCode is one reasonable place to look for edge cases before running them locally.