Reproduce the Reporter's Container Before You Patch an OSS Bug

Most rejected open-source patches fail for a boring reason. The bug was never reproduced on the reporter's environment. The diff looks reasonable, the tests pass locally, and the maintainer closes the pull request with one line: "cannot reproduce."

This post describes a five-step loop that turns an issue report into a patch a maintainer can verify. The loop is: capture an environment ledger, bake a reproducer container, require a failing exit code, patch the smallest diff, then re-run the same script. A hosted model can help review the evidence bundle at step five. It should never be asked to guess at the bug.

Nothing here depends on the project's language. The examples use Bash, Docker, and Python.

1. Capture the environment ledger before reading the diff

Issue reports contain the environment in prose. Contributors skim that prose and start editing. That is the first failure.

Extract the environment into a machine-readable ledger first. The ledger records only facts the reporter supplied.

# tools/env-ledger.sh
set -euo pipefail

issue="${1:?usage: env-ledger.sh <issue.md>}"
out="env-ledger.json"

jq -n \
  --arg os    "$(grep -iPo '(?<=^OS:\s).*' "$issue" | head -1)" \
  --arg runtime "$(grep -iPo '(?<=^Runtime:\s).*' "$issue" | head -1)" \
  --arg version "$(grep -iPo '(?<=^Version:\s).*' "$issue" | head -1)" \
  --arg repro "$(grep -iPo '(?<=^Steps:\s).*' "$issue" | head -1)" \
  '{os:$os, runtime:$runtime, runtime_version:$version, repro_steps:$repro, captured_from:"issue"}' \
  > "$out"

grep -q '"os":""' "$out" && { echo "ledger incomplete: ask reporter"; exit 2; }
echo "ledger written to $out"

A missing field is a request for information, not an invitation to guess. Send the ledger back in the issue thread and ask the reporter to fill the gaps.

2. Bake a reproducer container from the ledger

The ledger is the only input to the container definition. No version comes from the contributor's laptop.

# repro/Dockerfile
FROM python:3.11-slim

ARG RUNTIME_VERSION
RUN pip install --no-cache-dir "acme-lib==${RUNTIME_VERSION}"

COPY repro/ /repro/
WORKDIR /repro
ENTRYPOINT ["bash", "run.sh"]

Pin one thing at a time. If the ledger says 3.9.7, the build fails loudly on 3.9.8. That failure is useful evidence, not an inconvenience.

Record the image digest, not just the tag. Tags move; digests do not.

docker build \
  --build-arg RUNTIME_VERSION="$(jq -r .runtime_version env-ledger.json)" \
  -t acme-repro:local repro/ | tee build.log

docker image inspect acme-repro:local --format '{{index .RepoDigests 0}}' \
  | tee image-digest.txt

The digest line belongs in the pull request body. A maintainer can rebuild the exact same image from it.

3. Require a failing exit code before touching source

A reproducer that exits zero proves nothing. The script must assert the bug is present.

# repro/run.sh
set -uo pipefail

python - <<'PY' > output.txt 2>&1
try:
    import acme
    acme.parse(b'{"ok": true, "retries": 0}')
    print("NO_CRASH")
except Exception as exc:
    print(type(exc).__name__, exc)
PY

cat output.txt

grep -q 'NO_CRASH' output.txt && { echo "BUG NOT PRESENT"; exit 1; }
echo "BUG PRESENT"; exit 0

The exit codes are inverted on purpose. exit 0 means the bug reproduced. exit 1 means the environment is wrong and the contributor must stop.

Run this before writing any patch. Save the output as before.log.

4. Patch the smallest diff, then re-run the identical script

Edit source only. Do not edit the reproducer to make it pass. The reproducer is the contract.

After the edit, the same script must print NO_CRASH and exit 1.

docker run --rm acme-repro:local > after.log; echo "exit=$?"
# before.log -> Exception ... exit=0
# after.log  -> NO_CRASH     exit=1

Two log files and two exit codes are the whole argument. The table below covers the drift cases that appear most often.

Drift source	Symptom	Evidence to add
Unpinned dependency	Passes locally, fails for reporter	pip freeze diff in the ledger
OS-specific path handling	Bug vanishes in the container	before.log with the original traceback
Locale or timezone	Intermittent output	Container TZ and LANG values
Reporter already patched	BUG NOT PRESENT on first run	Version string from the issue
Non-deterministic seed	Flaky reproducer	Seed value plus ten consecutive runs

If BUG NOT PRESENT appears, the ledger is wrong. Go back to step one. Do not weaken the assertion.

5. Give the reviewer an evidence bundle, not a repository

Step five is where hosted models help, and where they mislead. A model reading a whole repository invents context. A model reading six artifacts stays grounded.

The bundle is small and fixed:

• env-ledger.json
• image-digest.txt
• before.log and after.log
• the unified diff
• the reproducer script, unchanged

A hashing step keeps the bundle honest and detects accidental edits during review.

find bundle -type f -print0 | sort -z \
  | xargs -0 sha256sum > bundle/SHA256SUMS
sha256sum -c bundle/SHA256SUMS

This is where MonkeyCode's free model access is relevant. The operator offers free model access and a free server option, and I use the free model access to read bundles like this one. The free server option matters when a reproducer build is slow and the contributor does not want to occupy a laptop for an hour. The bundle stays the same either way; only the machine running the build changes.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Ask the model narrow questions. "Does after.log show the same code path as before.log?" is answerable. "Is this patch correct?" is not, and the answer will be confident and unreliable.

Limitations

This loop assumes the reporter can describe their environment. Vague reports need a round trip before any code changes.

It also assumes a deterministic bug. Race conditions and memory-pressure failures need repeated runs and a different evidence format.

Digest pinning adds minutes to every build. That cost is acceptable for a bug that blocked someone in production, and wasteful for a documentation fix.

The reviewer budget is real. Sending a large bundle to any hosted model costs time. Keeping the bundle small is part of the method, not a preference.

Who should not use this loop

Do not use it for trivial patches. Typos and doc edits do not need a container.

Do not use it when the project forbids new CI artifacts. Some maintainers reject added build files, and their contribution guide wins over this workflow.

Do not use it as a substitute for reading the code. The reproducer proves the bug exists. It says nothing about whether the fix is correct.

Closing

Environment drift is boring, and boring problems have checkable answers. Capture the ledger, bake the container, require the failing exit code, patch one diff, then re-run the identical script. The evidence bundle then goes to a reviewer with a question it can actually answer.

Readers who want to try this loop without provisioning anything can start from the free model access and free server option that MonkeyCode provides. The workflow above works without them; they only remove a local build step.