I Parked Every Suggested Diff Until a Scratch Host Replayed the Failure

Have you ever accepted a tidy suggested edit, only to watch the same test fail on a machine that started empty? I have, and the embarrassing part was that my own laptop had been quietly supplying the missing piece. The local tree already had packages, environment variables, and cached wheels that a clean runner would never inherit. So the rule I would repeat is simple: park the suggested diff until a scratch host replays the failure without those leftovers.

Why a green local run was the wrong comfort

Treat the currency bug below as a reconstructed walkthrough, not as a customer incident or a measured product trial. A checkout helper returned the wrong currency code when a locale package was missing, but my laptop already had that package. An assistant proposed a three-line guard that looked careful, and the local test went green in well under a second. Would you have merged that guard, or would you have asked which machine was actually being tested by the pass?

I did not merge it, because a green result on a dirty tree answers a different question from the one I had asked. I needed a blank environment to fail the same way before the patch, and then to pass for a reason I could name. A local pass cannot separate those stories when the host is already full of leftovers from older experiments. I wanted a note I could reread two days later, not a feeling that the diff looked small enough to trust.

The forty-eight hour notebook

I split the work into four blocks so I would not skip the boring capture step when a suggestion arrived. Each block had one job, and I wrote the result into a file instead of trusting a chat scroll to remember it. Nothing here is a benchmark, and I am not claiming timings, token totals, hardware sizes, or a permanent free tier. The artifact is a small gate over two text logs, plus a table for deciding where that replay should live.

Hours 0 to 8: freeze the failure

I checked out the revision that still failed, and I ran the single test that showed the wrong currency code. I saved stdout and stderr into a file named captured.txt, and I stored the interpreter version beside that log. I also stored a hash of the lockfile, because a later run on a different dependency set is not the same replay. Why keep that hash if the test name already matches, and the command line looks identical on both hosts?

A matching name can still hide an extra wheel, a different interpreter, or an environment variable you forgot to unset. I refused to ask for a patch until the captured file existed and a second local run produced the same failure text. That second local run is not the clean replay, and it only proves that the failure is stable on the dirty host. Have you ever skipped that step and then argued with a log that you can no longer reproduce at all?

python -m pytest tests/test_checkout_locale.py -q > captured.txt 2>&1
python --version > host.txt
python - <<'PY'
import hashlib, pathlib
p = pathlib.Path("requirements.lock")
print(hashlib.sha256(p.read_bytes()).hexdigest())
PY

Hours 8 to 24: replay on a host that does not know me

I created a throwaway virtual environment from the lockfile alone, with no extra index and no copied site-packages. Then I ran the same test there and saved the combined output as replay.txt for the gate to read. If that file did not still show a failure, I stopped, because I had not reproduced the bug I meant to fix. Have you noticed how often the apparent fix is just a package you forgot you had installed last month?

python -m venv /tmp/scratch-venv
/tmp/scratch-venv/bin/python -m pip install -r requirements.lock
/tmp/scratch-venv/bin/python -m pytest tests/test_checkout_locale.py -q > replay.txt 2>&1
python scratch_gate.py captured.txt replay.txt

When the laptop itself is part of the contamination, a fresh virtual environment is not always a strong enough wall. Editor plugins, shell exports, and a global package cache can still leak into a story you thought was isolated. A second host, even a small temporary one, removes those leftovers if you create it from the lockfile only. That is the moment a free remote runner becomes useful, provided you treat it as disposable and you keep secrets off it.

Hours 24 to 40: only then read the suggested diff

I sent the captured failure, the lockfile hash, and the test file, and I asked for a patch with no new dependency. The suggestion came back shorter than I expected, which is not a virtue, because short code can still assume the wrong host. I applied it on the scratch environment only, reran the test, and saved the new log as after.txt next to the replay. A pass on that host was interesting, but I still read the patch for imports and paths I did not recognize.

Would I have trusted the suggestion if the scratch host had stayed red after the patch was applied there? No, I would not, because a confident explanation does not change an exit code or repair a bad assumption. I kept the failing replay beside the passing log so I could see which lines actually moved between the two runs. The assistant was useful as a draft of the edit, and it was useless as a witness that the edit was safe.

Hours 40 to 48: what broke, and what I would repeat

Two things broke during the window, and both mistakes were mine rather than a surprise in the test runner. First, I almost compared logs by hash only, and a timestamp line made two equivalent failures look unrelated. Second, I nearly copied my shell history onto the scratch host, which would have smuggled the variable I was excluding. I would repeat the gate, the lockfile hash, and the rule that a patch waits for a failed replay and a later pass.

I would not repeat any flow that lets an assistant commit, or that sends private fixtures to a machine I do not control. I would also not repeat a comparison that treats a timestamped log as a fingerprint of the underlying failure. The useful repeat is boring on purpose, and that boredom is what keeps a tidy suggestion from skipping the line. If the note takes longer than the patch, that is a sign the patch was never the expensive part of the work.

The gate I kept

The script below is a local harness you can reread later, not a score and not a client for a hosted product. It exits zero only when the replay log still looks like a failure, which is what you want before applying a patch. It exits one when the clean run is already green, because that means the bug was not reproduced on that host. Run it yourself before you trust it, since your failure text may not contain the markers I happened to check for.

#!/usr/bin/env python3
"""Compare a captured failure with a clean replay. Local harness only."""
import hashlib
import json
import sys
from pathlib import Path

MARKERS = ("FAIL", "ERROR", "Error", "Traceback", "AssertionError")

def signature(path: Path) -> dict:
    text = path.read_text(encoding="utf-8", errors="replace")
    return {
        "sha256": hashlib.sha256(text.encode("utf-8")).hexdigest(),
        "failed": any(marker in text for marker in MARKERS),
    }

def main() -> int:
    if len(sys.argv) != 3:
        print("usage: scratch_gate.py captured.txt replay.txt", file=sys.stderr)
        return 2
    captured = signature(Path(sys.argv[1]))
    replay = signature(Path(sys.argv[2]))
    same = captured["sha256"] == replay["sha256"]
    print(json.dumps({"captured": captured, "replay": replay, "same_hash": same}, indent=2))
    return 0 if replay["failed"] else 1

if __name__ == "__main__":
    raise SystemExit(main())

Where a free runner fits, and where it does not

I do not want the draft step and the blank machine to depend on the laptop I am already trying to distrust. MonkeyCode's free model access and free server option can cover that split when you still review every generated diff yourself. Disclosure: This article was prepared as part of MonkeyCode's product outreach. I am not listing models, quotas, hardware, or duration, because those details were not verified here and can change without notice.

Use the table before you move a replay, and do not treat a free option as a promise that the option will remain. If you want to try the split without spending the afternoon on billing setup, check the current docs first and then decide. A remote pass still does not review the code, and it still does not make an unread diff safe to merge. Throw the scratch environment away when the note is written, so the next failure starts from empty rather than from leftovers.

Question	Stay on the laptop	Move the replay
Can a fresh venv still show the failure?	Yes, start there and keep the log	No, the host state is the suspect
Does the fixture contain secrets or customer data?	Review offline, or replace it with fake data	Do not upload it to any free runner
Can you read the whole diff in one sitting?	Require that before any remote run	A remote pass does not review the code for you
Do you need an audit trail or a support contract?	Use whatever your team already operates	A free option is not a production promise

Who should skip this

Skip the remote half if your tests need production credentials, private keys, or data you cannot show a third party. Skip the text gate if the failure is visual, flaky, or timing-based, because a marker search will lie with confidence. Skip the whole approach if you wanted a ranking of tools, since I measured nothing and I am not scoring products. A free server can disappear, change size, or reject a workload, so do not let a release process depend on it remaining.

• Do not send secrets, tokens, or customer fixtures to a free runner that you do not administer yourself.
• Do not treat a text marker as proof when the bug is visual, flaky, or quietly dependent on the clock.
• Do not publish a quota, a model name, or a machine size that you have not confirmed in the current docs.

What I would tell my past self

Would I still ask an assistant for the first draft after the failure is frozen and the scratch host has failed the same way? Yes, I would, provided the prompt includes the captured log and the lockfile hash rather than a vague request to clean things up. Would I let that draft touch the working branch first, even when the edit is three lines and the explanation sounds calm? No, I would not, because calm wording has never been evidence that a second environment will agree with the first one.

The order I would repeat is short, and I keep it written down so a tidy diff cannot talk me out of it.

1. Freeze the failing log and the lockfile hash before you allow anyone to suggest a change at all.
2. Replay that same command on a host created from the lockfile, with no copied packages and no shell history.
3. Ask for a draft only after that replay is still red, and include the captured log instead of a vague prompt.
4. Read the diff yourself, rerun it on the scratch host, and discard that host as soon as the note is written.

The repeatable part is the order of operations: capture the failure, replay it cleanly, then patch, then read the diff yourself. The part I would drop is any sentence that treats a free runner as proof, rather than as a clean room I can discard. Order is the whole method, and the assistant stays a draft until a stranger host has told the same story twice. If that sounds slow, compare it with the hour you will spend explaining a green laptop run that nobody else can reproduce.