Notes From 48 Hours: I Scored the Second Patch Against a Tree That Left

I opened these notes because the second patch looked careless, and I wanted a culprit before I wanted a receipt. The first suggestion had left a small Python change that at least imported, while the later suggestion failed before the test runner started. Was I grading a wandering draft, or was I grading a checkout that had quietly become a different checkout? I had two drafts, a scratch server session, and no receipt that the first tree still existed.

What I tried before I had a stamp

I treated MonkeyCode's free model access as a second pair of eyes, and I kept the change deliberately small. Disclosure: This article was prepared as part of MonkeyCode's product outreach. I used that free model access to draft two fixes for one failing import, then applied each fix on the free server option. I never recorded which interpreter answered the version command, and I never recorded whether the lockfile was still the same lockfile.

The plan was simple enough that I skipped the boring part, which is exactly how these notes usually get expensive. I would paste the error, take the first patch, apply it, and run the smallest test that touched the broken import. If that looked fine, I would ask for a second patch and compare the two diffs by eye, as if eyesight were a measurement. Why did I trust an eyeball comparison when the disk between those two asks had never been named?

Here is the scratch sequence I would put in front of either patch, and the Python block is the artifact I would actually keep.

python3 -m venv .venv
. .venv/bin/activate
python -m pip install -r requirements.txt
python - <<'PY'
from pathlib import Path
import hashlib, json, platform, datetime
root = Path('.').resolve()

def sha(path):
    digest = hashlib.sha256()
    digest.update(Path(path).read_bytes())
    return digest.hexdigest()

lock = next((name for name in ('requirements.txt', 'poetry.lock', 'uv.lock') if Path(name).exists()), None)
stamp = {
    'utc': datetime.datetime.now(datetime.timezone.utc).isoformat(),
    'cwd': str(root),
    'python': platform.python_version(),
    'lock': lock,
    'lock_sha256': sha(lock) if lock else None,
    'file_count': sum(1 for p in root.rglob('*') if p.is_file() and '.venv' not in p.parts),
}
print(json.dumps(stamp, indent=2))
Path('.worktree-stamp.json').write_text(json.dumps(stamp, indent=2) + '\n')
PY
cp .worktree-stamp.json "$HOME/stamp-from-this-session.json"

That block is a reproducible artifact you can run, not a benchmark, and it names no model and no machine size. I would not trust either patch until I had read the printed JSON and copied it off the server. If a later session cannot see that stamp file, the two answers are no longer comparable, and the scoring should stop. A chat scrollback can still show the old error while the disk underneath it has already been replaced.

What broke when I came back

The break I am writing down is the failure this procedure is built to catch, not a timed outage I measured. I came back to a familiar prompt, typed the same test command, and expected the first tree to answer. A missing stamp, a missing virtualenv, and a clean checkout can all hide behind that familiar prompt. How do you score a second answer when you cannot prove the first workspace is still the one on disk?

I was ready to write that the second suggestion was worse, which would have been a story about taste. Three quiet mismatches can decide that grade before the model output says anything useful at all. Would that harsh sentence have survived if the first tree had still been there for a rerun? I do not think so, because I never separated the suggestion from the box that executed it.

• The session can treat the process home as disposable, so a reconnect may be a new disk even when the chat still shows the old traceback.
• A diff compared without a lockfile digest can hide a transitive install that changed behavior under an identical patch.
• Letting the shell pick python versus python3 can select a different program on more images than I like to admit.

None of those mismatches announce themselves inside the suggestion, which is why blaming the draft felt so easy. The prose of the second answer can be tidy while the tree under it is a stranger. I would rather lose an hour to a restamp than publish a verdict I cannot replay.

A decision table I would keep beside the stamp

I do not need a leaderboard to decide whether two answers may be compared, and I do not need invented latency numbers. The table is the rule I would repeat, and it stays qualitative so I cannot smuggle in a result I never ran. A pass means the second patch is eligible for review, not that the second patch is correct. A fail means I restamp and I stop the comparison, even if the chat still looks continuous.

Check	Pass means	If it fails
Stamp file still present	Same workspace identity as the first run	Stop, restamp, do not score the second patch
Python version string matches	Interpreter did not drift between asks	Rebuild the venv, then restamp
Lock digest matches	Dependencies were not silently refreshed	Reinstall from the recorded file, then restamp
git apply --check succeeds	The diff still matches this tree	Reject the diff and ask again against the stamped tree
Smallest test command is unchanged	You are not grading a different question	Write the command into the stamp notes first

git apply --check first-suggestion.diff
git apply --check second-suggestion.diff
python -m pytest tests/test_import_guard.py -q

If either apply check fails, I would not ask the model to try harder on a tree I have not named. A harder answer on a different tree is still an unfair comparison, and it will waste the next hour. I would paste the stamp JSON into the next prompt so the draft is tied to an interpreter and a lock digest. Is a slower loop annoying, and is that annoyance actually the price of a fair comparison?

Comparing the copy I kept with the box I just opened

The shell stamp is only half of the habit, because a file that lives only on the server can vanish with the server. I keep an external copy, then I refuse the comparison when the live stamp disagrees with that copy. The script below is an unexecuted example of that refusal, and it raises instead of printing a polite maybe.

import json
from pathlib import Path

def load(path):
    return json.loads(Path(path).read_text())

def mismatches(before, after):
    keys = ('python', 'lock', 'lock_sha256', 'cwd')
    return [key for key in keys if before.get(key) != after.get(key)]

if __name__ == '__main__':
    before = load('stamp-from-my-laptop.json')
    after = load('.worktree-stamp.json')
    bad = mismatches(before, after)
    if bad:
        raise SystemExit('not comparable: ' + ', '.join(bad))
    print('stamps match on interpreter, lock, and cwd')

I would run that check before I open the second diff, not after I have already decided the second diff is clumsy. A raised exit is the result I want when the tree moved, because a green test on a stranger tree is not evidence. Would I still call the model wrong if this script exits on lock_sha256 alone? No, I would call the comparison invalid and start again from the recorded lockfile.

What I would repeat on the next pass

I would repeat the stamp before the first prompt, not after the argument has already picked a villain. A chat transcript is not a worktree, and a worktree that might vanish is not a lab notebook. Free model access is useful as a drafter of small diffs, and the free server option is useful as a scratch bench. Both stay unpinned until the stamp says the interpreter, the lock digest, and the file count still match.

The loop I would repeat is short, and I would not add steps just to look thorough.

1. Stamp the tree and copy .worktree-stamp.json to a machine I control before I ask for any patch.
2. Ask for one small patch, apply it only after git apply --check, and run one named test whose command is already written down.
3. Reconnect or open a fresh server session only after that external copy of the stamp exists.
4. Restamp on the new session and refuse the comparison when any recorded digest differs from the copy.
5. Only then ask for a second patch, with the stamp JSON included so the draft is bound to this tree.

I would also repeat a refusal that felt fussy in the first hour and obvious on the second day. I would not accept a suggestion that edits files outside the stamped root, even if the prose sounds confident. I would not accept a suggestion that tells me to fetch an installer as part of the so-called fix. Who benefits if the patch and the environment both change before I have a second measurement?

Commands I would not skip just because the chat looks calm

A calm prompt is not a stable machine, so I would rather see the boring commands in the notes than reconstruct them from memory. These four are the ones I would paste back when a future me insists the tree could not have moved. None of them measure model quality, and that is intentional.

python -c 'import platform; print(platform.python_version())'
sha256sum requirements.txt 2>/dev/null || shasum -a 256 requirements.txt
test -f .worktree-stamp.json && echo 'stamp present' || echo 'stamp missing'
find . -path './.venv' -prune -o -type f -print | wc -l

If the stamp line says missing, I stop, even when the test failure looks exactly like the failure from last night. Same traceback text is not the same filesystem, and I have confused those two things often enough. Do I really want another suggestion before I can answer where the first files went?

Limitations, and who should skip this

This workflow does not prove a model is correct, and it does not prove a free server is safe for private code. I am not stating a quota, a hardware shape, a retention window, or a promise that the free options remain free. Those details were not supplied for this note, so inventing them would make the field notes dishonest. If the tree holds credentials, customer data, or an unreleased fix, do not park it on a shared free server.

Skip the approach when the change is already in production and you need a signed build plus a named runtime owner. Skip it when you cannot read the diff yourself, because a stamp only shows stability, not wisdom. Skip it when the task needs a specific model by name and that name has not been verified for your account. Guessing a model identity to make the notes sound precise would be the same mistake as guessing the tree.

The stamp also will not catch a bad idea that happens to apply cleanly on a stable tree, and I do not want it to pretend otherwise. File count can match while a single important file was rewritten, so I still read the diff and I still rerun the named test. If your review standard is a signed release, a second reviewer, or a locked production image, this scratch loop is the wrong tool. It is a way to avoid a false argument, not a way to ship.

If you want a scratch place to try the loop, MonkeyCode is the free model access and free server option these notes assume. Read the current plan yourself before you trust any session with a repository you cannot throw away. I would start from a disposable checkout, keep the stamp on a machine I control, and treat a missing file as a failed comparison. The suggestion can wait, because a second answer is only interesting after the first tree is still real.