FAQ: Can You Replay That Free Model Patch Tomorrow?

Did last night's free model patch actually replay today?
I keep hearing that story after late sessions.

People assume one green run means a frozen toolchain.
Do you still treat the model like gcc?

This FAQ is a replay checklist I actually use.
I want a diff I can apply tomorrow morning.
I do not want last night's confidence in Slack.

Why this FAQ exists

Free model access made cheap drafts feel endless.
A free server made those drafts feel like laptops.

Did either fact pin your inputs for you?
It did not pin mine for long.

I started treating every run as an experiment.
Experiments need notebooks you can hash later.
A chat thread will not serve that job.

What replay means here

Replay means a second tree rebuilds the same decision.
You need HEAD, a prompt file, and one test command.

You also need a diff that applies on clean HEAD.
If any piece is missing, replay is missing.
You only have a memory with syntax highlighting.

Where a free box fits

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

I sometimes park this harness on MonkeyCode.
It offers free model access and a free server option.

I still treat that box as untrusted scratch space.
The harness is the lesson, not the vendor.

Myth 1: The free model is a frozen compiler

People say the same prompt yields the same patch.
They cite the same repo, files, and temperature story.

Have you hashed the prompt file yet, honestly?
Have you pinned the test command inside a script?

A compiler prints a version string you can quote.
A free model path usually will not.

Corrected mental model

Treat the model as a generator, not a compiler.
Then treat git as the only freeze you own.

Evidence you can collect tonight

Use a throwaway clone for this check.
Skip production secrets while you do it.

git rev-parse HEAD
git status --porcelain
sha256sum prompts/fix-auth.md

If any value moved, the same run already died.
You did not change the English prompt text.
The inputs changed under your feet anyway.

Myth 2: The free server is a cleanroom

Is that box empty when you first log in?
Are leftover node_modules sitting from another job?

Is origin pointing at the repo you meant?
Are leftover branches still checked out from yesterday?

A free server is a rented desk, nothing more.
Yesterday's files can still be in the drawers.

Corrected mental model

Snapshot the environment before the model runs.
Then snapshot it again after the patch lands.

Tiny env snapshot

#!/usr/bin/env bash
set -euo pipefail
out="${1:-/tmp/env-snap.txt}"
{
  echo "date=$(date -u +%FT%TZ)"
  echo "pwd=$(pwd)"
  echo "head=$(git rev-parse HEAD)"
  echo "branch=$(git branch --show-current)"
  echo "user=$(id -un)"
  echo "python=$(command -v python3 || true)"
  echo "node=$(command -v node || true)"
  echo "origin=$(git remote get-url origin 2>/dev/null || true)"
  git status --porcelain
} > "$out"
sha256sum "$out"

Label this block as a proposed script only.
I am not claiming a benchmark for this snapshot.
It only answers what machine you actually touched.

Myth 3: Green tests pin the model

Tests passed, so this model is the one.
Which tests ran against which seed data though?

Did you run the same command your CI file runs?
A single green run is one sample.
It is not a pin, and it is not a release.

Corrected mental model

Pin the test command, and never trust vibes.
Record the stdout hash and the exit code.

Test plan I keep on a sticky note

1. Write the prompt into prompts/current.md before anything else.
2. Hash that file before the model sees it.
3. Run one explicit test command from a script.
4. Save the git diff into repro/patch.diff immediately.
5. Hash the diff and store both hashes in repro/manifest.json.
6. Reset the tree, apply the diff, and run tests again.
7. Fail the checklist if step six disagrees with step four.

That last step catches dirty-workspace luck cleanly.
The workspace is not your real merge base.

Myth 4: The prompt can live only in the thread

Why copy the prompt into git at all?
Because search in a thread is not an artifact.

Because what I meant is not the bytes.
Because tomorrow you will rewrite one sentence.

Then you will swear the model drifted overnight.
Maybe the model truly drifted overnight this time.
Maybe you edited the input without noticing.

Corrected mental model

The prompt file is an input artifact.
No prompt file means no honest replay later.

Prompt header I paste first

# prompt-id: fix-auth-2026-09-11
# repo-head: REPLACE_WITH_SHA
# test-command: python -m pytest tests/test_auth.py -q
# out-of-scope: do not edit CI or deploy files
# stop: do not commit

Task: add a failing test, then the smallest fix.
Do not refactor neighbors. Do not touch deploy files.

Notice the stop line in that header.
I do not want a free box committing for me.

Myth 5: Shipping the diff reproduced the run

The PR looks small and the tests look green.
So you think the original run was reproduced?

You reproduced a tree on a good day, maybe.
You did not reproduce the generator behind it.

That is fine for many ordinary patches.
It is not fine if you claim scientific replay.

Corrected mental model

Ship the diff plus the manifest every time.
Claim the patch applies on a clean tree.
Do not claim you pinned the model.

Decision table

Signal	Merge locally?	Re-run on laptop?	Reject?
Prompt file missing	No	Yes	If it looks secret
Manifest hash mismatch	No	Yes	If tests also drift
Diff touches CI or deploy	No	Always	Until reviewed
Tests only ran once on the box	No	Yes	If the suite is flaky
Diff plus tests replay on HEAD	Yes	Optional	No

Print that table next to the pull request.
Then ask the author which row they actually used.

The artifact: a replay harness

Here is a proposed script for local use.
It does not call any vendor API directly.
It only records things you already have locally.

#!/usr/bin/env bash
# repro-run.sh — proposed local harness, not a vendor wrapper
set -euo pipefail

prompt="${1:?prompt file required}"
test_cmd="${2:?test command required}"
mkdir -p repro

head_sha="$(git rev-parse HEAD)"
prompt_sha="$(sha256sum "$prompt" | awk '{print $1}')"

git status --porcelain > repro/status-before.txt

echo "Run the model against $prompt now."
echo "Press enter after the files change."
read -r _

git diff > repro/patch.diff
diff_sha="$(sha256sum repro/patch.diff | awk '{print $1}')"

set +e
bash -lc "$test_cmd" | tee repro/test-out.txt
test_ec=$?
set -e

cat > repro/manifest.json <<EOF
{
  "date": "$(date -u +%FT%TZ)",
  "head": "$head_sha",
  "prompt": "$prompt",
  "prompt_sha256": "$prompt_sha",
  "diff_sha256": "$diff_sha",
  "test_cmd": "$test_cmd",
  "test_exit": $test_ec
}
EOF

echo "Wrote repro/manifest.json"
cat repro/manifest.json

Then run the only check that actually matters.

git stash push -u -m "agent-scratch"
git checkout -- .
git apply repro/patch.diff
bash -lc "python -m pytest tests/test_auth.py -q"

If apply fails, you never had a replay.
You had a dirty tree and a good story.

Optional Python checker

Here is a proposed checker for the manifest.
Label this checker as unexecuted sample code.
Wire it into CI only after you read it.

# proposed: validate_manifest.py
import hashlib
import json
import pathlib
import sys

manifest = json.loads(pathlib.Path("repro/manifest.json").read_text())
diff = pathlib.Path("repro/patch.diff").read_bytes()
got = hashlib.sha256(diff).hexdigest()
if got != manifest["diff_sha256"]:
    sys.exit("diff hash mismatch")
print("manifest matches patch.diff")

What this does not prove

This harness does not pin a model identity.
It does not prove the free server was clean.

It does not replace a human code review.
It does not make a flaky test suddenly honest.

I am not publishing latency numbers in this FAQ.
I do not have a fair benchmark for the harness.

Who should not use this approach

Skip this if you need bit-identical model output.
Skip this if the repo holds production secrets.

Skip this if policy forbids untrusted hosted shells.
Skip this if you cannot run tests twice locally.

Also skip it for a one-line typo fix.
You do not need a manifesto for typos.

Limitations I keep repeating out loud

These limits are the whole point of the FAQ.

• Free model access is not a version pin.
• A free server is not your laptop clone.
• A green sample is not a release gate.
• A thread is not the prompt file.
• git apply is replay, not the narrative.

If any bullet surprises you, start with that bullet.

A corrected mental model

Think in three layers before you merge.

Inputs: prompt file, HEAD, and test command.
Generator: some model sitting on some box.
Outputs: diff, test log, and manifest hashes.

You own layer one and layer three.
You do not own layer two on a free path.

So stop arguing with layer two in standups.
Measure the outputs and keep the inputs boring.

What I do before I merge

1. The prompt file exists in git or the PR.
2. Manifest hashes match the attached diff bytes.
3. git apply works on a clean HEAD checkout.
4. The recorded test command exits zero twice.
5. No CI, secrets, or surprise deploy file edits.

Those five checks carry no extra drama.
Would you really merge this without check three tonight?
I would not, and I have been tempted.

Closing

Can you replay that free model patch tomorrow?
If the answer is I think so, you cannot.

Save the prompt, hash the diff, and apply it cold.
That is the whole FAQ, on purpose.

Paste the decision table on the next agent PR.