The Model Replied. You Still Have No Freeze File.

A green chat is not a green build.
The missing freeze file is the real outage.
Free remote inference does not pin a compiler.

The public feed keeps arguing about model skill.
Did AI already beat most working developers?
That question skips the only check I run.

Did you pin the run at all?
Or did you screenshot a lucky chat?
I keep a blunt catalog for this failure.

Each entry lists symptoms and a root cause.
Each one also names a replacement pattern.
I am not ranking vendors in this post.

I am not posting latency or quality charts.
I am talking about unpinned AI coding runs.
Folklore patches still look brilliant in a thread.

What I mean by a freeze file

A freeze file is a local contract.
It records what the model actually saw.
It also records the patch you accepted.

The proposed schema sits below.
Treat it as an example until you execute it.
Do not paste it into production CI untested.

{
  "prompt_sha256": "replace-me",
  "patch_sha256": "replace-me",
  "test_command": "pytest -q tests/test_billing.py",
  "created_at": "2026-09-16T12:00:00Z",
  "runtime_kind": "scratch-pad",
  "reproduced_locally": false,
  "secrets_present": false
}

I only trust prompt, patch, and test hashes.
Drop one hash and the run becomes folklore.
Git then stores a story, not a replay.

I gate merges on that file.
No freeze file, no merge.
The validator is boring on purpose.

#!/usr/bin/env python3
"""Refuse an AI coding run with no freeze file.

Local gate only. It never calls a model.
"""
from __future__ import annotations

import json
import sys
from pathlib import Path

REQUIRED = (
    "prompt_sha256",
    "patch_sha256",
    "test_command",
    "created_at",
    "runtime_kind",
)
ALLOWED_RUNTIME = {"local", "scratch-pad", "ci-pinned"}


def load(path: Path) -> dict:
    data = json.loads(path.read_text(encoding="utf-8"))
    if not isinstance(data, dict):
        raise ValueError("freeze file must be an object")
    return data


def validate(data: dict) -> list[str]:
    errors: list[str] = []
    for key in REQUIRED:
        if not data.get(key):
            errors.append(f"missing {key}")
    runtime = data.get("runtime_kind")
    if runtime not in ALLOWED_RUNTIME:
        errors.append(f"runtime_kind {runtime!r} is not pinable")
    if runtime == "scratch-pad" and not data.get("reproduced_locally"):
        errors.append("scratch-pad run was never reproduced locally")
    if data.get("secrets_present") is True:
        errors.append("freeze file claims secrets in the prompt")
    return errors


def main() -> int:
    if len(sys.argv) != 2:
        print("usage: validate_freeze.py path/to/freeze.json", file=sys.stderr)
        return 2
    path = Path(sys.argv[1])
    if not path.is_file():
        print(f"no freeze file at {path}", file=sys.stderr)
        return 1
    errors = validate(load(path))
    if errors:
        print("unpinned run:")
        for item in errors:
            print(f"- {item}")
        return 1
    print("freeze file ok")
    return 0


if __name__ == "__main__":
    raise SystemExit(main())

Run the local gate like this.

sha256sum prompt.txt patch.diff
python3 validate_freeze.py freeze.json
pytest -q tests/test_billing.py

Want a shorter rule of thumb?
If validate_freeze.py cannot run, you vibe-coded.
The chat UI will not argue with that.

Anti-pattern 1: Unpinned inference

You asked the box. It answered.
You shipped the vibe anyway.
You never named the runtime kind.

Symptoms

• Tomorrow's answer flips on the same prompt.
• Nobody can name the runtime kind.
• "Works on my machine" now means "works in chat."

Root cause

You treated a remote completion like gcc -v.
Compilers publish versions you can pin.
Many chat boxes do not.

Replacement pattern

Record runtime_kind before you accept a patch.
Reproduce the patch with local tests.
Refuse to merge folklore from a thread.

jq -r .runtime_kind freeze.json
# expected: local | scratch-pad | ci-pinned

Did the model get smarter overnight?
Or did the unnamed runtime drift?
I assume drift until the freeze says otherwise.

Anti-pattern 2: Chat log as the audit trail

The thread looks complete and friendly.
It even has jokes in the margins.
Auditors cannot replay jokes later.

Symptoms

• The only spec lives in a vendor UI.
• Patch files have no prompt hash.
• New hires cannot reconstruct the change.

Root cause

You stored conversation, not evidence.
A chat log is not an artifact.
It is a diary with extra tokens.

Replacement pattern

Export three files, always, in one commit.
Keep prompt.txt, patch.diff, and freeze.json.
Then hash them before anyone reviews the patch.

git add prompt.txt patch.diff freeze.json
git commit -m "pin billing patch to freeze file"

Did the assistant summarize the diff for you?
I do not care about that summary.
I care whether patch_sha256 matches git diff.

python3 - <<'PY'
import hashlib, pathlib, json, sys
patch = pathlib.Path("patch.diff").read_bytes()
digest = hashlib.sha256(patch).hexdigest()
frozen = json.loads(pathlib.Path("freeze.json").read_text())
sys.exit(0 if frozen.get("patch_sha256") == digest else 1)
PY

Anti-pattern 3: Free server as CI

Exploration is cheap, and that is fine.
CI is a different machine with different trust.
Do not blur those two on purpose.

I still want a scratch pad for messy prompts.
Disclosure: This article was prepared as part of MonkeyCode's product outreach.
MonkeyCode offers free model access and a free server option.

I map that pair to runtime_kind: scratch-pad only.
Draft there if you need a pad.
Then copy the patch home and run pytest locally.

Symptoms

• Shared scratch boxes absorb flaky tests.
• Failures get blamed on "the model."
• Secrets leak into a communal prompt.

Root cause

You confused availability with isolation.
A free server is a shared whiteboard.
Whiteboards do not sign releases.

Replacement pattern

Keep the scratch pad in the draft lane.
Never point test_command at a shared box.
Set reproduced_locally only after local pytest.

{
  "runtime_kind": "scratch-pad",
  "reproduced_locally": true,
  "test_command": "pytest -q tests/test_billing.py"
}

If reproduced_locally is still false, stop.
You have a sketch, not a candidate patch.
The validator will say the same thing.

Anti-pattern 4: Tests written after the patch

The model "solved" the ticket first.
Then you asked it for tests.
The tests learned the first answer.

Symptoms

• Tests assert the first returned string.
• Edge cases never appear in the file.
• Refactors break tests that never encoded rules.

Root cause

You used tests as applause.
Applause is not a specification.
The model taught the oracle.

Replacement pattern

Write the failing test first.
Commit it on main before any prompt.
Only then paste the prompt that cites it.

# tests/test_billing.py — proposed example, not executed here
def test_prorate_ignores_trailing_partial_day():
    assert prorate("2026-09-01", "2026-09-16", 3100) == 1500

Run this loop without skipping step one.

1. pytest must fail on main.
2. Freeze the prompt that cites that test.
3. Accept a patch only if the same test passes.
4. Re-run python3 validate_freeze.py freeze.json.

If step one was already green, you cheated.
The model did not invent your spec.
You let it grade its own homework.

Decision table I actually use

What I saw	Story I tell myself	Anti-pattern	Replacement
Same prompt, new answer	The model got worse	Unpinned inference	Freeze runtime, replay locally
Long chat history	We have an audit	Chat log as audit	Commit prompt, patch, freeze
Tests on a shared box	CI is covered	Free server as CI	Local pytest gate
Tests after the patch	We are thorough	Post-hoc tests	Failing test first

Print that table next to the PR.
If a row matches, name the anti-pattern.
Do not argue about model IQ instead.

A concrete debugging workflow

I use this when a "smart" patch rots.
It is a proposed workflow, not a benchmark.
I have not executed it against your repo.

1. Print the freeze file. Missing file means stop.
2. Check secrets_present. True means rotate keys.
3. Confirm reproduced_locally is actually true.
4. Replay test_command on a dirty checkout.
5. If replay fails, blame the freeze, not the model.

test -f freeze.json || { echo "unpinned"; exit 1; }
python3 validate_freeze.py freeze.json
git stash push -u -m "temp"
bash -lc "$(jq -r .test_command freeze.json)"

Why stash first?
Because dirty trees hide lucky passes.
I want the freeze to survive a clean tree.

Limitations

A freeze file is not magic.
Hashing a prompt does not freeze a remote model.
The validator cannot see server-side drift.

scratch-pad is an honesty label.
It does not claim isolation or uptime.
It does not claim a service level.

Do not use this approach when you need these.

• Production traffic on the same box
• Secret-bearing prompts on a shared server
• A contractual SLA for inference
• A merge that skips local replay

Regulated workflows need a runtime you control.
This catalog will not give you that.
It will only stop you lying to git.

Solo explorers can skip the freeze for toys.
Throwaway katas do not need freeze.json.
Anything you might merge tomorrow does.

What I ship

I ship a patch, a test, and a freeze file.
I do not ship a chat screenshot.
A green chat is still an unpinned run.

If the freeze is missing, I failed.
The model did not fail me first.
Pin the run or admit you vibe-coded.