Freeze-and-Thaw for Model-Drafted Docs: A Determinism Gate Before Human Ownership

Conclusion first: a section generated by a model should not become human-editable until two independent runs of the same prompt produce identical bytes and the accepted bytes are hashed into a lock file. After that freeze, every later edit to that section must arrive with an explicit Thawed-Section: trailer that names the section and transfers ownership to a person. The gate below enforces both halves with standard-library Python and no service dependency.

Why determinism is the missing half of doc review

Most documentation arguments are about who may change a page, not about whether the page can be reproduced at all. If the same prompt and the same source anchors yield different prose on Tuesday than on Monday, review comments describe a snapshot nobody can regenerate, and the diff becomes the only surviving record of what the model intended.

Determinism is also unusually cheap to test, because you do not need ground truth to test it. You need two runs, a section marker convention, and a hash function, and the comparison is exact rather than statistical.

The division of labour: three section classes

draft_class	Who writes it	What the gate checks
machine	Free model, free-form prose	Non-empty anchors list, plus a thaw trailer before any human edit
collate	Code assembles it from anchors (tables, command blocks, config rows)	Identical bytes across two runs, plus a thaw trailer before edits
owned	A named human, always	Must exist in every file; the model has no write path here

The contract invariant is blunt on purpose. Every file needs at least one owned section, and machine prose without declared anchors fails the contract check immediately. Anchors are file paths, captured command output, or an issue link, meaning anything a reviewer can actually open.

Step 1: mark section boundaries in the file itself

Sections live behind an HTML comment marker on its own line, directly above the heading they describe. The marker is invisible in rendered output but stable in git diff, which is the property the gate depends on.

<!-- section: error-table -->
## Error codes

| code | meaning |
| --- | --- |
| `E_AUTH` | token rejected before routing |

<!-- section: retry-guide -->
## Retrying safely

Step 2: record who may write where

The contract is a plain JSON file that maps each file and section id to a draft class, plus anchors or an owner. It answers a design-time question: may the model write here at all.

{
  "docs/api/errors.md": {
    "error-table": { "draft_class": "collate", "anchors": ["src/errors.ts"] },
    "rationale":   { "draft_class": "owned",   "owner": "@averylin" },
    "retry-guide": { "draft_class": "machine", "anchors": ["ci/logs/retry-429.txt"] }
  }
}

Step 3: draft twice, then compare

The drafting job must run twice as two independent processes, ideally on a host that is not your laptop. In this pipeline I ran both passes on MonkeyCode's free server option, using the free model access for the prose sections. Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The invocation below is a placeholder, because the correct command shape depends on your own workspace; what matters is that both runs are independent and write to separate directories.

# Replace $DRAFT_CMD with your own invocation.
$DRAFT_CMD --out .draft/a
$DRAFT_CMD --out .draft/b
python scripts/docgate.py unstable .draft/a .draft/b

unstable walks both trees, parses sections by marker, and reports any section whose body hash differs between the two runs. A section that never reproduces is not reviewable, so it must be rewritten as collate output or handed to a human as owned.

# scripts/docgate.py — part 1 of 4: parsing and the two-run comparison
#!/usr/bin/env python3
"""docgate: determinism and thaw gate for model-drafted documentation."""
from __future__ import annotations

import hashlib, json, re, subprocess, sys
from pathlib import Path

LOCK = Path("docs.lock.json")
CONTRACT = Path("docs.contract.json")
MARKER = re.compile(r"^<!--\s*section:\s*(?P<id>[a-z0-9._-]+)\s*-->$")
HUNK = re.compile(r"^@@ -\d+(?:,\d+)? \+(?P<start>\d+)(?:,(?P<count>\d+))? @@")

def git(*args: str) -> str:
    return subprocess.run(["git", *args], check=True, text=True,
                          capture_output=True).stdout

def sections(text: str) -> list[dict]:
    lines = text.splitlines()
    marks = [(i, MARKER.match(l.strip()).group("id"))
             for i, l in enumerate(lines, 1) if MARKER.match(l.strip())]
    out = []
    for n, (start, sid) in enumerate(marks):
        end = marks[n + 1][0] - 1 if n + 1 < len(marks) else len(lines)
        body = "\n".join(lines[start:end])
        out.append({"id": sid, "start": start + 1, "end": end,
                    "sha256": hashlib.sha256(body.encode()).hexdigest()})
    return out

def unstable(a: Path, b: Path) -> list[str]:
    flagged = []
    for draft in sorted(a.rglob("*.md")):
        twin = b / draft.relative_to(a)
        if not twin.exists():
            flagged.append(f"{draft.relative_to(a)}: missing in second run")
            continue
        first = {s["id"]: s["sha256"] for s in sections(draft.read_text())}
        second = {s["id"]: s["sha256"] for s in sections(twin.read_text())}
        if first.keys() != second.keys():
            flagged.append(f"{draft.relative_to(a)}: section set differs")
        for sid in first.keys() & second.keys():
            if first[sid] != second[sid]:
                flagged.append(f"{draft.relative_to(a)}#{sid}: body differs")
    return flagged

def report_unstable(a: Path, b: Path) -> int:
    flagged = unstable(a, b)
    print("\n".join(flagged) if flagged else "all sections reproduce byte-for-byte")
    return 1 if flagged else 0

Step 4: freeze the accepted draft

Freezing converts the reviewed draft into a lock file that stores, per file, each section id with its line span and body hash. The lock answers a second question that the contract cannot: what exactly did we accept, at which lines.

# scripts/docgate.py — part 2 of 4: freeze the accepted bytes

def freeze(draft: Path, lock: Path = LOCK) -> None:
    data = {p.relative_to(draft).as_posix(): sections(p.read_text())
            for p in sorted(draft.rglob("*.md"))}
    lock.write_text(json.dumps(data, indent=2, sort_keys=True) + "\n")
    print(f"froze {len(data)} files into {lock}")

python scripts/docgate.py freeze .draft/a
cp -r .draft/a/docs docs/          # only after human review of the accepted run
git add docs docs.lock.json

Step 5: thaw is a commit trailer, not a conversation

A human edit to a drafted section is legitimate, but it must be explicit and greppable. The trailer below is the entire ownership transfer mechanism, and it is deliberately boring so that git log stays the audit trail.

git commit -m "docs(errors): fix retry wording

Thawed-Section: retry-guide"

Step 6: gate the pull request

The gate maps every changed line in a markdown file back to the frozen line spans, then checks whether each hit section was granted a thaw in the same branch. Hunks come from git diff -U0 so that only genuinely touched lines are attributed to a section.

# scripts/docgate.py — part 3 of 4: diff-to-section attribution and the gate

def changed_lines(base: str, path: str) -> set[int]:
    touched: set[int] = set()
    for line in git("diff", "-U0", "--no-color", base, "--", path).splitlines():
        m = HUNK.match(line)
        if not m:
            continue
        start, count = int(m.group("start")), int(m.group("count") or 1)
        touched.update(range(start, start + count) if count else {max(start, 1)})
    return touched

def thawed(base: str) -> set[str]:
    log = git("log", f"{base}..HEAD", "--format=%B")
    return set(re.findall(r"^Thawed-Section:\s*([a-z0-9._-]+)\s*$", log, re.M))

def gate(base: str, lock: Path = LOCK) -> int:
    granted, failures = thawed(base), []
    for rel, spans in json.loads(lock.read_text()).items():
        touched = changed_lines(base, rel)
        for s in spans:
            hit = sorted(ln for ln in touched if s["start"] <= ln <= s["end"])
            if hit and s["id"] not in granted:
                failures.append(f"{rel}#{s['id']} (lines {hit[0]}-{hit[-1]})")
    if not failures:
        print("gate ok: every edited draft section carries a thaw trailer")
        return 0
    print("un-thawed edits to model-drafted sections:")
    for f in failures:
        print(f"  - {f}")
    print("Add 'Thawed-Section: <id>' to a commit, or revert the hunk.")
    return 1

The contract check is the design-time complement, and it fails closed on undeclared or stale section ids.

# scripts/docgate.py — part 4 of 4: contract invariants and entry point

def contract(path: Path = CONTRACT) -> int:
    rules, problems = json.loads(path.read_text()), []
    for md in sorted(Path("docs").rglob("*.md")):
        rel, ids = md.as_posix(), {s["id"] for s in sections(md.read_text())}
        declared = rules.get(rel)
        if not declared:
            problems.append(f"{rel}: no contract entry")
            continue
        if ids - declared.keys():
            problems.append(f"{rel}: undeclared {sorted(ids - declared.keys())}")
        if declared.keys() - ids:
            problems.append(f"{rel}: stale {sorted(declared.keys() - ids)}")
        if not any(r["draft_class"] == "owned" for r in declared.values()):
            problems.append(f"{rel}: no human-owned section")
        for sid, r in declared.items():
            if r["draft_class"] == "machine" and not r.get("anchors"):
                problems.append(f"{rel}#{sid}: machine prose without anchors")
    print("\n".join(problems) if problems else "contract ok")
    return 1 if problems else 0

if __name__ == "__main__":
    cmd, rest = sys.argv[1], sys.argv[2:]
    if cmd == "unstable":
        sys.exit(report_unstable(Path(rest[0]), Path(rest[1])))
    if cmd == "freeze":
        freeze(Path(rest[0]))
        sys.exit(0)
    if cmd == "gate":
        sys.exit(gate(rest[0]))
    if cmd == "contract":
        sys.exit(contract())
    sys.exit(f"unknown command: {cmd}")

Planted-failure test plan

Run these in a scratch clone against a small fixture file with two marked sections. The expected exit codes describe the implementation above, not a hosted service.

#	Planted change	Expected result	Why
1	Reword a machine section, no trailer	exit 1, section named	Line spans map into a frozen section
2	Same edit plus Thawed-Section: trailer	exit 0	The grant set contains the section id
3	Change only an owned section	exit 0 from the gate	owned sections are not in the lock's drafted spans
4	Add a new unmarked section	exit 1 from contract	Undeclared ids fail closed
5	Rerun the same prompt twice with a temperature change	exit 1 from unstable	Body hashes diverge across runs

What the free tier changes, and what it does not

The double-draft pass doubles generation work, so a free model and a free server turn an otherwise wasteful step into a routine one. That is the only place the platform participates here, and the gate itself never calls out to a network. If you want the cheapest path into this workflow, the free model access and the free server option are the two things worth starting with, because everything else is git and Python.

Limitations and who should not use this

• Line spans are frozen, so any commit that reflows a file must rerun freeze in the same commit or later spans become wrong.
• The marker line itself is outside the span, so editing a marker is not gated by gate. The contract check catches renames only after a freeze pass.
• A pure deletion produces a -U0 hunk with count zero, which the code anchors to the line above; that can misattribute a deletion by one line, so treat deletions in drafted regions as thaw-worthy regardless.
• The gate cannot distinguish a bot commit from a human commit. If automation commits to docs/, add an author allowlist check beside it.
• Not for teams without a stable spec source, because anchors will point at moving targets and every anchor change invalidates the freeze.
• Not for single-author repositories where freeze-and-thaw overhead exceeds the cost of just writing the section yourself.
• Free tiers change without notice. This pipeline depends only on two independent runs and a git repository, so it keeps working if free access disappears.

Gate the determinism, then gate the ownership, and the model's drafts stop being unreproducible personal artifacts. The two commits that matter are the freeze and the thaw, and both of them stay in git log where a reviewer can read them.