Cite-or-Freeze: Split AI Docs Into Cited Claims and Frozen Commands

The on-call page landed at 02:14, and the first link in the runbook still said to restart worker-b. The repo had renamed that process six days earlier. Generated Markdown still rendered. It simply pointed operators at a binary that was gone.

That failure is not a grammar problem. Models are fluent. Fluency hides drift. The dangerous sections are the ones people copy into a shell at 2 a.m., not the product-overview paragraph that nobody treats as a contract.

This article proposes a split pipeline. The model may draft narrative that cites a source symbol. A human must freeze recovery commands, warnings, and any sentence that cannot point at a file. A small linter enforces the split. If every product mention below disappeared, the gate would still be usable.

The failure mode is citation, not tone

Happy-path READMEs usually survive a regenerate. Troubleshooting does not. Error catalogs do not. Runbooks do not. Those pages mix two kinds of text:

• Derived claims — “UploadService retries with backoff.” This can be checked against code.
• Operational commitments — “Page #platform-oncall after two failed rollbacks.” This is a human policy. No symbol in src/ proves it.

A single “AI wrote the docs” pass flattens both. The first kind should be cheap to refresh. The second kind should be expensive to change.

Ownership split, in one table

Doc region	Model may draft?	Required artifact	Human must freeze?
Architecture overview	Yes, with cites	path#symbol on each factual sentence	No
Public endpoint list	Yes, from the spec	OpenAPI operationId or proto RPC name	No
Code sample that compiles	Draft only inside a fixture	Extracted file under docs/fixtures/	Human owns the fixture
Shell recovery steps	No	Fenced block marked freeze:command	Yes
Severity, paging, SLA language	No	Fenced block marked freeze:policy	Yes
Security warnings	No	freeze:warning	Yes
Changelog “why we did this”	Draft ok	Cite the PR or issue id	Human owns the “why”

The rule is narrow. If a sentence asserts a fact about the repo, it needs a cite. If a sentence tells a human to do something irreversible, it needs a freeze block. Everything else is optional color and should not be in the critical path.

The cite-or-freeze contract

Mark draftable sections with a lane comment. Mark frozen blocks with an info string on the fence. Keep both in ordinary Markdown so reviews stay readable.

<!-- lane: draftable -->
`InventoryWorker` reads from `jobs` and writes `inventory_events`.
<!-- cite: src/workers/inventory.py#InventoryWorker -->

<!-- lane: frozen -->

freeze:command
systemctl restart inventory-worker
journalctl -u inventory-worker -n 200

yaml

Three invariants follow:

1. Every factual sentence in a draftable lane has a following cite: comment.
2. A cite: target resolves to a path that exists, and the fragment exists in that file.
3. A freeze:* block is byte-identical to the last human-approved copy in docs/freeze/.

The model never writes into docs/freeze/. CI compares, it does not “improve.”

Artifact: a claim map plus a linter

The example below is a self-contained proposal. Run it on the fixtures in this article. It is not a production scanner and it does not parse every language.

1. Claim map

# docs/claim_map.yaml
version: 1
draftable_globs:
  - "docs/product/**/*.md"
  - "docs/api/overview.md"
frozen_globs:
  - "docs/runbooks/**/*.md"
  - "docs/security/**/*.md"
cite_required_lanes:
  - draftable
frozen_kinds:
  - command
  - policy
  - warning
index:
  roots:
    - src
    - proto
    - openapi.yaml
allow_missing_fragment_languages:
  - md

2. Sample source and a lying doc

# src/workers/inventory.py
class InventoryWorker:
    QUEUE = "jobs"
    OUT_STREAM = "inventory_events"
    MAX_ATTEMPTS = 5

<!-- file: docs/product/inventory.md -->
<!-- lane: draftable -->
`InventoryWorker` reads `jobs` and emits `inventory_events`.
<!-- cite: src/workers/inventory.py#InventoryWorker -->

Retries stop after three attempts.
<!-- cite: src/workers/inventory.py#MAX_ATTEMPTS -->

The second cite is the interesting case. The symbol exists. The sentence is still wrong: the code says 5, the prose says three. Citation is necessary, not sufficient. A later section treats that gap.

3. Linter (runnable example)

#!/usr/bin/env python3
"""claim_lint.py — cite-or-freeze gate. Proposal / example, not a full parser."""
from __future__ import annotations

import pathlib
import re
import sys

CITE = re.compile(r"<!--\s*cite:\s*([^#\s]+)(?:#([^\s]+))?\s*-->")
LANE = re.compile(r"<!--\s*lane:\s*(draftable|frozen)\s*-->")
FREEZE = re.compile(r"^```

freeze:(command|policy|warning)\s*$", re.M)
FACTISH = re.compile(
    r"`[^`]+`|\b(retries|timeout|port|queue|endpoint|restart)\b",
    re.I,
)

def iter_sentences(block: str) -> list[str]:
    parts = re.split(r"(?<=[.!?])\s+", block.strip())
    return [p for p in parts if p]

def file_has_fragment(path: pathlib.Path, fragment: str | None) -> bool:
    if not path.exists():
        return False
    text = path.read_text(encoding="utf-8")
    if not fragment:
        return True
    return fragment in text

def lint(md_path: pathlib.Path, repo: pathlib.Path) -> list[str]:
    text = md_path.read_text(encoding="utf-8")
    errors: list[str] = []
    lane = "unknown"
    for raw_line in text.splitlines():
        m = LANE.search(raw_line)
        if m:
            lane = m.group(1)

    if lane == "draftable":
        sentences = iter_sentences(re.sub(r"```.*?```", "", text, flags=re.S))
        cites = list(CITE.finditer(text))
        fact_rows = [s for s in sentences if FACTISH.search(s)]
        if len(cites) < len(fact_rows):
            errors.append(
                f"{md_path}: {len(fact_rows)} fact-like sentences, {len(cites)} cites"
            )
        for c in cites:
            rel, frag = c.group(1), c.group(2)
            target = repo / rel
            if not file_has_fragment(target, frag):
                errors.append(f"{md_path}: missing cite {rel}#{frag or ''}")

    for kind in FREEZE.findall(text):
        freeze_copy = repo / "docs" / "freeze" / f"{md_path.stem}.{kind}.txt"
        if not freeze_copy.exists():
            errors.append(f"{md_path}: no frozen snapshot for {kind}")
            continue
        fenced = re.search(
            rf"`{% endraw %}{% raw %}``freeze:{kind}\n(.*?)``{% endraw %}{% raw %}`", text, flags=re.S
        )
        if fenced and fenced.group(1) != freeze_copy.read_text(encoding="utf-8"):
            errors.append(f"{md_path}: freeze:{kind} drifted from docs/freeze/")

    if lane == "frozen" and CITE.search(text) is None and not FREEZE.search(text):
        errors.append(f"{md_path}: frozen lane has neither cite nor freeze block")
    return errors

def main() -> int:
    repo = pathlib.Path.cwd()
    docs = repo / "docs"
    failures: list[str] = []
    for md in docs.rglob("*.md"):
        failures.extend(lint(md, repo))
    for item in failures:
        print(item)
    return 1 if failures else 0

if __name__ == "__main__":
    sys.exit(main())
```

### 4. Commands

``{% endraw %}{% raw %}`bash
chmod +x claim_lint.py
python3 claim_lint.py
mkdir -p docs/freeze docs/product src/workers
# After a human edits a recovery block:
python3 - <<'PY'
from pathlib import Path
import re, sys
p = Path("docs/runbooks/inventory.md")
text = p.read_text()
for kind in ("command", "policy", "warning"):
    m = re.search(rf"`{% endraw %}{% raw %}``freeze:{kind}\n(.*?)``{% endraw %}{% raw %}`", text, re.S)
    if m:
        Path(f"docs/freeze/{p.stem}.{kind}.txt").write_text(m.group(1))
print("freeze snapshots written")
PY
```

A green run means every draftable fact has a pointer, and every freeze block matches the snapshot. A red run means regenerate is blocked. That is the point.

## What the model may draft

Keep the prompt boring. Feed it the claim map, the source files listed in `index.roots`, and the current Markdown with freeze blocks stripped. Ask for sentences, not for commands.

``{% endraw %}{% raw %}`text
Draft only <!-- lane: draftable --> sections.
Each factual sentence must be followed by <!-- cite: path#symbol -->.
Do not invent symbols. If the source does not contain the fact, omit the sentence.
Do not modify ```freeze:*``` blocks.
Do not write paging, rollback, or credential instructions.
```

The output is a patch against draftable lanes. Reviewers read the patch plus the linter diff, not a 2,000-word regenerate.

## What a human must own

Freeze these even when the model is confident:

- Any command that mutates state (`restart`, `kubectl delete`, `DROP`, `chmod`, token rotation).
- Severity labels and paging targets.
- Numbers that are policy, not code: customer-facing RTO, support hours, “we will notify.”
- Threat text. A wrong warning is worse than a missing adjective.
- Examples that need secrets, production hostnames, or customer data shapes.

A useful review question: “If this sentence is wrong, does someone run a command or trust a guarantee?” If yes, freeze it. If no, cite it.

## Numeric claims need a second gate

The `MAX_ATTEMPTS` example shows the remaining hole. The fragment exists, so `claim_lint.py` passes, while the prose still says “three.” Add a narrow check for integer literals next to known names.

``{% endraw %}{% raw %}`python
# proposal: numeric_cite.py — unexecuted extension, same fixtures
import ast, pathlib, re

ASSIGN = re.compile(r"^(\w+)\s*=\s*(\d+)", re.M)

def constants_in(path: pathlib.Path) -> dict[str, str]:
    text = path.read_text(encoding="utf-8")
    found = dict(ASSIGN.findall(text))
    return found

def prose_numbers(sentence: str) -> list[str]:
    words = {
        "one": "1", "two": "2", "three": "3", "four": "4",
        "five": "5", "six": "6", "seven": "7", "eight": "8",
        "nine": "9", "ten": "10",
    }
    out = re.findall(r"\b\d+\b", sentence)
    for w, n in words.items():
        if re.search(rf"\b{w}\b", sentence, re.I):
            out.append(n)
    return out
```

Wire it only for cites whose fragment is a constant. If the sentence contains a number and it is not in `{value, value-1}` of that constant, fail. Do not try to NLP the whole paragraph. The cheap check catches the common lie: “retries three times” next to `MAX_ATTEMPTS = 5`.

## Where a free drafting host fits

The generate step is bursty and disposable. The gate belongs in CI on the repo you already trust. Those two jobs should not share credentials.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. If you need a drafting loop without standing up a paid GPU box, MonkeyCode’s free model access and free server option can host the generate step that rewrites `draftable` lanes. Keep `docs/freeze/` and `claim_lint.py` on your own runner. The model never needs permission to overwrite recovery commands.

## Limitations

- **Citation is not truth.** A symbol can exist and the sentence can still invert its meaning. The numeric extra-gate covers a subset. Semantic inversion (“does not retry”) still needs a human.
- **Fragment matching is string search.** `InventoryWorker` in a comment will satisfy the linter. Generated comments inside source can launder a bad cite. Own the source comments too.
- **Freeze snapshots fight formatting.** Re-wrapping a shell block fails CI. That is intended. Do not auto-format freeze files.
- **This will not satisfy legal or compliance review.** Terms of service, privacy notices, and SOC narratives need a different control plane.
- **Monorepos with generated code** can cite generated stubs that lag the spec. Index the spec, not the stub, when both exist.
- **No performance claims are implied.** The script walks Markdown and does substring checks. That is the whole method.

## Who should not use this

Skip the pipeline if the repo has no stable symbols (a pure design-docs tree), if every page is already hand-written and rarely regenerated, or if on-call runbooks live in a pager tool you cannot lint. Do not use it as a reason to skip incident review. A green linter after an outage only means the cites still resolve.

## A short adoption path

1. Pick one runbook and one overview page. Do not start with the whole `docs/` tree.
2. Move commands into `freeze:command` and snapshot them.
3. Add cites only to sentences that name a type, queue, flag, or numeric constant.
4. Run `claim_lint.py` in CI as a required check on `docs/**`.
5. Allow the model to patch `draftable` lanes. Reject any patch that touches `docs/freeze/`.

The on-call page at 02:14 does not need a better adjective. It needs a command block that cannot be regenerated by accident, and a narrative that cannot mention a worker the tree no longer contains. Cite the symbol. Freeze the command. Leave the rest of the prose cheap.