Hash-Lock the SLA: Generate Reference, Freeze Policy

The on-call README still promised a 99.9% monthly SLA at 02:13. At 02:14 a documentation job finished. The guarantees section had been “clarified.” The number was gone. In its place: “best-effort availability.”

This is a reconstructed incident pattern, not a personal postmortem. The diff was hundreds of lines of regenerated tables, so the policy rewrite hid in noise. Reviewers skimmed the tables. They missed the sentence that actually bound the team.

Cheap generation does not make that class of text cheap to own. Reference can be rebuilt from source. Policy cannot. The rest of this article is a pipeline that encodes that split in files a CI job can refuse to merge.

Two kinds of documentation text

Most README failures after an AI draft pass are not missing parameters. They are paraphrased promises. A model is willing to smooth a constraint until it is no longer a constraint.

Treat every section as one of two classes.

Rebuildable reference. Signatures, path lists, status codes, generated config keys, example request bodies taken from fixtures. If the source of truth is in the repo, a draft may be wrong, but the wrongness is detectable.

Human policy. SLAs, data-retention periods, threat-model scope, “when not to use this,” security contacts, license exceptions, on-call severity rules. If a generator is allowed to rewrite those sentences, you no longer have a policy. You have a suggestion.

The operational rule is simple. The model never receives frozen text. An assembler splices frozen blobs back in after the draft. Hash checks prove the blobs did not move.

Ownership decision table

Use this table before you add a section to a generator prompt. If a row is ambiguous, default to human ownership.

Section type	Source of truth	Model may draft?	Human must own?	Freeze?
OpenAPI path table	openapi.yaml	Yes	Review only	No
CLI flag list	--help snapshot or clap/cobra defs	Yes	Review only	No
Example curl from fixtures	tests/fixtures/*.json	Yes	Review only	No
Environment variable names	code / schema	Yes	Review only	No
SLA / availability numbers	contract or SRE doc	No	Yes	Yes
Data retention and deletion	legal / privacy	No	Yes	Yes
Threat model and non-goals	security review	No	Yes	Yes
“When not to use this”	maintainer judgment	No	Yes	Yes
Security contact and disclosure	SECURITY.md	No	Yes	Yes
Incident severity / paging	on-call policy	No	Yes	Yes

A draft worker that only sees the first four rows cannot “improve” the last six. That is the point of the freeze map. Prompt policy alone does not hold under retry, model upgrade, or a well-meaning Please make the README shorter instruction.

Marker convention

Keep one ownership file and HTML comment markers in the Markdown. Comments survive most renderers and are easy to parse.

<!-- section:title:start -->
# Payments API
<!-- section:title:end -->

<!-- section:sla:start -->
## Availability

Monthly availability target is 99.9% for the `POST /v1/charges` path,
excluding announced maintenance windows in status.example.com.
<!-- section:sla:end -->

<!-- section:api_reference:start -->
## HTTP reference

_This block is generated. Do not edit by hand._
<!-- section:api_reference:end -->

The generator may replace only the interior of api_reference. It must copy sla from a freeze store byte-for-byte. Title stays human-owned so a model cannot rebrand the service in a draft pass.

The freeze map

docs.ownership.yml is the contract the scripts obey. Proposed schema:

# docs.ownership.yml
version: 1
doc: README.md
freeze_dir: docs/frozen
sections:
  - id: title
    owner: human
    freeze: true
  - id: sla
    owner: human
    freeze: true
  - id: threat_model
    owner: human
    freeze: true
  - id: api_reference
    owner: model
    freeze: false
    sources:
      - openapi.yaml
  - id: env_vars
    owner: model
    freeze: false
    sources:
      - src/config.rs

Frozen interiors live as raw files, not as model context:

docs/frozen/title.md
docs/frozen/sla.md
docs/frozen/threat_model.md

docs/frozen/SHA256SUMS pins them:

9c6b0a1c2f7e4d8a0b1c2d3e4f5061728394a5b6c7d8e9f0011223344556677  sla.md

If a human intends to change the SLA, they edit docs/frozen/sla.md, run the hash updater, and review that diff in isolation. They do not ask a model to “refresh the README.”

Validator (proposed harness)

Label: this script is a proposed local check, not a published benchmark. Run it in CI before the generator.

#!/usr/bin/env python3
"""Fail if frozen README sections drift from docs/frozen hashes."""
from __future__ import annotations

import hashlib
import re
import sys
from pathlib import Path

import yaml

SECTION_RE = re.compile(
    r"<!-- section:(?P<id>[a-z0-9_]+):start -->\n"
    r"(?P<body>.*?)"
    r"<!-- section:(?P=id):end -->",
    re.DOTALL,
)


def sha256_text(text: str) -> str:
    return hashlib.sha256(text.encode("utf-8")).hexdigest()


def load_sums(path: Path) -> dict[str, str]:
    sums = {}
    for line in path.read_text(encoding="utf-8").splitlines():
        if not line.strip() or line.startswith("#"):
            continue
        digest, name = line.split()
        sums[name] = digest
    return sums


def main() -> int:
    cfg = yaml.safe_load(Path("docs.ownership.yml").read_text(encoding="utf-8"))
    readme = Path(cfg["doc"]).read_text(encoding="utf-8")
    found = {m.group("id"): m.group("body") for m in SECTION_RE.finditer(readme)}
    sums = load_sums(Path(cfg["freeze_dir"]) / "SHA256SUMS")
    errors: list[str] = []

    for section in cfg["sections"]:
        sid = section["id"]
        if sid not in found:
            errors.append(f"missing markers for {sid}")
            continue
        if not section.get("freeze"):
            continue
        blob_name = f"{sid}.md"
        frozen = (Path(cfg["freeze_dir"]) / blob_name).read_text(encoding="utf-8")
        if found[sid] != frozen:
            errors.append(f"{sid}: README body != docs/frozen/{blob_name}")
        if sums.get(blob_name) != sha256_text(frozen):
            errors.append(f"{sid}: frozen file does not match SHA256SUMS")

    if errors:
        print("freeze check failed:")
        for item in errors:
            print(f"  - {item}")
        return 1
    print("freeze check ok")
    return 0


if __name__ == "__main__":
    sys.exit(main())

Install and run:

pip install pyyaml
python3 scripts/check_doc_freeze.py

A non-zero exit is the entire product. Do not warn and continue. Frozen-section drift is a merge blocker, the same way a lockfile mismatch is a merge blocker.

Generator that cannot write frozen regions

The draft step should be boring. Collect only unfrozen sources. Ask the model for those sections. Assemble the README from frozen files plus model output. Never send docs/frozen/sla.md to the model, even “for context.”

#!/usr/bin/env python3
"""Assemble README.md. Frozen sections are copied, not generated."""
from __future__ import annotations

from pathlib import Path

import yaml

# Label: stub. Replace `draft_reference` with your model client.


def draft_reference(section_id: str, source_text: str) -> str:
    """Return Markdown for one unfrozen section.

    Implementation note: pass only `source_text`. Do not attach frozen
    policy files as extra context.
    """
    raise NotImplementedError(section_id)


def wrap(section_id: str, body: str) -> str:
    body = body if body.endswith("\n") else body + "\n"
    return (
        f"<!-- section:{section_id}:start -->\n"
        f"{body}"
        f"<!-- section:{section_id}:end -->\n"
    )


def main() -> None:
    cfg = yaml.safe_load(Path("docs.ownership.yml").read_text(encoding="utf-8"))
    parts: list[str] = []
    for section in cfg["sections"]:
        sid = section["id"]
        if section.get("freeze"):
            body = (Path(cfg["freeze_dir"]) / f"{sid}.md").read_text(encoding="utf-8")
            parts.append(wrap(sid, body))
            continue
        source = []
        for rel in section.get("sources", []):
            source.append(f"# source: {rel}\n{Path(rel).read_text(encoding='utf-8')}")
        drafted = draft_reference(sid, "\n\n".join(source))
        parts.append(wrap(sid, drafted))
    Path(cfg["doc"]).write_text("\n".join(parts) + "\n", encoding="utf-8")


if __name__ == "__main__":
    main()

Suggested order in CI:

1. python3 scripts/check_doc_freeze.py on the incoming tree.
2. python3 scripts/generate_docs.py for unfrozen sections only.
3. python3 scripts/check_doc_freeze.py again. Frozen hashes must still match.
4. Fail the job if step 3 drifts. That is how you catch a generator that “helpfully” rewrote policy.

python3 scripts/check_doc_freeze.py
python3 scripts/generate_docs.py
python3 scripts/check_doc_freeze.py
git diff --exit-code -- docs/frozen README.md || true

Keep git diff on docs/frozen strict. Unfrozen reference may change when OpenAPI changes. Frozen files may not change in the same commit as a routine regen.

Where the draft worker runs

The hash gate is local. It does not need a GPU. The only networked piece is draft_reference, and it only sees rebuildable sources.

You can run that worker in existing CI, on a laptop, or on a free server. MonkeyCode is an open source project with free model access and a free server option that can host the draft worker if you do not want to keep a model client inside the main pipeline image.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The ownership YAML, frozen blobs, and SHA256SUMS still live in your repository. If the worker is unavailable, reference sections go stale. Policy sections stay correct. That failure mode is acceptable. The inverse is not.

What this catches, and what it does not

Catches

• A regen job that paraphrases an SLA while updating an endpoint table.
• A prompt change that says “tighten the README” and quietly drops paging rules.
• Accidental edits to human-owned sections mixed into a 400-line tables diff.
• A model upgrade that becomes more willing to invent softer language.

Does not catch

• Wrong reference inside an unfrozen section. If OpenAPI is wrong, the table is wrong.
• A human updating docs/frozen/sla.md to a number the business did not approve. Hashes prove stability, not truth.
• Policy written outside markers. Unmarked prose is invisible to the check.
• Legal meaning. “99.9%” that is hash-stable can still be the wrong commitment.

If you need correctness of unfrozen sections, add a separate checker: compare generated path lists to openapi.yaml and fail on extras. That is a reference linter. Do not overload the freeze script with it.

Limitations

The marker parser is whitespace-sensitive. A formatter that rewrites HTML comments, or a Markdown linter that strips them, will break the gate. Pin formatter config for those files.

SHA-256 here is integrity against accidental rewrite, not secrecy. Frozen files are still in git. Do not put credentials in docs/frozen/.

The assembler will happily publish a confident, wrong API table. Ownership split is not evaluation. If you cannot point to a source file for a generated sentence, that sentence should not be in an unfrozen section.

Teams that generate docs from chat transcripts will not get a clean source list. This workflow assumes code, OpenAPI, or fixtures exist. If they do not, freeze the whole document and stop generating it.

Who should not use this

Do not use this pipeline if every sentence in the doc is a regulated claim. Then there is no unfrozen class. Human authorship of the entire file is cheaper than a generator plus a false sense of safety.

Do not use it for marketing pages whose job is tone. Freeze maps protect commitments, not voice.

Do not use it as a substitute for review on the unfrozen tables. A model may still invent an endpoint that is not in openapi.yaml unless you lint that output.

Skip it on throwaway prototypes with no external users. A freeze map is overhead. Add it when someone outside the authoring pair will treat the README as a promise.

The durable artifact is not the model. It is the list of sections the model is not allowed to see. Keep that list in git. Let the draft worker be replaceable.