A payments partner voided the same invoice three times. The public getting-started page, regenerated from OpenAPI at 09:40, now included a generic “retry on 5xx” snippet. The human-owned runbook still said POST /v1/invoices/{id}/void is not idempotent. Both texts lived in docs/payments.md. The model did not invent a new policy. It filled a heading that had no owner.
That is the failure this article treats. Generated prose is cheap. Ownership is not. If a heading can be rewritten on every schema refresh, it must be classified as draftable. If a heading is a promise—retry rules, deprecation windows, PII handling, escalation paths—it must have a human owner and a hash that CI can refuse to silently change.
The rest of this piece is a proposed, reproducible method: a heading-level ownership registry, a stdlib checker, a slicer that keeps owned text out of the model prompt, and a decision table for what belongs on each side.
The incident, reduced to a file
Composite, but typical. One Markdown file. Two jobs. A generator rewrites “Getting started” from the spec. A human is supposed to own “Voiding an invoice.” After the morning regen, git blame pointed at the bot for both headings. On-call learned about the retry text from a partner ticket, not from review.
Markdown has headings. It does not have owners. Tools that “just regenerate the docs” treat the file as one blob. The blob is the bug.
Draftable vs owned is not a style choice
Classify by consequence, not by how confident the model sounds. A fluent paragraph about retries is still a promise if a customer will act on it.
Decision table
| Doc surface | Model may draft? | Human must own? | Why | Failure if the model rewrites it |
|---|---|---|---|---|
| Parameter tables derived from OpenAPI | Yes | No | Spec is the source | Stale field names |
Sandbox curl examples |
Yes | Review examples that hit auth | Shape can regen | Token pasted into prod |
| Observed rate limits (informational) | Yes, if labeled current | No | Snapshot, not contract | Partners treat a snapshot as SLA |
| Contractual rate limits | No | Yes | Billing and abuse | Silent tightening or loosening |
| Retry / idempotency guidance | No | Yes | Side effects | Duplicate charges, double voids |
| Deprecation dates and sunset windows | No | Yes | Calendar is a promise | Early break, late break |
| Support hours and escalation | No | Yes | Staffing | Page the wrong rotation |
| PII, retention, subprocessors | No | Yes | Legal | Unreviewed processing claims |
| Architecture rationale | Yes | Edit for accuracy | Explanatory | Harmless drift, usually |
| Breaking-change language | No | Yes | Version contract | Accidental commitment |
If a row can move money, data, or a page, it is owned. If a row can be rebuilt from a machine-readable source, it is draftable.
Artifact: a heading ownership registry
Proposed format. JSON, so the checker stays on the standard library. One registry file per Markdown doc, checked in next to it.
{
"file": "docs/payments.md",
"headings": [
{
"id": "getting-started",
"match": "Getting started",
"level": 2,
"policy": "draft",
"owner": null,
"forbidden": ["SLA", "guarantee", "do not retry", "PII", "we will not"]
},
{
"id": "auth-headers",
"match": "Authentication headers",
"level": 3,
"policy": "draft",
"owner": null,
"forbidden": ["production secret", "never rotate"]
},
{
"id": "voiding",
"match": "Voiding an invoice",
"level": 2,
"policy": "own",
"owner": "payments-oncall",
"hash": "replace-me"
},
{
"id": "deprecation",
"match": "Deprecation window",
"level": 2,
"policy": "own",
"owner": "api-council",
"hash": "replace-me"
}
]
}
Rules the registry encodes:
- Every
##/###in the doc must appear inheadings. - Every registry row must match exactly one heading.
-
policy: ownrows store a SHA-256 of the normalized section body. -
policy: draftrows may change freely, but CI rejects forbidden phrases. - Changing an owned hash is a human action (
sign_owned.py), not a regen side effect.
Normalize, then hash
Whitespace and trailing newlines will false-fail any lock. Normalize before hashing. Proposed function:
# ownership_lib.py
from __future__ import annotations
import hashlib
import json
import re
from pathlib import Path
HEADING_RE = re.compile(r"^(#{2,3})\s+(.+?)\s*$", re.M)
def normalize(text: str) -> str:
lines = [line.rstrip() for line in text.replace("\r\n", "\n").split("\n")]
while lines and lines[-1] == "":
lines.pop()
return "\n".join(lines) + "\n"
def sha256(text: str) -> str:
return hashlib.sha256(normalize(text).encode("utf-8")).hexdigest()
def split_sections(markdown: str) -> list[dict]:
matches = list(HEADING_RE.finditer(markdown))
sections = []
for i, match in enumerate(matches):
start = match.start()
end = matches[i + 1].start() if i + 1 < len(matches) else len(markdown)
level = len(match.group(1))
title = match.group(2).strip()
sections.append({
"level": level,
"title": title,
"body": markdown[start:end],
})
return sections
def load_registry(path: Path) -> dict:
return json.loads(path.read_text(encoding="utf-8"))
Unexecuted example: if your renderer injects extra blank lines, keep normalize() as the only hash input. Do not hash the raw file.
Checker: fail closed on owned drift
# check_ownership.py
from __future__ import annotations
import argparse
import sys
from pathlib import Path
from ownership_lib import load_registry, sha256, split_sections
def main() -> int:
parser = argparse.ArgumentParser()
parser.add_argument("--registry", required=True)
parser.add_argument("--root", default=".")
args = parser.parse_args()
registry = load_registry(Path(args.registry))
doc_path = Path(args.root) / registry["file"]
markdown = doc_path.read_text(encoding="utf-8")
sections = split_sections(markdown)
by_title = {(s["level"], s["title"]): s for s in sections}
errors: list[str] = []
seen = set()
for row in registry["headings"]:
key = (row["level"], row["match"])
seen.add(key)
section = by_title.get(key)
if section is None:
errors.append(f"missing heading: {row['match']}")
continue
if row["policy"] == "own":
digest = sha256(section["body"])
if digest != row.get("hash"):
errors.append(
f"owned heading drifted: {row['match']} "
f"owner={row.get('owner')} expected={row.get('hash')} got={digest}"
)
elif row["policy"] == "draft":
lower = section["body"].lower()
for phrase in row.get("forbidden") or []:
if phrase.lower() in lower:
errors.append(
f"draft heading contains owned language: {row['match']!r} -> {phrase!r}"
)
else:
errors.append(f"unknown policy on {row['id']}: {row['policy']}")
for section in sections:
key = (section["level"], section["title"])
if key not in seen:
errors.append(f"unregistered heading: {section['title']}")
for err in errors:
print(err, file=sys.stderr)
if errors:
print(f"{len(errors)} ownership error(s)", file=sys.stderr)
return 1
print(f"ok: {registry['file']} ({len(registry['headings'])} headings)")
return 0
if __name__ == "__main__":
sys.exit(main())
Run it the same way in CI and locally:
python check_ownership.py --registry docs/payments.ownership.json --root .
Expected first run: every own row fails until you sign. That is intended. Unsigned owned text is not “flexible.” It is unowned.
Sign only after a human edit
# sign_owned.py
from __future__ import annotations
import argparse
import json
from pathlib import Path
from ownership_lib import load_registry, sha256, split_sections
def main() -> None:
parser = argparse.ArgumentParser()
parser.add_argument("--registry", required=True)
parser.add_argument("--root", default=".")
parser.add_argument("--id", required=True, help="heading id to re-sign")
args = parser.parse_args()
path = Path(args.registry)
registry = load_registry(path)
markdown = (Path(args.root) / registry["file"]).read_text(encoding="utf-8")
sections = {(s["level"], s["title"]): s for s in split_sections(markdown)}
for row in registry["headings"]:
if row["id"] != args.id:
continue
if row["policy"] != "own":
raise SystemExit(f"{args.id} is not owned")
section = sections[(row["level"], row["match"])]
row["hash"] = sha256(section["body"])
path.write_text(json.dumps(registry, indent=2) + "\n", encoding="utf-8")
print(f"signed {args.id} -> {row['hash']}")
return
raise SystemExit(f"unknown id: {args.id}")
if __name__ == "__main__":
main()
A regen job may rewrite draft headings and commit them. It must not invoke sign_owned.py. If the hash file changes in the same commit as a bot regen, review should reject the commit. That review rule is social. The checker is mechanical. You want both.
Slice owned text out of the prompt
Most “the model overwrote the runbook” bugs start with a full-file prompt. The model sees the promise, then rephrases it into the how-to. Stop giving it the promise.
# slice_draftable.py
from __future__ import annotations
import argparse
import json
from pathlib import Path
from ownership_lib import load_registry, split_sections
PROMPT_PREFIX = """You may rewrite only the sections below.
Do not add guarantees, SLAs, retry bans, PII rules, or deprecation dates.
Keep heading titles unchanged. Return Markdown sections only.
"""
def main() -> None:
parser = argparse.ArgumentParser()
parser.add_argument("--registry", required=True)
parser.add_argument("--root", default=".")
parser.add_argument("--out", required=True)
args = parser.parse_args()
registry = load_registry(Path(args.registry))
markdown = (Path(args.root) / registry["file"]).read_text(encoding="utf-8")
sections = {(s["level"], s["title"]): s for s in split_sections(markdown)}
parts = [PROMPT_PREFIX]
for row in registry["headings"]:
if row["policy"] != "draft":
continue
section = sections[(row["level"], row["match"])]
parts.append(section["body"].rstrip() + "\n")
Path(args.out).write_text("\n".join(parts), encoding="utf-8")
print(f"wrote {args.out}")
if __name__ == "__main__":
main()
Proposed loop:
-
slice_draftable.pywritesdraft-packet.md. - A model rewrites only that packet.
- A small merge script replaces draft headings by title, never owned ones.
-
check_ownership.pyruns. Owned hashes must be unchanged. Draft forbidden phrases must be absent. - Humans update owned headings in a separate PR, then
sign_owned.py.
The merge script is boring on purpose. Title match, then splice. If a title is missing, fail. Do not fuzzy-match a deprecation section into “Getting started.”
Where a free model pass fits
The draft packet is the only thing that should leave the repo. Owned sections stay local. That split is the method. The model is interchangeable.
Disclosure: This article was prepared as part of MonkeyCode's product outreach. If you want the draft pass off your laptop, MonkeyCode is one open-source option with free model access and a free server option. Point it at draft-packet.md, not at docs/payments.md. The registry and the checker do not depend on that choice. They still run in CI if you swap the model later.
Keep product setup out of the owned headings. A getting-started draft can mention a sandbox base URL. It should not mention on-call aliases, retention days, or “safe to retry.”
Test plan (run these before you trust CI)
Label: unexecuted until you point it at a real file.
-
Coverage. Add an unregistered
## Shadow heading. Checker must exit 1. -
Owned drift. Append a sentence to an
ownsection without signing. Checker must exit 1 and print the new hash. -
Sign path. Run
sign_owned.py --id voiding. Checker must exit 0. -
Draft poison. Insert
we guaranteeunder adraftheading. Checker must exit 1. -
Prompt isolation. Confirm
draft-packet.mdcontains no owned heading titles from the registry. -
Bot boundary. In a sample diff, a regen commit that touches
hashmust be treated as a review failure even if the checker was skipped.
If test 5 fails, stop generating. You are still pasting promises into the prompt.
Limitations
Heading policy is coarse. A promise buried in a bullet under “Overview” will be classified as draftable if that heading is draftable. Move promises under their own H2.
Hashing is brittle against prettier, link checkers that rewrite URLs, and locale-specific quotes. Normalize more aggressively, or freeze owned sections as separate files (voiding.md) and !include them. This article does not implement includes.
Forbidden-phrase lists are not a legal review. They catch accidental contract language in how-to text. They will miss a politely worded commitment that names no keyword.
The splitter only handles ## and ###. Front matter, HTML <h2>, and Setext headings are invisible to it. Extend HEADING_RE before you roll this out to a mixed corpus.
Who should not use this
Do not use a heading registry as the system of record for contracts, SOC reports, or anything a lawyer must sign. Generate those from a source that is already owned, or do not generate them.
Skip this if the entire README is a promise (status pages, incident policy, data-processing addenda). There is nothing draftable. A model has no lane.
Skip this if you have no CI gate. An unsigned JSON file next to the docs is documentation of intent, not enforcement.
Teams that already generate reference pages from OpenAPI and keep runbooks in a different repo may not need this. Their split is the directory. This method is for the common case: one Markdown file, two kinds of truth, one regen job that cannot tell them apart.
The model can draft the how. A human still owns the promise. Register the heading before the next OpenAPI refresh, not after the partner retries void.
Top comments (0)