DEV Community

Morgan Sun
Morgan Sun

Posted on

Who Owns This Heading? A Contract for AI-Drafted Docs

The incident ticket looked tidy. A generated runbook listed three “standard” recovery steps for a backed-up ingest queue. On-call followed step two: purge the retry topic and replay from the last checkpoint.

There was no retry topic. The checkpoint field was a leftover from another service the model had seen somewhere else. The backlog was real. The procedure was not.

That failure was not a grammar problem. It was an ownership problem. Models draft fluent sections. Humans still own the sections that can page someone at 2 a.m.

The wrong grain: file-level “AI docs”

Most documentation debates stay at the file level. Generate the README, or do not. That grain is too coarse. One markdown file mixes two kinds of claims, and they fail in different ways.

Reconstructible claims can be checked against the repo or the running binary. Public function names. Flag lists copied from --help. Column names that already live in a schema file. A model can draft these. A script can catch a miss.

Privileged claims are true only because a human decided them. Paging policy. Retention exceptions. “Safe to replay.” Vendor escalation. Which queue is allowed to drop. No training corpus owns that. If your checker cannot recover the claim from git, runtime, or a signed human note, it does not belong in a model-only paragraph.

Ownership contract (worked example)

The artifact below is a proposed workflow, not production telemetry. It treats each heading as a contract row. The model may fill rows marked model_draft. Rows marked human_own must contain a signed marker before merge.

# doc_ownership.yaml
version: 1
default_owner: human_own
rules:
  - match: "^# "
    owner: human_own
    reason: "Title and product promise."
  - match: "^## (Overview|Introduction)$"
    owner: model_draft
    reason: "Can paraphrase README purpose; must not invent SLAs."
  - match: "^## (API|CLI|Flags|Configuration Reference)$"
    owner: model_draft
    reason: "Must be regenerated from --help or OpenAPI."
  - match: "^## (Examples|Quickstart)$"
    owner: model_draft
    reason: "Commands must compile against the tree."
  - match: "^## (Runbook|On-call|Incident|Rollback|Recovery)$"
    owner: human_own
    reason: "Pages a human. Invented topics are outages."
  - match: "^## (Security|Threat Model|Secrets|PII)$"
    owner: human_own
    reason: "False negatives here are breaches, not typos."
  - match: "^## (Retention|Privacy|Compliance)$"
    owner: human_own
    reason: "Legal text is not a style problem."
  - match: "^## (Limitations|Non-goals)$"
    owner: human_own
    reason: "Models omit constraints that make the demo weaker."
  - match: "^## (Changelog|Migration)$"
    owner: human_own
    reason: "Breaking changes need a named reviewer."
signature:
  marker_prefix: "<!-- DOC-OWN"
  required_fields: ["owner", "date", "reviewer"]
Enter fullscreen mode Exit fullscreen mode

A human-owned heading is unsigned until it contains a marker like this:

## Runbook

<!-- DOC-OWN owner=human_own date=2026-09-06 reviewer=j.lee -->

If `ingest.lag` is above 5 minutes for 10 minutes, page `#oncall-ingest`.
Do not purge `events.raw`. Replay is not supported in v3.
Enter fullscreen mode Exit fullscreen mode

Unsigned, or signed with owner=model_draft on a human_own rule, is a gate failure. Fluency is irrelevant.

The gate: fail closed on unsigned ops

Label: unexecuted example you can run locally. Save as ownership_gate.py.

#!/usr/bin/env python3
"""Fail if human-owned headings are missing a DOC-OWN signature."""
from __future__ import annotations

import argparse
import re
import sys
from pathlib import Path

try:
    import yaml
except ImportError:
    sys.stderr.write("pip install pyyaml\n")
    raise SystemExit(2)

HEADING = re.compile(r"^(#{1,6})\s+(.+?)\s*$", re.M)
SIGN = re.compile(
    r"<!--\s*DOC-OWN\s+owner=(?P<owner>\S+)\s+date=(?P<date>\S+)\s+reviewer=(?P<reviewer>\S+)\s*-->"
)

def load_rules(path: Path) -> dict:
    data = yaml.safe_load(path.read_text(encoding="utf-8"))
    if not data or "rules" not in data:
        raise ValueError("doc_ownership.yaml needs a rules list")
    return data

def owner_for(heading: str, rules: list, default: str) -> tuple[str, str]:
    for rule in rules:
        if re.search(rule["match"], heading):
            return rule["owner"], rule.get("reason", "")
    return default, "default_owner"

def sections(md: str) -> list[tuple[str, str]]:
    hits = list(HEADING.finditer(md))
    out = []
    for i, m in enumerate(hits):
        start = m.end()
        end = hits[i + 1].start() if i + 1 < len(hits) else len(md)
        title = f"{m.group(1)} {m.group(2)}"
        out.append((title, md[start:end]))
    return out

def check_file(md_path: Path, cfg: dict) -> list[str]:
    text = md_path.read_text(encoding="utf-8")
    errors = []
    default = cfg.get("default_owner", "human_own")
    for title, body in sections(text):
        owner, reason = owner_for(title, cfg["rules"], default)
        if owner != "human_own":
            if SIGN.search(body):
                errors.append(
                    f"{md_path}: {title} is model_draft but has a DOC-OWN marker ({reason})"
                )
            continue
        found = SIGN.search(body)
        if not found:
            errors.append(f"{md_path}: {title} is human_own and UNSIGNED ({reason})")
            continue
        if found.group("owner") != "human_own":
            errors.append(
                f"{md_path}: {title} signed as {found.group('owner')}, expected human_own"
            )
        if found.group("reviewer") in {"model", "ai", "todo", "tbd"}:
            errors.append(f"{md_path}: {title} reviewer={found.group('reviewer')} is not a person")
    return errors

def main() -> int:
    p = argparse.ArgumentParser()
    p.add_argument("--contract", type=Path, default=Path("doc_ownership.yaml"))
    p.add_argument("paths", nargs="+", type=Path)
    args = p.parse_args()
    cfg = load_rules(args.contract)
    errors = []
    for path in args.paths:
        errors.extend(check_file(path, cfg))
    for err in errors:
        print(err, file=sys.stderr)
    print(f"checked={len(args.paths)} errors={len(errors)}")
    return 1 if errors else 0

if __name__ == "__main__":
    raise SystemExit(main())
Enter fullscreen mode Exit fullscreen mode

Run it against a fixture, not against hope:

pip install pyyaml
python ownership_gate.py --contract doc_ownership.yaml docs/runbook.md docs/api.md
Enter fullscreen mode Exit fullscreen mode

A passing run prints checked=2 errors=0. A failing run names the heading. That is the entire point: the build should refuse invented recovery, not debate tone.

What the model may draft

Keep the allow-list boring. Boring is checkable.

  1. API and CLI reference generated from OpenAPI, JSON Schema, or a captured --help dump. Diff the dump in CI. If the dump moved and the doc did not, fail.
  2. Happy-path quickstarts whose commands are executed in a container against the same commit.
  3. Table-of-contents and cross-links after headings exist. Link rot is a graph problem, not a prose problem.
  4. Paraphrase of in-repo purpose when README already states it. The model is a compressor here, not a source of truth.
  5. Error catalog scaffolding when codes come from an enum in source. The model may list codes. It may not invent mitigation.

If a section cannot be rebuilt from a command, delete the draft and regenerate. Do not “touch up” a hallucinated flag. Touch-ups train the next draft to keep the flag.

What a human must own

These sections can be prompted, but they cannot be authorized by a model.

  1. Runbooks and rollback. Topic names, purge permission, replay windows, and “never do X” lists.
  2. Paging and severity. Who wakes up, after how long, and which metric. A wrong threshold is an SLO change.
  3. Security boundaries. What is secret, what is PII, what is logged, what is redacted.
  4. Retention and deletion. Especially anything that contradicts a default the model likes (“we keep logs 30 days”).
  5. Migration and breakage. A changelog line that says “backwards compatible” without a named reviewer is a rumor.
  6. Limitations. Capacity, single-region, eventual consistency, “we do not support replay.” Models drop these because they make the story less complete.

A useful house rule: if following the paragraph can destroy data or wake a person, the reviewer field cannot be a model id.

A loop that keeps the split honest

Do not start from a blank page. Start from headings plus owner tags.

  1. Check in doc_ownership.yaml with the docs, not in a wiki aside.
  2. Scaffold headings only. Leave human_own bodies empty except for <!-- DOC-OWN ... UNSIGNED --> if you want a louder failure.
  3. Let a model draft model_draft sections from local artifacts: --help, OpenAPI, schema files. Paste nothing from chat history that you cannot re-derive.
  4. Run ownership_gate.py in CI. Then run the source checks: help-text diff, example commands, link checker. The ownership gate does not replace those.
  5. A named reviewer fills human_own sections from runbooks, tickets, and production config. They sign the marker. They do not ask the model to “make it sound official.”

Disclosure: This article was prepared as part of MonkeyCode's product outreach. If you want a scratch box to draft the model_draft sections and run the gate without standing up GPU hardware, MonkeyCode’s free model access and free server option can host that loop. The gate is still ordinary Python. Treat model output as untrusted input to step 4, not as a signature for step 5.

Decision table

Section kind Model may draft? Human must sign? Machine check
CLI flags Yes, from --help No Byte-diff against captured help
Quickstart commands Yes No Execute in CI
Overview paraphrase Yes, from README Spot-check No extra claims vs README
On-call runbook Scaffold headings only Yes DOC-OWN marker + reviewer ≠ model
Rollback / purge No Yes Marker; reject verbs like “simply drop” without allow-list
Security / PII No Yes Marker; secret-scan the doc
Retention No Yes Marker; compare to config defaults
Limitations Notes only Yes Marker; fail if section missing
Migration Diff summary only Yes Marker on breaking headings

Use the table as a PR template. If a reviewer cannot point to a row, the section does not merge.

Limitations

The gate is syntactic. It cannot tell a true runbook from a confident wrong one. A signed paragraph can still be stale the day after a topology change.

Heading regexes are brittle. Rename ## Runbook to ## When things break and the rule misses unless you update the contract. That is a feature if you treat the YAML as code. It is a footgun if you treat it as a style guide.

The workflow assumes English ATX markdown. ReStructuredText, generated HTML, and Notion exports need a different parser. Nested lists inside a human-owned section are not validated for facts.

Free model access does not change the ownership split. A cheaper draft still cannot authorize a purge. A free server is a place to run the linter and the help-text diff. It is not a substitute for a reviewer who has seen production.

Who should not use this

Skip this contract if your docs are throwaway spikes with no on-call. A README that will die with the branch does not need signatures.

Do not use it as a substitute for generated reference docs. If you already emit API pages from OpenAPI, keep that pipeline. Wrapping those pages in DOC-OWN markers only adds theater.

Do not use it to launder privileged claims. Filling reviewer=alice because the model suggested Alice is worse than an unsigned heading. The marker is an audit record. Forged records are a process bug.

Teams that cannot name a human for security and rollback should not ship those sections at all. Silence is better than a fluent invention.

Close the loop on the next PR

Pick one existing markdown file. Split its headings into the two columns. Add the YAML and the script. Merge nothing until errors=0.

The model can still write. It just cannot countersign the parts that hurt when they are wrong.

Top comments (0)