DEV Community

Morgan Sun
Morgan Sun

Posted on

Model-Drafted vs Human-Owned: A Contract for Generated Docs

The incident started with a README. A generated Retry section told operators to wait 30 seconds, then POST /v2/reprocess. That route did not exist. The backoff was a fluent guess.

The model had been asked to fill in missing ops docs. It completed the pattern. Nobody had decided which headings a model was allowed to write.

Cheap generation did not create a cheap source of truth. It created a cheap way to publish authority language that no one owned.

The failure is not bad prose

Most generated documentation fails in a specific way. The sentences parse. The headings look complete. The dangerous part is the speech-act: guarantees, deadlines, contact paths, and version promises.

A model is a strong drafter of shape. It is a weak owner of commitment. Those are different jobs.

Reviewing every paragraph after the fact is slow. Banning the model from entire files is blunt. A section-level ownership contract sits in the middle.

The rule is simple to state. The model may draft only sections you classify as draftable. A human must author, or at least sign, every section that makes a commitment.

An ownership contract, not another review pass

This workflow does not score writing quality. It answers two questions before a generator runs:

  1. Which headings may a model emit?
  2. Which headings are invalid unless a named human signed them?

If a draftable section starts speaking like a contract, the checker fails. If a human-owned section is missing signed_by, the checker fails. The generator never receives the human-owned bodies.

That split is the artifact. Everything else is plumbing.

Decision table: what may be drafted

Use this table as a starting map, then shrink it. If a row is ambiguous, mark it human-owned.

Section class Model may draft? Human must own Default reason
Install from a repo script that already exists Yes, after a dry-run The dry-run still belongs to a human Shape is recoverable from files
Local examples copied from tests Yes Public API names stay human-owned Tests are a source; marketing names are not
Parameter tables from OpenAPI or JSON Schema Yes Breaking-change notes stay human Schema is primary; prose is not
Architecture narrative Draft allowed Constraints and non-goals stay human Easy to invent components
Changelog bullets from merged PR titles Draft allowed SemVer and upgrade windows stay human Dates and breakage are commitments
SLA, uptime, RPO/RTO No Yes Operational promise
Rate limits not read from config No Yes Invented numbers become policy
Security reporting and vulnerability response No Yes Abuse and legal surface
Pricing, plan limits, "unlimited" claims No Yes Commercial speech
Support hours, on-call, phone or email No Yes Contact paths rot quietly
Deprecation dates and sunset clocks No Yes Time is a promise
Compliance badges (SOC 2, HIPAA, GDPR) No Yes False certification is worse than silence

The table is a policy. Put it in git. Do not keep it in a prompt.

Repository layout

Keep three files next to the docs. The checker is standard-library Python. No package install is required.

docs/
  README.md
  ownership.json
  ownership_check.py
Enter fullscreen mode Exit fullscreen mode

ownership.json is the contract. Headings not listed are unknown. Unknown is not draftable.

{
  "files": ["docs/README.md"],
  "human_owned": [
    {"id": "sla", "heading": "Service level"},
    {"id": "security", "heading": "Security contact"},
    {"id": "limits", "heading": "Rate limits"},
    {"id": "deprecations", "heading": "Deprecations"}
  ],
  "draftable": [
    {"id": "install", "heading": "Install"},
    {"id": "quickstart", "heading": "Quickstart"},
    {"id": "flags", "heading": "CLI flags"}
  ],
  "authority_phrases": [
    "we guarantee",
    "guaranteed",
    "99.9%",
    "always available",
    "zero downtime",
    "no downtime",
    "unlimited",
    "immediately",
    "SOC 2",
    "HIPAA",
    "GDPR compliant",
    "will never",
    "24/7"
  ]
}
Enter fullscreen mode Exit fullscreen mode

Tune the phrase list per product. A games SDK and a payments API should not share the same ban list.

Marker format

Each contracted heading carries an HTML comment. Reviewers can read it. The checker can parse it. Rendered Markdown ignores it.

<!-- docs-own id=install class=draftable -->
## Install

Copy `scripts/install.sh` and run it in a throwaway directory first.

<!-- docs-own id=sla class=human signed_by=aisha date=2026-09-05 -->
## Service level

This project publishes no uptime target. Treat any generated percentage as a bug.
Enter fullscreen mode Exit fullscreen mode

Rules the checker enforces:

  • Every id in ownership.json exists in the target file.
  • class=human requires a non-empty signed_by token.
  • class=draftable must not contain an authority phrase (case-insensitive, literal match).
  • Unknown docs-own ids fail closed.
  • A contracted heading whose comment is missing fails closed.

Fail-closed is the point. Generated docs tend to fail open.

The checker

The script below is a proposed, runnable harness. It does not call a model. It only validates ownership. Run it in CI before you run any drafter.

#!/usr/bin/env python3
"""Fail if generated docs speak with unowned authority."""
from __future__ import annotations

import argparse
import json
import re
import sys
from pathlib import Path

OWN_RE = re.compile(
    r"<!--\s*docs-own\s+([^>]+?)-->\s*\n##\s+(.+?)\s*$",
    re.MULTILINE,
)
ATTR_RE = re.compile(r"(\w+)=([^\s]+)")


def parse_comment(raw: str) -> dict[str, str]:
    return {k: v for k, v in ATTR_RE.findall(raw)}


def extract_sections(text: str) -> list[dict]:
    found: list[dict] = []
    for m in OWN_RE.finditer(text):
        attrs = parse_comment(m.group(1))
        attrs["heading"] = m.group(2).strip()
        attrs["body_start"] = m.end()
        found.append(attrs)
    for i, sec in enumerate(found):
        end = found[i + 1]["body_start"] if i + 1 < len(found) else len(text)
        sec["body"] = text[sec["body_start"]:end]
    return found


def check_file(md_path: Path, contract: dict) -> list[str]:
    errors: list[str] = []
    text = md_path.read_text(encoding="utf-8")
    sections = {s.get("id"): s for s in extract_sections(text) if "id" in s}

    expected: dict[str, tuple[str, str]] = {}
    for row in contract["human_owned"]:
        expected[row["id"]] = ("human", row["heading"])
    for row in contract["draftable"]:
        expected[row["id"]] = ("draftable", row["heading"])

    for id_, (cls, heading) in expected.items():
        sec = sections.get(id_)
        if not sec:
            errors.append(f"{md_path}: missing marker id={id_} ({heading})")
            continue
        if sec.get("class") != cls:
            errors.append(
                f"{md_path}: id={id_} class={sec.get('class')} expected {cls}"
            )
        if sec.get("heading") != heading:
            errors.append(
                f"{md_path}: id={id_} heading {sec.get('heading')!r} != {heading!r}"
            )
        if cls == "human" and not sec.get("signed_by"):
            errors.append(f"{md_path}: id={id_} human section lacks signed_by")
        if cls == "draftable":
            body = sec["body"]
            for phrase in contract["authority_phrases"]:
                if re.search(re.escape(phrase), body, re.I):
                    errors.append(
                        f"{md_path}: id={id_} draftable text contains {phrase!r}"
                    )
                    break

    extra = set(sections) - set(expected)
    for id_ in sorted(extra):
        errors.append(f"{md_path}: unknown docs-own id={id_}")
    return errors


def main() -> int:
    parser = argparse.ArgumentParser()
    parser.add_argument(
        "--contract", type=Path, default=Path("docs/ownership.json")
    )
    args = parser.parse_args()
    contract = json.loads(args.contract.read_text(encoding="utf-8"))
    errors: list[str] = []
    for rel in contract["files"]:
        errors.extend(check_file(Path(rel), contract))
    if errors:
        print("ownership check failed:")
        for err in errors:
            print(f"  - {err}")
        return 1
    print("ownership check passed")
    return 0


if __name__ == "__main__":
    sys.exit(main())
Enter fullscreen mode Exit fullscreen mode

What a failing run looks like

python3 docs/ownership_check.py --contract docs/ownership.json
Enter fullscreen mode Exit fullscreen mode

Typical output after a generator "completes" a README:

ownership check failed:
  - docs/README.md: id=quickstart draftable text contains 'we guarantee'
  - docs/README.md: id=sla human section lacks signed_by
  - docs/README.md: missing marker id=security (Security contact)
Enter fullscreen mode Exit fullscreen mode

Those are cheap failures. They are cheaper than an invented /v2/reprocess.

A CI step can be three lines. If the check fails, do not call the model.

# proposal: GitHub Actions snippet
- name: Docs ownership
  run: python3 docs/ownership_check.py --contract docs/ownership.json
Enter fullscreen mode Exit fullscreen mode

Drafting only allowed sections

The drafter should be boring. Strip human-owned bodies from the prompt. Pass headings, nearby code, and the ownership table. Ask for Markdown that keeps the docs-own comments intact.

A minimal prompt shape, labeled as a proposal:

You may write bodies only for headings marked class=draftable.
Copy every HTML comment unchanged.
Do not invent headings.
Do not fill class=human sections.
If a fact is not in the attached files, write TODO(source-needed)
instead of a number, date, limit, or URL.
Enter fullscreen mode Exit fullscreen mode

Then re-run the checker on the model output. Optional: grep for TODO(source-needed) in the same job. Draftable sections may stall. They may not fabricate.

A small pre-pass helps. Before the model runs, delete bodies under class=draftable and leave the comments plus headings. The model then fills holes. It does not rewrite signed sections it cannot see.

# proposal: keep a clean copy of human-owned sections
cp docs/README.md /tmp/readme.before.md
python3 docs/ownership_check.py --contract docs/ownership.json || exit 1
# run your drafter against a redacted README here
python3 docs/ownership_check.py --contract docs/ownership.json || exit 1
Enter fullscreen mode Exit fullscreen mode

Where a free model and a free server fit

The split is what makes a free-tier model usable. You are not asking it to own production language. You are asking it to rearrange install steps and flag tables that already live in the repo.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode offers free model access and a free server option, which is enough to host this drafter-plus-checker loop if you do not want to keep a workstation awake. The checker should still run in your primary CI, on your rules, against your ownership.json. Tokens do not sign signed_by.

Give the model only draftable headings. Route human-owned files around it entirely.

Limitations

This contract does not prove that a signed section is true. It proves that someone accepted ownership.

Known gaps:

  • HTML comments are visible in the repo and can be copied without thought. signed_by=bot is a process failure, not a parser failure.
  • Authority phrases are English-centric and easy to bypass with softer wording ("we aim for four nines").
  • Heading matching is exact. Rename Rate limits to Limits and the checker fails until the contract moves with it. That is intentional, but noisy.
  • The script does not parse nested headings or non-## levels. Extend OWN_RE before you use this on a book-length manual.
  • Generated code samples can still be wrong even when they contain no banned phrases. Pair this with a command dry-run if install steps matter.
  • A free server is a convenience boundary, not an availability target. Do not document the drafter host as if it were production.

If your docs are a legal artifact, this is a lint step. It is not counsel.

Who should skip this

Skip the contract if a technical writer already owns every public page and the generator is not allowed near that tree. Skip it if your change-control system already requires named approvers per heading. Adding markers on top of that process is duplicate ritual.

Also skip it if you want the model to write runbooks, incident scripts, or customer emails. Those documents are commitments. They belong on the human side of the table, even when the prose is tedious.

For everyone else, start with one README. Map five headings. Put the checker in CI before the drafter. The interesting output is not a longer document. It is a shorter list of sentences a model is forbidden to finish.

Top comments (0)