DEV Community

Avery Lin
Avery Lin

Posted on

Draft Rights, Not Draft Quality: A Gate That Separates What a Model May Write From What a Human Signs

The most expensive documentation defect is rarely a clumsy sentence; it is a promise that nobody with signing authority ever approved. Quality reviews catch the first kind and routinely miss the second, because a fluent paragraph about refunds or rate limits looks exactly like a fluent paragraph about a function signature. The rule this article builds toward is narrow: a model may draft text that the repository itself can adjudicate, while a human must own every clause that binds the product. The artifact is small enough to audit in one sitting — a JSON rules file mapping headings to authority classes, a ~90-line Python gate that enforces the mapping, and three fixtures that prove the gate fails when it should.

Three authority classes, not one quality score

Classify by who can settle the statement, not by how technical it sounds. A heading belongs to repo-checkable when a reviewer can confirm it by reading code, an OpenAPI file, or a config default without asking anyone. A heading belongs to operator-supplied when only a person with operational knowledge can supply the value, such as a retention window chosen by an on-call rotation. A heading belongs to promise when the sentence creates an obligation toward a customer, including security posture, limits, pricing, refunds, and support hours.

Class Example headings Who may draft What makes it pass
repo-checkable ## List users, ## Response fields Model At least one provenance marker in the block
operator-supplied ## Retention window, ## On-call escalation Human owner:human marker with a written reason
promise ## Refunds, ## Rate limits, ## Security Human owner:human reason, zero model markers, no unapproved promise tokens

The distinction between reporting and permission matters here. Counting who wrote what is a reporting question that belongs in a handoff log, while deciding who is allowed to write a given block is a permission question that belongs in review. This gate answers only the permission question, and it deliberately knows nothing about prose quality, factual freshness, or stylistic drift.

Step 1 — Encode the mapping in a rules file

The default class is promise, so an unclassified heading fails closed instead of drifting into model-drafted text by accident. Longest matching glob wins, which lets a specific *#Rate limits rule override a broad docs/api/*.md#* rule in the same pass.

{
  "default": "promise",
  "classes": {
    "repo-checkable": {
      "owner": "model",
      "globs": ["README.md#*", "docs/api/*.md#*", "docs/changelog.md#*"]
    },
    "operator-supplied": {
      "owner": "human",
      "globs": ["docs/ops/*.md#*"]
    },
    "promise": {
      "owner": "human",
      "globs": [
        "docs/pricing.md#*",
        "docs/legal/*.md#*",
        "*#Security",
        "*#Rate limits",
        "*#Data retention"
      ]
    }
  }
}
Enter fullscreen mode Exit fullscreen mode

Keep this file in tools/ next to the gate, and review changes to it the way you review changes to a CODEOWNERS file. Widening a repo-checkable glob is a permission change, so it deserves a second reviewer rather than a drive-by edit inside a larger docs PR.

Step 2 — Mark provenance inside the draft

Provenance is block-scoped, meaning the marker sits inside the section it describes and never leaks to sibling sections. The human marker requires a reason string of at least eight characters, which forces a short sentence instead of a reflexive checkbox.

## Retention window
<!-- owner:human reason="ops confirmed 30 days on 2026-10-12" -->
Logs are retained for 30 days and then deleted by the nightly job.

## List users
<!-- draft:model -->
Returns one page of users; see `GET /v1/users` in `openapi.yaml`.
Enter fullscreen mode Exit fullscreen mode

Two markers carry the entire contract: draft:model says a model produced the block from repository evidence, and owner:human says a named-but-not-named-in-public human signed it. Neither marker asserts that the text is correct, so the gate can stay silent about truth while remaining strict about authority.

Step 3 — Run the gate

The gate parses headings up to level three, resolves each block against the rules file, and reports violations with a file, line, class, and reason. It uses only the standard library, which matters later when you schedule it on a free host.

#!/usr/bin/env python3
"""draft_gate.py - fail docs changes that hand a promise to a model."""

import argparse
import fnmatch
import json
import re
import sys
from pathlib import Path

HEADING = re.compile(r"^(#{1,3})\s+(.+?)\s*$")
MARK_DRAFT = re.compile(r"<!--\s*draft:model\s*-->")
MARK_OWNER = re.compile(r'<!--\s*owner:human\s+reason="([^"]{8,})"\s*-->')
MARK_PROMISE = re.compile(r"<!--\s*promise:approved\s*-->")
PROMISE_WORDS = (
    "guarantee", "guaranteed", "sla", "uptime", "unlimited",
    "refund", "24/7", "will never", "no downtime",
)


def split_blocks(text):
    blocks, cur = [], {"heading": None, "line": 1, "body": []}
    for number, line in enumerate(text.splitlines(), 1):
        match = HEADING.match(line)
        if match:
            blocks.append(cur)
            cur = {"heading": match.group(2), "line": number, "body": []}
        else:
            cur["body"].append((number, line))
    blocks.append(cur)
    return [b for b in blocks if b["heading"] or any(l.strip() for _, l in b["body"])]


def classify(rules, path, heading):
    target = f"{path}#{heading}" if heading else f"{path}#"
    best = None
    for name, spec in rules["classes"].items():
        for glob in spec["globs"]:
            if fnmatch.fnmatch(target, glob):
                if best is None or len(glob) > best[0]:
                    best = (len(glob), name)
    return best[1] if best else rules["default"]


def check_block(rules, path, block):
    text = "\n".join(line for _, line in block["body"])
    klass = classify(rules, path, block["heading"])
    owner = rules["classes"][klass]["owner"]
    where = f"{path}:{block['line']}"
    label = block["heading"] or "(preamble)"
    problems = []

    if owner == "human":
        if MARK_DRAFT.search(text):
            problems.append(f"{where} [{klass}] model draft inside human-owned '{label}'")
        if not MARK_OWNER.search(text):
            problems.append(f"{where} [{klass}] '{label}' has no owner:human reason")
        return problems

    if not (MARK_DRAFT.search(text) or MARK_OWNER.search(text)):
        problems.append(f"{where} [{klass}] '{label}' has no provenance marker")
    if MARK_PROMISE.search(text):
        return problems
    for number, line in block["body"]:
        lowered = line.lower()
        hit = next((w for w in PROMISE_WORDS if w in lowered), None)
        if hit:
            problems.append(f"{where} [{klass}] promise token '{hit}' at line {number}")
            break
    return problems


def main():
    ap = argparse.ArgumentParser()
    ap.add_argument("--rules", required=True)
    ap.add_argument("--root", default=".")
    ap.add_argument("paths", nargs="*", help="glob patterns, default **/*.md")
    args = ap.parse_args()

    rules = json.loads(Path(args.rules).read_text(encoding="utf-8"))
    problems, seen = [], 0
    for pattern in (args.paths or ["**/*.md"]):
        for path in sorted(Path(args.root).glob(pattern)):
            if not path.is_file():
                continue
            seen += 1
            rel = path.relative_to(args.root).as_posix()
            for block in split_blocks(path.read_text(encoding="utf-8")):
                problems.extend(check_block(rules, rel, block))
    for p in problems:
        print(p)
    print(f"checked {seen} file(s), {len(problems)} violation(s)")
    return 1 if problems else 0


if __name__ == "__main__":
    sys.exit(main())
Enter fullscreen mode Exit fullscreen mode

The promise-token scan is intentionally crude, because a crude rule with a documented escape hatch beats a clever rule nobody can predict. When a model-drafted section legitimately needs the word "unlimited", a reviewer adds <!-- promise:approved --> and the block passes on the next run, which leaves an auditable trace in the diff.

Step 4 — Prove the gate with three fixtures

Create fixtures/ with one directory per expected outcome, then run the gate against each and compare the exit code and the violation list.

python3 tools/draft_gate.py --rules tools/draft-rights.json --root fixtures/ok "**/*.md"
python3 tools/draft_gate.py --rules tools/draft-rights.json --root fixtures/leak_promise "**/*.md"
python3 tools/draft_gate.py --rules tools/draft-rights.json --root fixtures/leak_owner "**/*.md"
echo "last exit: $?"
Enter fullscreen mode Exit fullscreen mode
Fixture Content Expected result
ok docs/api/users.md with draft:model under ## List users exit 0, 0 violation(s)
leak_promise docs/api/limits.md with draft:model plus "unlimited requests per second" under ## Rate limits exit 1, one promise token line
leak_owner docs/pricing.md with draft:model under ## Refunds exit 1, two lines: model draft inside human-owned section, missing owner:human reason

The third fixture is the one that earns its keep, because it reproduces the exact failure mode this workflow exists to prevent. A model writes a competent refund paragraph from a changelog entry, the paragraph passes spell check and tone review, and nobody notices that the section now states a policy the company never adopted.

Step 5 — Wire it into review without slowing review

Run the gate in two places, both offline. A pre-commit hook catches the mistake before the author loses context, and a CI job re-runs it as a backstop for commits made with --no-verify.

# .pre-commit-config.yaml (excerpt)
- repo: local
  hooks:
    - id: draft-gate
      name: draft rights gate
      entry: python3 tools/draft_gate.py --rules tools/draft-rights.json --root .
      language: system
      files: \.md$
Enter fullscreen mode Exit fullscreen mode
# CI job (excerpt)
docs-gate:
  image: python:3.12-slim
  script:
    - python3 tools/draft_gate.py --rules tools/draft-rights.json --root . "docs/**/*.md" "README.md"
Enter fullscreen mode Exit fullscreen mode

Keep the job dependency-free, because a gate that installs packages will eventually break for reasons unrelated to documentation authority. The whole run finishes in well under a second for a few hundred Markdown files, so there is no incentive to cache or skip it.

Where free model access and a free server fit

The drafting half needs a model that is cheap enough to run on every pull request, and the enforcement half needs a host that stays up when nobody is watching. MonkeyCode's free model access and free server are the operator-supplied options this workflow assumes for those two halves: the model drafts only repo-checkable sections such as response-field tables and changelog restatements, and the server runs the gate on a nightly schedule across docs/. Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Two design constraints keep that division honest. First, the gate never calls the model service, so documentation authority remains checkable during an outage, a rate limit, or a plan change. Second, the drafting job writes into a branch with draft:model markers already inserted, so a model that produces nothing simply yields an empty diff instead of an unattested section. Free-tier availability, quota, and hardware are not guaranteed by this article, and the workflow should be validated against current terms before you depend on it.

Limitations, and who should skip this

The gate is syntactic, so it cannot tell whether a repo-checkable paragraph actually matches the code it cites; pair it with a link checker and a code-excerpt test if that matters to you. Block boundaries stop at heading level three, which means prose sitting directly under a level-one heading is not inherited by its level-three children. The promise-word list will produce false positives in tutorials that quote policy language, and every escape hatch you add weakens the guarantee slightly.

Skip this approach if your documentation is fewer than roughly twenty pages and a single maintainer reads every change, because a human reviewer already provides the property the gate enforces. Skip it as well if your team cannot commit to writing owner:human reasons, since a gate whose escape hatch is a one-click approval trains reviewers to click rather than read.

What to keep when you leave

Three things transfer to any documentation pipeline: models draft what the repository can adjudicate, humans sign what binds the product, and the check runs offline with no dependencies. The rules file is the part worth arguing about in review, because every glob you widen is a small transfer of authority from a person to a generator.

Top comments (0)