DEV Community

Avery Lin
Avery Lin

Posted on

When a Model-Drafted Doc Section Earns a Human Owner: An Escalation Ladder and a Trailer Gate

Most documentation guardrails try to police the draft, but this one decides when a section stops being draftable at all. The mechanism is a four-trigger escalation table plus a small git-trailer gate, and the payoff is that reviewers spend attention on the few sections that carry real risk. Everything below runs on repo-native inputs: headings, anchors, git grep hits, commit churn, and one hand-maintained impact list.

The scope is deliberate. This workflow covers reference and how-to pages that a model may partially regenerate, and it assumes you already have CI that runs on pull requests. It does not attempt to decide whether a sentence is true, because that question needs a source registry and a different gate entirely.

Why ownership beats per-run gating

A draft gate answers one question per generation run: is this output acceptable right now? An ownership table answers a different question that survives across runs: is this section allowed to be generated at all by anyone other than a named human? The first is a filter, and the second is a boundary, and you need both because filters are forgettable while boundaries are enforceable.

The practical failure mode is silent promotion. A model drafts an auth section in March, a human fixes two sentences in April, and by June nobody remembers that the paragraph about token expiry was machine-authored text nobody ever verified. A trailer on the commit plus a per-section owner marker makes that history greppable instead of tribal.

Step 1: Name four escalation triggers

A trigger is only useful if a script can evaluate it without asking a human for an opinion. I use four, each mapped to a command you already have installed, and each with a threshold you will need to tune to your repo size. Write the thresholds down in the script, not in a wiki page, so a threshold change shows up in code review.

  1. Reach — the section anchor appears in three or more source or example files. High reach means a wrong sentence propagates into user code.
  2. Churn — the section's file changed in three or more commits in the last ninety days. Churn means the underlying behavior is still moving and the text tracks it.
  3. Impact — the anchor appears on a hand-maintained list of sections that were cited in a support ticket or incident retrospective. This is the only subjective input, and keeping it in a reviewed JSON file keeps it small.
  4. Irreversibility — the section covers install, authentication, security, migration, or billing. A reader who follows a wrong irreversible step cannot undo it cheaply, so this trigger escalates alone.

The rule is then mechanical: irreversibility always escalates, and any two of the other three escalate. One trigger alone leaves the section model-draftable, which keeps the human-owned set small enough that it actually gets read.

Step 2: Score sections from repo data

The scorer walks the docs tree, extracts headings as anchors, and evaluates each trigger. It writes a single JSON manifest that later steps consume, so the gate never re-implements the thresholds. Treat this as a reference implementation and profile it before wiring it into a large corpus.

#!/usr/bin/env python3
"""tools/doc_owner_score.py — classify doc sections as model-draftable or human-owned."""
import json
import re
import subprocess
from pathlib import Path

ROOT = Path(__file__).resolve().parents[1]
DOCS = ROOT / "docs"
IMPACT_FILE = DOCS / "impact.json"  # {"anchors": ["auth-tokens", "billing-proration"]}
IRREVERSIBLE = ("install", "auth", "security", "migration", "billing")

REACH_MIN, CHURN_MIN, CHURN_DAYS = 3, 3, 90
HEADING = re.compile(r"^(##|###)\s+(.+)$", re.M)


def git(*args: str) -> str:
    return subprocess.run(
        ["git", *args], cwd=ROOT, capture_output=True, text=True, check=False
    ).stdout


def slug_of(title: str) -> str:
    return re.sub(r"[^a-z0-9]+", "-", title.lower()).strip("-")


def code_refs(slug: str) -> int:
    hits = git("grep", "-l", "-F", f"#{slug}", "--", "src", "examples")
    return len([h for h in hits.splitlines() if h.strip()])


def churn(path: Path, slug: str) -> int:
    """Count recent commits whose patch mentions this anchor. Cache this in CI."""
    count = 0
    for sha in git("log", f"--since={CHURN_DAYS}.days", "--format=%H", "--", str(path)).split():
        patch = git("show", "--unified=0", "--format=", sha, "--", str(path)).lower()
        count += slug in patch
    return count


def classify(slug: str, path: Path, impact: set[str]) -> dict:
    flags = {
        "reach": code_refs(slug) >= REACH_MIN,
        "churn": churn(path, slug) >= CHURN_MIN,
        "impact": slug in impact,
        "irreversible": any(word in slug for word in IRREVERSIBLE),
    }
    secondary = sum(flags[k] for k in ("reach", "churn", "impact"))
    owner = "human" if flags["irreversible"] or secondary >= 2 else "model"
    return {"anchor": slug, "file": str(path.relative_to(ROOT)), "owner": owner, "flags": flags}


def main() -> None:
    impact = set(json.loads(IMPACT_FILE.read_text()).get("anchors", []))
    sections = []
    for path in sorted(DOCS.rglob("*.md")):
        for match in HEADING.finditer(path.read_text()):
            sections.append(classify(slug_of(match.group(2)), path, impact))
    DOCS.joinpath("ownership.json").write_text(
        json.dumps({"sections": sections}, indent=2) + "\n"
    )
    human = [s["anchor"] for s in sections if s["owner"] == "human"]
    print(f"{len(sections)} sections, {len(human)} human-owned")
    for anchor in human:
        print(f"  human: {anchor}")


if __name__ == "__main__":
    main()
Enter fullscreen mode Exit fullscreen mode

The churn function runs git show once per commit, which is acceptable for a nightly job and unacceptable for a pre-commit hook. Cache the patch text by SHA if you need it on every push, and expect the first run on a large history to take noticeably longer than later runs.

Step 3: Mark ownership inline, not out of band

The manifest tells CI what the rules produced, but a reviewer reading the Markdown should not have to open a second file. Put a single HTML comment above each section that states its owner, and let the manifest be a derived artifact that the gate cross-checks. Comments render invisibly on the published page and survive Markdown linting.

<!-- owner:human anchor:auth-tokens -->
## Rotating API tokens

<!-- owner:model anchor:list-supported-regions -->
## Supported regions
Enter fullscreen mode Exit fullscreen mode

If the manifest and the inline marker disagree, the gate should fail rather than pick a winner. A disagreement means someone hand-edited the manifest or forgot to re-score after a large content change, and both are worth a human look before a model touches the page again.

Step 4: Gate model-drafted commits with a trailer

Ownership only binds if the commit that violates it cannot merge. Tag machine-assisted commits with a Drafted-By trailer, then have the gate map each changed line range in a pull request back to its nearest preceding owner marker. Git's --trailer flag handles the tagging on recent versions, and older clients can append the same two lines to the message body by hand.

git commit -m "docs: regenerate model-owned reference sections" \
  --trailer "Drafted-By: model" \
  --trailer "Draft-Run: 2026-10-17T09:00Z"
Enter fullscreen mode Exit fullscreen mode
#!/usr/bin/env python3
"""tools/doc_gate.py — fail a model-drafted commit that edits human-owned lines."""
import re
import subprocess
import sys
from pathlib import Path

MARK = re.compile(r"<!--\s*owner:(human|model)\b")
HUNK = re.compile(r"^@@ -\d+(?:,\d+)? \+(\d+)(?:,\d+)? @@")


def drafted_by() -> str:
    out = subprocess.run(
        ["git", "log", "-1", "--format=%(trailers:key=Drafted-By,valueonly)"],
        capture_output=True, text=True, check=False,
    ).stdout.strip().lower()
    return out


def owner_at(path: Path, line_no: int) -> str:
    marks = [(path.read_text()[: m.start()].count("\n") + 1, m.group(1)) for m in MARK.finditer(path.read_text())]
    current = "human"  # unmarked sections default to human
    for mark_line, owner in marks:
        if mark_line <= line_no:
            current = owner
    return current


def main() -> int:
    if drafted_by() != "model":
        print("human-authored commit, skipping ownership gate")
        return 0

    diff = subprocess.run(
        ["git", "diff", "--unified=0", "origin/main...HEAD", "--", "docs"],
        capture_output=True, text=True, check=False,
    ).stdout

    violations, path, pending = [], None, None
    for line in diff.splitlines():
        if line.startswith("+++ b/"):
            path = Path(line[6:])
        elif (hunk := HUNK.match(line)):
            pending = int(hunk.group(1))
        elif pending is not None and path is not None and line[:1] in ("+", "-"):
            if owner_at(path, pending) == "human":
                violations.append(f"{path}:{pending}")

    if violations:
        print("model-drafted commit touched human-owned sections:")
        for spot in sorted(set(violations)):
            print(f"  {spot}")
        return 1
    return 0


if __name__ == "__main__":
    sys.exit(main())
Enter fullscreen mode Exit fullscreen mode
name: doc-ownership
on: [pull_request]
jobs:
  gate:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with: { fetch-depth: 0 }
      - name: Ownership gate
        run: python3 tools/doc_gate.py
Enter fullscreen mode Exit fullscreen mode

The gate defaults unmarked sections to human, which means adding a new page requires an explicit owner:model decision before a batch job may rewrite it. That default is intentional and slightly annoying, because the annoying version is the one that survives a busy week.

Step 5: Draft only the model-owned set

The nightly job reads docs/ownership.json, filters to owner == "model", and regenerates only those sections from their source inputs. I run that batch against MonkeyCode's free model access, which keeps the marginal cost of a full regeneration pass at zero for a repo of this size. The scheduler itself lives on the free server option rather than a paid runner, so a failed pass costs nothing but a retry. Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Because a free tier carries no availability promise, the job is written to be idempotent and restartable: it writes to a branch, never to main, and it re-reads the manifest on every run so a section promoted by the scorer cannot be regenerated. I cannot quote quotas or guarantees here, and you should not assume any before scheduling a job that must run at a fixed hour.

python3 tools/doc_owner_score.py
git checkout -B docs/auto-refresh
python3 tools/draft_model_sections.py --manifest docs/ownership.json --out docs/
git commit -am "docs: regenerate model-owned reference sections" \
  --trailer "Drafted-By: model" \
  --trailer "Draft-Run: $(date -u +%Y-%m-%dT%H:%MZ)"
Enter fullscreen mode Exit fullscreen mode

The escalation table at a glance

Reach Churn Impact Irreversible Owner Reviewer action
any any any yes human read every changed sentence
yes yes no no human read every changed sentence
yes no yes no human read every changed sentence
no yes yes no human read every changed sentence
yes no no no model skim diff, check anchors
no no no no model skim diff, check anchors

When a section flips from model to human, the flip is the deliverable. Open a small pull request that only changes the marker comment and regenerates the manifest, so the promotion gets its own reviewable moment instead of hiding inside a large regeneration diff.

Limitations and who should skip this

Four triggers are a heuristic, not a measurement of correctness, and a section can be wrong while scoring zero on every axis. The scoring script assumes anchors are stable, so renaming headings resets churn history and quietly un-escalates sections that deserve attention. Repos with fewer than roughly twenty doc pages will find the table overhead larger than the review time it saves.

Skip this if every sentence in your docs already needs a named human author for compliance reasons, because then the whole tree is human-owned and the gate degenerates into an always-fail check. Skip it too if you have no pull-request CI, since a trailer nobody enforces is just a commit message.

The piece worth adapting first is the trigger thresholds, not the code. Lower REACH_MIN to two on a small public API surface and watch how many sections flip to human before you automate anything downstream.

Top comments (0)