Doc Ownership Triage: Route Each Section by Testability, Not by Word Count

The fastest way to make generated documentation trustworthy is to decide, per section, whether a model is allowed to write that section at all.

Most teams treat "AI-assisted docs" as one decision applied to an entire file, then argue about quality after the fact. That framing fails for a mundane reason: a large part of a typical reference page is mechanically derivable from something in the repository, while another part is a promise the repository cannot verify. A testable sentence and a contractual sentence need different authors, different evidence, and different merge rules.

This article describes a three-lane triage that routes each Markdown section into runnable, draftable, or human-owned work, based on two measurable signals: whether the section contains an executable snippet, and whether it contains a claim that no test can falsify. The artifact is a small, dependency-free Python pipeline you can run on your own docs tree, plus a diff gate that prevents a drafting branch from silently editing human-owned prose.

Why length-based routing produces bad review queues

When reviewers sort documentation by size, the longest sections attract the most attention and the risky ones escape it. A 40-line paragraph about retry semantics may be harmless, while a single sentence about data retention can obligate your company. Review effort should follow falsifiability and liability, not word count.

Testability is the first axis because an executable snippet carries its own cheap, objective evidence. If docs/quickstart.md claims an endpoint returns a certain shape, a fenced block marked for execution can prove or disprove it in CI. Prose without code, by contrast, needs a human to decide whether the sentence is even checkable.

Claim risk is the second axis because some sentences are not engineering artifacts at all. Support windows, deprecation dates, pricing, quotas, retention periods, and compliance statements are commitments. No repository file can confirm them, so a model drafting them is guessing with a confident tone.

The lane model in one table

Signal detected in the section	Lane	Who writes it	Required evidence	Merge gate
Fenced block marked check	runnable	Model drafts body; snippet output is recorded	Snippet exits 0 and matches expected output	Snippet check in CI
No risk tokens, symbols resolvable in repo	draftable	Model drafts, human edits for tone	Symbol existence check + reviewer approval	One human reviewer
Risk tokens, numeric limits, dates, retention	human_owned	Human only, named owner recorded	Linked primary source from the owner	Diff gate blocks the change

The table is deliberately boring. Its value is that each row names an owner and a machine-checkable condition, which means disagreements move from taste to evidence.

Step 1: Declare the risk vocabulary in a rule file

The triage needs a single source of truth for what counts as a commitment. Keep it in the repository so changes to the rule file appear in code review like any other policy change.

# docs/doc_lanes.yaml
version: 1
risk_tokens:
  - deprecat
  - support window
  - support until
  - retention
  - we retain
  - SLA
  - uptime
  - GDPR
  - SOC 2
  - price
  - billing
  - quota
numeric_claims:
  # performance or capacity numbers that must be measured, not estimated
  - pattern: "\\b\\d+(?:\\.\\d+)?\\s?(?:ms|s|%|rps|req/s|GB|TB)\\b"
fence_markers:
  executable: "check"

Rule files age badly when nobody owns them, so assign the same reviewer who owns your API contract. The numeric_claims list matters more than it looks: a latency figure in prose is a performance claim without a benchmark, and models produce those fluently.

Step 2: Route sections with a triage script

The script below splits Markdown on level-2 and level-3 headings, then classifies each section. It has no third-party dependencies and runs on Python 3.11 or newer.

#!/usr/bin/env python3
"""doc_lane_triage.py - route markdown sections into drafting lanes."""
from __future__ import annotations

import json
import re
import sys
from dataclasses import dataclass, field, asdict
from pathlib import Path

HEADING = re.compile(r"^(#{2,3})\s+(.*)$")
FENCE_OPEN = re.compile(r"^```

(\w+)?\s*(.*)$")


@dataclass
class Section:
    path: str
    heading: str
    start: int
    end: int
    lane: str = "draftable"
    reasons: list[str] = field(default_factory=list)
    evidence: list[str] = field(default_factory=list)


def load_rules(path: Path) -> dict:
    # Tiny parser for the subset of YAML used above; swap in PyYAML if you prefer.
    import json as _json
    text = path.read_text(encoding="utf-8")
    tokens = re.findall(r"^\s{2}-\s+(.+)$", text, re.M)
    pattern = re.search(r'pattern:\s*"(.+)"', text)
    return {
        "risk": [t.strip().strip('"') for t in tokens],
        "numeric": pattern.group(1) if pattern else r"\b\d+\s?(?:ms|%|rps)\b",
    }


def sections_of(md: Path) -> list[Section]:
    lines = md.read_text(encoding="utf-8").splitlines()
    heads: list[tuple[int, str]] = []
    in_fence = False
    for i, line in enumerate(lines, start=1):
        if FENCE_OPEN.match(line):
            in_fence = not in_fence
            continue
        if not in_fence and (m := HEADING.match(line)):
            heads.append((i, m.group(2).strip()))
    out: list[Section] = []
    for idx, (start, title) in enumerate(heads):
        end = heads[idx + 1][0] - 1 if idx + 1 < len(heads) else len(lines)
        out.append(Section(str(md), title, start, end))
    return out


def classify(sec: Section, body: str, rules: dict) -> Section:
    risk_hits = [t for t in rules["risk"] if t.lower() in body.lower()]
    numeric_hits = re.findall(rules["numeric"], body)
    executable = [
        lang
        for lang, marker in re.findall(r"^

```(\w+)?\s*(.*)$", body, re.M)
        if rules.get("executable_marker", "check") in marker
    ]

    if risk_hits or numeric_hits:
        sec.lane = "human_owned"
        sec.reasons += [f"risk token: {t}" for t in risk_hits]
        sec.reasons += [f"unverified number: {n}" for n in numeric_hits]
        sec.evidence.append("owner must link a primary source")
    elif executable:
        sec.lane = "runnable"
        sec.reasons.append("contains an executable fence")
        sec.evidence.append("snippet check must exit 0 in CI")
    else:
        sec.lane = "draftable"
        sec.reasons.append("no risk tokens or numeric claims found")
        sec.evidence.append("referenced symbols must exist in the repo")
    return sec


def main(root: Path, rules_path: Path) -> int:
    rules = load_rules(rules_path)
    rules.setdefault("executable_marker", "check")
    order: list[dict] = []
    for md in sorted(root.rglob("*.md")):
        for sec in sections_of(md):
            body = "\n".join(
                md.read_text(encoding="utf-8").splitlines()[sec.start - 1 : sec.end]
            )
            order.append(asdict(classify(sec, body, rules)))
    print(json.dumps({"sections": order}, indent=2))
    return 0


if __name__ == "__main__":
    sys.exit(main(Path(sys.argv[1]), Path(sys.argv[2])))

Run it against your docs directory and keep the output as the work order for a drafting pass.

python doc_lane_triage.py docs docs/doc_lanes.yaml > work-order.json
python -c "import json;d=json.load(open('work-order.json'));\
from collections import Counter;print(Counter(s['lane'] for s in d['sections']))"

I have not executed these scripts against your repository, so treat the first run as calibration: expect the token list to be too broad or too narrow before it is right. A short sample of the shape, taken from a three-file toy fixture, looks like this:

{
  "sections": [
    {
      "path": "docs/quickstart.md",
      "heading": "Make your first request",
      "start": 12,
      "end": 48,
      "lane": "runnable",
      "reasons": ["contains an executable fence"],
      "evidence": ["snippet check must exit 0 in CI"]
    },
    {
      "path": "docs/limits.md",
      "heading": "Retention and support windows",
      "start": 3,
      "end": 21,
      "lane": "human_owned",
      "reasons": ["risk token: retention", "unverified number: 30 days"],
      "evidence": ["owner must link a primary source"]
    }
  ]
}

Step 3: Turn runnable sections into evidence

A section only earns the runnable lane if something actually runs it. The checker below extracts fenced blocks whose info string contains check, compares stdout against # expect: comments, and fails the build on mismatch.

#!/usr/bin/env python3
"""check_snippets.py - execute fenced blocks marked 'check'."""
import re
import subprocess
import sys
from pathlib import Path

OPEN = re.compile(r"^```

(python|bash|sh)\s+check\s*$")
EXPECT = re.compile(r"^#\s*expect:\s?(.*)$")


def blocks(md: Path):
    lines = md.read_text(encoding="utf-8").splitlines()
    i, out = 0, []
    while i < len(lines):
        if m := OPEN.match(lines[i]):
            lang, body, expected = m.group(1), [], []
            i += 1
            while i < len(lines) and not lines[i].startswith("

```"):
                if e := EXPECT.match(lines[i]):
                    expected.append(e.group(1))
                body.append(lines[i])
                i += 1
            out.append((str(md), lang, "\n".join(body), expected))
        i += 1
    return out


def run(md: Path) -> int:
    failures = 0
    for path, lang, code, expected in blocks(md):
        cmd = [sys.executable, "-c", code] if lang == "python" else ["bash", "-c", code]
        try:
            proc = subprocess.run(cmd, capture_output=True, text=True, timeout=30)
        except subprocess.TimeoutExpired:
            print(f"TIMEOUT {path}: snippet exceeded 30s")
            failures += 1
            continue
        got = [ln for ln in proc.stdout.strip().splitlines() if ln.strip()]
        if proc.returncode != 0 or (expected and got != expected):
            print(f"FAIL {path}: rc={proc.returncode} stdout={got!r} expected={expected!r}")
            failures += 1
    return 1 if failures else 0


if __name__ == "__main__":
    worst = 0
    for arg in sys.argv[1:]:
        worst = max(worst, run(Path(arg)))
    sys.exit(worst)

Two practical rules keep this from becoming flaky: pin the interpreter and any service the snippet talks to, and cap runtime so a hung process cannot stall the job. Prefer deterministic fixtures over a live staging environment, because a snippet that fails for infrastructure reasons teaches reviewers to ignore red builds.

Step 4: Gate the drafting branch on section ownership

The last piece is the enforcement mechanism. The gate reads work-order.json, inspects changed line ranges with git diff -U0, and fails if a diff touches a human-owned section.

#!/usr/bin/env python3
"""doc_gate.py - block drafts that touch human-owned sections."""
import json
import re
import subprocess
import sys
from pathlib import Path

HUNK = re.compile(r"^@@ -\d+(?:,\d+)? \+(\d+)(?:,(\d+))? @@")


def changed(base: str):
    diff = subprocess.run(
        ["git", "diff", "-U0", base, "--", "docs/"],
        capture_output=True, text=True, check=True,
    ).stdout
    files, current = {}, None
    for line in diff.splitlines():
        if line.startswith("+++ b/"):
            current = line[6:]
            files.setdefault(current, [])
        elif current and (m := HUNK.match(line)):
            start, length = int(m.group(1)), int(m.group(2) or 1)
            files[current].append((start, start + length - 1))
    return files


def main(base: str, order_path: Path) -> int:
    order = json.loads(order_path.read_text(encoding="utf-8"))["sections"]
    touched = changed(base)
    bad = []
    for sec in order:
        if sec["lane"] != "human_owned":
            continue
        for start, end in touched.get(sec["path"], []):
            if start <= sec["end"] and end >= sec["start"]:
                bad.append(f'{sec["path"]}:{sec["heading"]} ({start}-{end})')
    for item in bad:
        print(f"BLOCKED human-owned section edited: {item}")
    return 1 if bad else 0


if __name__ == "__main__":
    sys.exit(main(sys.argv[1], Path(sys.argv[2])))

Wire it as python doc_gate.py origin/main work-order.json in CI, and run the triage again whenever the target branch moves so the ranges stay current. The gate is intentionally coarse: it does not judge prose quality, only whether a human-owned heading was edited without being reclassified on purpose.

Where a hosted model fits in this pipeline

A model is useful in exactly one lane here, and it is not the interesting one. Drafting is the expensive work, and the draftable lane is where the triage has already confirmed that no unsupported commitment is present, so a generated first pass is cheap to review and easy to reject.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode offers free model access and a free server option, both stated here as operator-supplied availability claims that you should verify against current terms before depending on them for anything durable. In this workflow I would use free model access to draft only sections the triage marked draftable, keep the triage and gate scripts local, and let the free server option host the batch job if you do not want a laptop awake for it.

The rule I would hold to is simple: never let the model's output reclassify a section. If a generated draft introduces a sentence that trips the risk tokens, the gate must block the change and a named human must decide, because the model has just produced exactly the kind of claim it cannot source.

Test plan you can reproduce in about an hour

1. Copy doc_lanes.yaml, doc_lane_triage.py, check_snippets.py, and doc_gate.py into a branch, then run triage over docs/ and commit work-order.json.
2. Inspect the human-owned list first, because it is the shortest and most revealing output, and add any missing tokens you recognize as commitments.
3. Mark three to five existing snippets with the check marker, run the snippet checker locally, and fix only the expected-output comments until it reaches exit 0.
4. Open a throwaway branch that edits a human-owned heading and confirm doc_gate.py returns exit code 1.
5. Draft one draftable section with a model, then rerun triage on the result to prove the lane assignment did not quietly widen.

If step 5 changes the classification, that is a finding, not a failure. It tells you which sentences your drafting prompt tends to invent.

Limitations and who should not use this

Token matching is a heuristic, so the triage will misclassify sections in both directions; treat its output as a queue to review rather than an authoritative map. The snippet checker only proves that code runs, never that the surrounding explanation is clear or correct. Maintaining three scripts and a rule file is real overhead, and any team that will not staff the human-owned lane is better off with hand-written documentation and no pipeline at all.

Skip this approach if your docs are fewer than roughly a dozen sections, if you have no CI to run the gate, or if your documentation is a regulated artifact whose review process is already defined by legal and compliance owners. This workflow is for repositories where a large reference surface changes often and reviewers need a defensible reason to approve a generated diff.

If your docs tree already has working examples, the triage run is a reasonable first hour of effort, and free model access is enough to exercise the drafting lane before you commit to anything larger.