Split a Doc Diff by Claim Class Before It Reaches Review

Most documentation-generation failures are review failures rather than generation failures. A model writes twelve clean paragraphs, the reviewer skims the diff, and the single sentence that required a human decision ships unread. The workflow below classifies every changed documentation region before review, then fails CI when a generated commit touches a region that nobody is allowed to generate.

The claim is modest and checkable: the bottleneck is where reviewer attention lands, not how fluent the draft is. If you can route each changed region to the correct reviewer with the correct evidence attached, the model can draft far more of the page than most teams currently allow.

1. Three claim classes, three kinds of evidence

The classification only works if the classes map to different evidence requirements, not different writing styles. Style-based rules collapse under review pressure; evidence-based rules do not.

Class	Who may write it	Required evidence	Reviewer role	Gate behaviour
executable	model or human	committed test, schema, or fixture that the section describes	verify-by-execution	passes only if every evidence path exists in the tree
prose	model or human	none beyond the surrounding page	prose review	routed, never blocked
judgment	human only	a recorded decision, not an artifact	domain owner	blocked unless the commit carries an approval trailer

judgment is the class people misclassify most often, usually by treating "we decided to deprecate X" as prose. It is not prose, because no artifact in the repository can confirm or refute it; only a person can.

2. The claim map is the only new file

Sections are addressed as path#heading-slug, which keeps the map diffable in one review and avoids polluting published Markdown with sentinel comments.

# docs/claimmap.yaml
classes: [executable, prose, judgment]

sections:
  - key: "reference/rate-limits.md#response-headers"
    class: executable
    evidence:
      - tests/test_rate_limit_headers.py
      - openapi/rate-limit.yaml

  - key: "reference/rate-limits.md#retry-guidance"
    class: judgment
    id: rate-limits-retry-policy

  - key: "guides/migrating-v2.md#step-2-update-the-client"
    class: prose

The tradeoff is granularity. Heading-level regions are coarse, so a long section that mixes a measured table with an editorial recommendation has to be split before it can be classified honestly. That splitting work is the real cost of this approach, and it is also the part that improves the page.

3. Route the diff by hunk, not by file

File-level ownership is too blunt for documentation, because a single generation commit usually rewrites some sections and leaves others untouched. The gate below reads git diff --unified=0 against a base ref, parses each @@ -a,b +c,d @@ header, and maps the new-file line ranges onto heading spans.

This is why -U0 matters: with default context you get changed lines plus unchanged neighbours, and the extra context blurs which region the commit actually touched. With zero context, a hunk is exactly the set of added or replaced lines.

Two consequences follow. First, pure deletions produce no new-file lines and therefore no route, which is usually correct and occasionally wrong; treat deletion-only hunks as a manual follow-up. Second, a hunk that spans a heading boundary inside one added block is attributed to both regions, which is safe but noisy.

4. A reference gate you can adapt

The script below is a reference implementation rather than a tested product. It is labelled as unexecuted here: run it against a throwaway branch and inspect the routing report before you put it in front of reviewers.

#!/usr/bin/env python3
"""docgate.py: route a documentation diff by claim class.

Usage: python docgate.py --base origin/main --map docs/claimmap.yaml
Exit codes: 0 routed, 1 blocked, 2 config error.
"""
from __future__ import annotations

import argparse
import re
import subprocess
import sys
from pathlib import Path

import yaml  # pip install pyyaml

HEADING = re.compile(r"^(#{1,6})\s+(.+?)\s*$")
HUNK = re.compile(r"^@@ -\d+(?:,\d+)? \+(\d+)(?:,(\d+))? @@")
NON_SLUG = re.compile(r"[^a-z0-9]+")


def slug(text: str) -> str:
    text = re.sub(r"[`*\[\]()]", "", text).lower()
    return NON_SLUG.sub("-", text).strip("-")


def heading_regions(path: Path):
    """Return (slug, start_line, end_line) for every heading, 1-based."""
    lines = path.read_text(encoding="utf-8").splitlines()
    heads = []
    for i, line in enumerate(lines, start=1):
        m = HEADING.match(line)
        if m:
            heads.append((i, len(m.group(1)), slug(m.group(2))))
    out = []
    for idx, (line_no, level, name) in enumerate(heads):
        end = len(lines)
        for nxt_line, nxt_level, _ in heads[idx + 1:]:
            if nxt_level <= level:
                end = nxt_line - 1
                break
        out.append((name, line_no, end))
    return out


def changed_ranges(base: str, files: list[str]):
    """Map new-file line ranges per path, using zero-context hunks."""
    diff = subprocess.run(
        ["git", "diff", "--unified=0", "--no-color", base, "--", *files],
        capture_output=True, text=True, check=True,
    ).stdout
    current, ranges = None, {}
    for line in diff.splitlines():
        if line.startswith("+++ b/"):
            current = line[6:]
            ranges.setdefault(current, [])
        elif line.startswith("@@") and current:
            m = HUNK.match(line)
            if m:
                start = int(m.group(1))
                count = int(m.group(2) or 1)
                if count:  # deletion-only hunks add no reviewable lines
                    ranges[current].append((start, start + count - 1))
    return ranges


def region_for(regions, line: int):
    for name, start, end in regions:
        if start <= line <= end:
            return name
    return None


def main() -> int:
    ap = argparse.ArgumentParser()
    ap.add_argument("--base", default="origin/main")
    ap.add_argument("--map", default="docs/claimmap.yaml")
    ap.add_argument("--trailer", default="Docs-Judgment")
    args = ap.parse_args()

    spec = yaml.safe_load(Path(args.map).read_text(encoding="utf-8"))
    classes = set(spec["classes"])
    sections = {s["key"]: s for s in spec["sections"]}
    for sec in spec["sections"]:
        if sec["class"] not in classes:
            sys.exit(f"config error: unknown class {sec['class']!r} in {sec['key']}")

    files = sorted({key.split("#", 1)[0] for key in sections})
    ranges = changed_ranges(args.base, files)

    message = subprocess.run(
        ["git", "log", "-1", "--pretty=%B"],
        capture_output=True, text=True, check=True,
    ).stdout
    approved = set(re.findall(rf"^{args.trailer}:\s*(\S+)", message, re.M))

    routed = {"verify-by-execution": [], "prose-review": [], "judgment": []}
    blocked: list[str] = []

    for path, hunks in ranges.items():
        regions = heading_regions(Path(path))
        for start, end in hunks:
            keys = {region_for(regions, ln) for ln in range(start, end + 1)}
            for key in sorted(k for k in keys if k):
                full = f"{path}#{key}"
                sec = sections.get(full)
                if sec is None:
                    blocked.append(f"{full}: changed but unclassified")
                    continue
                klass = sec["class"]
                if klass == "judgment":
                    sec_id = sec.get("id", full)
                    if sec_id in approved:
                        routed["judgment"].append(f"{full} (approved by trailer)")
                    else:
                        blocked.append(
                            f"{full}: judgment region changed without "
                            f"{args.trailer}: {sec_id}"
                        )
                elif klass == "executable":
                    missing = [p for p in sec.get("evidence", []) if not Path(p).exists()]
                    if missing:
                        blocked.append(f"{full}: missing evidence {missing}")
                    else:
                        routed["verify-by-execution"].append(full)
                else:
                    routed["prose-review"].append(full)

    for role, keys in routed.items():
        print(f"\n[{role}]")
        for key in keys or ["-"]:
            print(f"  {key}")
    if blocked:
        print("\n[blocked]")
        for item in blocked:
            print(f"  {item}")
        return 1
    return 0


if __name__ == "__main__":
    sys.exit(main())

A passing run prints three reviewer buckets and exits zero. A failing run prints the same buckets plus a blocked list naming the region and the reason, so the fix is visible without reading the diff again.

5. Wire the gate into the drafting loop

1. Land the evidence first. Commit the test, schema, or fixture change on its own, before any documentation commit exists.
2. Draft only the sections whose evidence moved. Give the model the evidence files and the region keys, and forbid numbers, paths, or flags that do not appear in those files.
3. Commit generated edits with the same trailer scheme your repository already uses, and never edit a judgment section in that commit.
4. Run python docgate.py --base origin/main --map docs/claimmap.yaml locally, then in CI with the same base ref to avoid branch drift.
5. Send the routing report to the review thread, so each reviewer receives only the regions assigned to their role.

Step 2 is where a hosted drafting model earns its place. According to the operator, MonkeyCode provides free model access and a free server option, which is enough to run repeated rewrite passes over a small evidence set. Disclosure: This article was prepared as part of MonkeyCode's product outreach. The gate itself has no product dependency; it is a short Python job that runs anywhere git and Python 3.11 exist.

If a reviewer genuinely intends to change a judgment section, they add the approval trailer to their own commit and re-run the gate, which converts a silent edit into a recorded decision.

6. Limitations and who should skip this

The gate checks provenance, not truth. A confident sentence inside an executable region still passes when its evidence file exists but no longer covers the claim, so evidence freshness remains a human responsibility.

• Content before the first heading has no region key and is skipped; move that material under a heading or extend the parser.
• Renames, CRLF files, and whitespace-only rewrites confuse zero-context hunk parsing on some git versions.
• One map entry per heading becomes a second artifact to maintain, and a stale map silently misroutes whole sections.
• Fully generated API references collapse into one giant executable region, where routing adds nothing over an existing schema diff.

Teams whose documentation lives in a wiki or CMS outside version control should not adopt this, because the mechanism depends on git as the record of who changed what. Single-maintainer projects with no review queue should also skip it; there is nobody to route to, and the map becomes overhead.

Start small: classify ten sections, run the gate on the next generated draft, and compare the routing report against how review actually went. If the two disagree, the map is describing your documentation rather than your review process, and the map is the part that needs editing.