A Drift Budget for Model-Drafted Docs: Re-Verify Only the Sections That Changed

Documentation drifts because a draft records the wording it produced, but never the source that made that wording true. The workflow below binds every drafted section to the exact source blobs it restates, then re-verifies only the sections whose sources actually moved. That converts an unbounded review queue into a per-section verdict: redraft automatically, or hold for a human owner.

Why full regeneration is a weak review strategy

When a documentation job regenerates every page on a schedule, the reviewer receives a diff large enough that skimming becomes the only realistic option. A byte-level diff also fails to say which claim weakened, so a renamed flag and a changed rate limit look equally suspicious. The practical result is that nobody can tell whether the docs got better or merely newer.

A drift budget replaces that single large diff with a list of sections that lost their evidence. Each entry names the section, the sources it depends on, and whether the change falls inside what a model is allowed to restate. Review effort then scales with real change rather than with page count.

The distinction matters most in repositories where docs and code ship together. If your docs live in a separate system with a separate owner, this approach still works, but the binding step becomes manual and slower to maintain.

Decide what the model may draft before generating anything

The gate is only as honest as the content classification behind it. Classify each section once, store the classification next to the section, and let the classifier decide the drift verdict instead of a reviewer's mood.

Section content	Model may draft	Human must own	Drift verdict
Flag names, defaults, exit codes copied from source	yes	nothing	redraft
CLI transcripts produced by a captured run	yes, verbatim insert	which commands get run	redraft
Rate limits, quotas, SLA numbers	no	yes	hold
Deprecation promises and dates	no	yes	hold
Migration steps with data-loss risk	first pass only	final wording and ordering	hold
Glossary and conceptual framing	yes	scope and tone	redraft

The rule underneath the table is narrow: a model may restate something a machine can check, and a human must author anything that commits the project to future behavior. Numbers, promises, and risk instructions are commitments, not descriptions.

Steps to build the drift gate

1. Bind each section to source blobs in front matter

Record the content hash of every file a section restates, and annotate the class at the same time. The hashes below are truncated for readability; generate yours with the same command the checker uses.

---
owner: docs-platform
sections:
  - id: rate-limits
    draft_mode: model
    sources:
      - path: src/limits.go
        blob: 9c1f0a3d
        class: code
      - path: docs/policy/rate-limit-policy.md
        blob: 4b7e2c81
        class: policy
---

Get a hash with git hash-object -- src/limits.go. Because it hashes the working-tree bytes, the check works before you commit, which keeps the feedback loop short.

2. Compute the current hashes and compare

The checker walks the docs tree, reads front matter, and compares recorded blobs against current ones. Sections with no sources entry are reported as ungoverned rather than clean, because silence is not evidence.

#!/usr/bin/env python3
"""Flag drafted doc sections whose bound sources drifted.

Exit codes: 0 = nothing to do, 2 = redraft allowed, 3 = human review required.
"""
from __future__ import annotations

import pathlib
import subprocess
import sys

import yaml

DOCS = pathlib.Path("docs")
MODEL_CLASSES = {"code", "cli-output"}


def blob_sha(path: str) -> str:
    out = subprocess.run(
        ["git", "hash-object", "--", path],
        check=True, capture_output=True, text=True,
    )
    return out.stdout.strip()


def read_front_matter(path: pathlib.Path) -> dict:
    text = path.read_text(encoding="utf-8")
    if not text.startswith("---\n"):
        return {}
    _, block, _ = text.split("---\n", 2)
    return yaml.safe_load(block) or {}


def verdict(stale: list[dict]) -> str:
    if not stale:
        return "clean"
    if all(s["class"] in MODEL_CLASSES for s in stale):
        return "redraft"
    return "hold"


def main() -> int:
    worst = "clean"
    for doc in sorted(DOCS.rglob("*.md")):
        for section in read_front_matter(doc).get("sections", []) or []:
            sources = section.get("sources") or []
            if not sources:
                print(f"ungoverned {doc}::{section['id']}")
                worst = "hold"
                continue
            stale = [s for s in sources if blob_sha(s["path"]) != s["blob"]]
            result = verdict(stale)
            if result != "clean":
                print(f"{result:7} {doc}::{section['id']} stale={len(stale)}/{len(sources)}")
            if result == "hold":
                worst = "hold"
            elif result == "redraft" and worst == "clean":
                worst = "redraft"
    return {"clean": 0, "redraft": 2, "hold": 3}[worst]


if __name__ == "__main__":
    sys.exit(main())

3. Map verdicts to CI behavior

Exit code 0 means nothing moved, 2 means a model may replace the prose, and 3 means the change touched a human-owned class. Keeping those three outcomes distinct prevents the common failure where a pipeline silently regenerates a section that promised an SLA.

name: docs-drift
on:
  schedule: [{cron: "17 6 * * *"}]
  workflow_dispatch:
jobs:
  drift:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: pip install pyyaml
      - id: scan
        continue-on-error: true
        run: |
          set +e
          python tools/drift_guard.py > drift.txt 2>&1
          echo "code=$?" >> "$GITHUB_OUTPUT"
          cat drift.txt
      - name: Redraft model-owned sections
        if: steps.scan.outputs.code == '2'
        run: python tools/redraft.py --plan drift.txt
      - name: Block on human-owned drift
        if: steps.scan.outputs.code == '3'
        run: exit 1

Regeneration is the one step where a hosted model earns its place. MonkeyCode's free model access drafts replacement prose for sections the gate marked redraft, and its free server option can run the scheduled scan so the job does not depend on a laptop staying awake. Both of those are availability claims supplied by the operator rather than verified here, so check the current terms and limits before wiring them into a required check. Disclosure: This article was prepared as part of MonkeyCode's product outreach.

4. Verify the gate itself with a falsification test

The checker is only useful if it fails in the right places. Run these three commands on a scratch branch and confirm the exit codes match the table above.

# 1. baseline: expect clean, exit 0
python tools/drift_guard.py; echo $?

# 2. touch a source bound as class: code -> expect redraft, exit 2
printf '\n// touched\n' >> src/limits.go
python tools/drift_guard.py; echo $?

# 3. touch a policy source -> expect hold, exit 3
printf '\n' >> docs/policy/rate-limit-policy.md
python tools/drift_guard.py; echo $?

After a redraft, commit the refreshed blob hashes in the same change as the new prose, so the section returns to clean only when its evidence was re-read. Record how many sections land in each verdict every week; without that record the drift budget stays a guess.

Limitations and honest failure modes

A blob hash covers file bytes, not behavior, so a reformat triggers a false redraft while a semantic change elsewhere stays invisible. Renames and moved files break bindings silently, which is why ungoverned sections must never default to clean. The gate also cannot judge whether prose is correct, only whether its evidence moved.

Anything a model rewrites still needs the owning engineer to read it before merge. The verdict is a routing decision, not an approval, and treating exit code 2 as automatic publication defeats the entire design.

Who should not use this

Skip this approach when documentation lives outside the repository, because manual bindings decay faster than they help. Skip it when compliance or legal review owns every sentence, since the redraft path is then unusable and the gate only adds noise. Skip it for single-file READMEs, where front matter costs more than the drift it prevents.

Start with one page that restates a real flag or limit, bind it, and run the falsification test before rolling the gate out. If the verdicts match what a careful reviewer would have said anyway, you have a check worth scheduling; the scheduled variant is the one I would run on MonkeyCode's free server option, with the drafting step handled by its free model access.