Doc Drift Is Measurable: A Git-Based Index That Tells the Model What to Draft

Documentation rot is a measurement problem before it is a writing problem. Most teams notice stale docs after a support ticket, not after a release candidate. A simple git-based drift index can rank every code file by how far its documentation lags behind its commits, and that ranking becomes a triage queue for model-generated drafts. The workflow below builds that index, schedules it on a free tier, and preserves a strict ownership border between what the model may draft and what a human must approve.

This article targets a failure pattern that most teams recognize instantly: a module changed twelve times, its doc changed zero times, and nobody noticed until a customer read the outdated sentence aloud. Style guides, reminders, and documentation sprints do not fix that pattern, because none of them measure it. The alternative is a number per file, computed from history, that makes the backlog visible, the draft work bounded, and the review gate explicit. The scanner itself is provider-agnostic; only the drafting step chooses a model provider, and that step stays replaceable.

Why Commit History Predicts Rot

Two signals from commit history predict documentation rot better than intuition does. The first is asymmetry, the gap between how often code moved and how often its paired doc moved. The second is recency, whether any doc update landed inside the analysis window at all. The drift score compresses both signals into one number:

drift = code_commits / (1 + doc_commits)

A score of 6.0 means six code changes for every doc change in the window, and a score below 1.0 means the docs kept pace with the code. A file with no paired doc keeps a raw commit count as its score, which ranks it above any documented file with the same activity. The default window of thirty days keeps the signal fresh and the git log fast.

The Drift Index Script

The scanner is deliberately small because it should run anywhere a repository is cloned. It lists tracked files, counts commits per path inside the window, and writes both a human-readable table and a machine-readable queue. Keep it in the repository root with a docs directory that mirrors source paths.

#!/usr/bin/env python3
"""doc_drift.py — rank code files by documentation drift.

The drift score is code_commits / (1 + doc_commits) inside the window.
"""
import argparse
import json
import subprocess
from datetime import datetime, timedelta, timezone

CODE_EXTS = {".py", ".js", ".ts", ".go", ".rs", ".java", ".c", ".cpp", ".h"}
CODE_SUFFIXES = {ext.lstrip(".") for ext in CODE_EXTS}


def git(repo, *args):
    return subprocess.run(
        ["git", "-C", repo, *args],
        capture_output=True, text=True, check=True,
    ).stdout


def commit_count(repo, path, since):
    log = git(repo, "log", "--oneline", f"--since={since}", "--", path)
    return len(log.splitlines())


def main():
    ap = argparse.ArgumentParser(description=__doc__)
    ap.add_argument("--repo", default=".")
    ap.add_argument("--docs-dir", default="docs")
    ap.add_argument("--window-days", type=int, default=30)
    ap.add_argument("--min-commits", type=int, default=2)
    ap.add_argument("--top", type=int, default=10)
    args = ap.parse_args()

    since = (datetime.now(timezone.utc) - timedelta(days=args.window_days)).isoformat()
    tracked = set(git(args.repo, "ls-files").splitlines())

    rows = []
    for path in sorted(tracked):
        if path.rsplit(".", 1)[-1] not in CODE_SUFFIXES:
            continue

        code_hits = commit_count(args.repo, path, since)
        if code_hits < args.min_commits:
            continue

        stem = path.rsplit(".", 1)[0]
        candidates = [
            f"{args.docs_dir}/{stem}.md",
            f"{args.docs_dir}/{stem}.rst",
            f"{args.docs_dir}/{stem}.adoc",
        ]
        existing = [d for d in candidates if d in tracked]
        doc_hits = max((commit_count(args.repo, d, since) for d in existing), default=0)

        rows.append({
            "file": path,
            "code_commits": code_hits,
            "doc_commits": doc_hits,
            "drift": round(code_hits / (1 + doc_hits), 2),
            "has_doc": bool(existing),
        })

    rows.sort(key=lambda r: r["drift"], reverse=True)
    top = rows[: args.top]

    print("drift  code  doc  file")
    for row in top:
        flag = "" if row["has_doc"] else "  (no doc)"
        print(f"{row['drift']:5.2f}  {row['code_commits']:4d}  "
              f"{row['doc_commits']:4d}  {row['file']}{flag}")

    with open("draft_queue.json", "w") as fh:
        json.dump(top, fh, indent=2)
    print(f"\nWrote {len(top)} entries to draft_queue.json")


if __name__ == "__main__":
    main()

Run it from the repository root:

python3 doc_drift.py --repo . --window-days 30 --top 8

Sample output:

drift  code  doc  file
12.00    12    0  src/pricing/engine.go   (no doc)
 4.50     9    1  src/api/client.py
 3.00     6    1  src/workers/sync.py
 1.00     4    3  src/auth/session.py

Wrote 8 entries to draft_queue.json

The report answers one question per file: did the code outrun its documentation this month? The JSON queue then feeds the drafting step, so the model receives numbers instead of vibes.

From Queue to a Constrained Draft Prompt

Raw drift numbers become useful only when they produce an actionable artifact. The artifact here is a prompt with hard output constraints, built from the queue by a small converter. The model may draft exactly two things per file and nothing else.

#!/usr/bin/env python3
"""queue_to_prompt.py — turn draft_queue.json into a constrained prompt."""
import json

with open("draft_queue.json") as fh:
    queue = json.load(fh)

lines = [
    f"- {row['file']} (drift {row['drift']}, {row['code_commits']} code commits)"
    for row in queue
]

prompt = f"""You are drafting release notes from a documentation backlog.
For each file, draft only:
1. One sentence describing the behavioral change in the recent commits.
2. One verification step a reader can run to confirm the behavior.
Do not rewrite API contracts, invariants, security properties, or
performance promises. Those stay with the human maintainer.

Backlog:
{chr(10).join(lines)}
"""

print(prompt)

The prompt forbids rewriting contracts, invariants, security properties, and performance promises. That constraint is the whole point: the draft stays inside the trust boundary because the prompt refuses to cross it. The prompt is inspectable before any model call, which makes the drafting step auditable instead of magical.

What the Model May Draft, What You Must Own

A written table makes the ownership border enforceable instead of aspirational:

Draft layer	Model may draft	Human must own
Behavior notes	Summaries extracted from the merged diff	Contract semantics and API guarantees
Migration steps	Concrete commands drawn from merged code	Anything that touches production data
Examples	Snippets copied from code that already merged	Security-sensitive or quantified claims
Deprecations	Warnings derived from changelog entries	Compliance, legal, or SLA statements

The drift queue labels each entry with its module, and the draft pull request cannot merge until the named owner verifies the two drafted items. Automation gathers the evidence, the model drafts the noise, and the human owns the claims. That division is the measurable version of an old rule: the machine may propose, but the maintainer asserts.

A Weekly Pipeline on a Free Tier

The pipeline needs a periodic trigger instead of a laptop left open on a cron tab. GitHub Actions with a schedule is the zero-setup path, and the same job runs on any cron-capable host. MonkeyCode's free server option is one such host, and its free model access with an advertised 10-million-token allowance covers the drafting step without a paid runner. Disclosure: This article was prepared as part of MonkeyCode's product outreach. At the prompt size above, that allowance stretches across thousands of runs, so a small team's weekly drafting volume stays comfortably inside it.

name: doc-drift-draft

on:
  schedule:
    - cron: "17 3 * * 1"   # Mondays at 03:17 UTC
  workflow_dispatch:

permissions:
  contents: write
  pull-requests: write

jobs:
  scan-and-draft:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0

      - name: Measure drift
        run: python3 doc_drift.py --window-days 30 --top 8

      - name: Build the constrained prompt
        run: python3 queue_to_prompt.py > draft_prompt.txt

      - name: Draft via the configured model provider
        env:
          MODEL_ENDPOINT: ${{ secrets.MODEL_ENDPOINT }}
          MODEL_TOKEN: ${{ secrets.MODEL_TOKEN }}
        run: |
          # Thin adapter: replace with the command your provider documents.
          ./send_draft.sh draft_prompt.txt > draft_notes.md

      - name: Open the draft PR for the human gate
        env:
          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
        run: |
          git config user.name "github-actions[bot]"
          git config user.email "github-actions[bot]@users.noreply.github.com"
          if git diff --quiet -- draft_queue.json draft_notes.md; then
            echo "No drift changes this week."
            exit 0
          fi
          git checkout -b docs/drift-draft
          git add draft_queue.json draft_notes.md
          git commit -m "docs: draft notes from drift queue"
          git push -u origin docs/drift-draft
          gh pr create --title "docs: draft notes from drift queue" --body-file draft_notes.md

The workflow keeps the provider call behind an adapter so changing models never touches the drift logic. The only placeholder is send_draft.sh, and you fill it with whatever command your provider documents. The hypothesis-testing loop stays intact: measure, draft, review, merge.

The Human Gate Is a Merge Condition

Automation ends at the pull request, and that is intentional. The draft PR opens with a checklist that names the file, the owner, and the assertions the owner must verify. A tiny template keeps the gate readable:

## Drift draft for <module>

- [ ] The behavior sentence matches the merged diff, not the model's guess.
- [ ] The verification step runs in a clean checkout of this revision.
- [ ] No contract, invariant, or security statement was reworded.
- [ ] The diff is small enough to read in five minutes.

Teams that skip the gate get faster drafts and slower trust, because a merged model sentence becomes an unstated contract. Teams that keep the gate get a measurable loop: drift scores fall, doc commits rise, and the queue shrinks across consecutive weeks. The gate converts the drift index from a report into a policy.

Limitations and Who Should Skip This

The drift index is a proxy, not a truth meter. A file with perfect scores can still contain wrong examples, and a file with high drift can be intentionally stable and simply undocumented. The per-file git log calls also get slow beyond several thousand files, so large repositories should parse one git log --numstat dump instead.

Teams with a single source of truth that generates docs should skip this workflow entirely, because drift is meaningless when docs are build artifacts. Teams without a named owner per module will also struggle, since the queue assigns responsibility and a script cannot create it. Use this index when a human already owns each contract and wants a data-driven backlog, not when the goal is to automate away review.

If you want to benchmark the drafting step against your own backlog, the free tier above is a low-risk place to start. The durable artifact is the drift index, and the model stays replaceable.