Pin Each Drafted Doc Section to a Source Digest and Re-Draft Only What Moved

The documentation failures I keep running into are rarely hallucinated sentences; they are correct sentences that quietly outlive the source they paraphrase. A model drafts a rate-limit paragraph from a config file, a human approves the wording, and weeks later the constant changes with no matching doc change. Nobody re-opens the page because nobody knows the page is now wrong.

The fix is not a better prompt or a stronger model. It is a pin: every drafted section records a digest of the sources it paraphrases plus a digest of the approved prose. When CI compares those two digests against the current repository, it can tell you which sections actually moved and who has to act.

Below is a complete workflow built around a four-state pin machine, a runnable checker, and an explicit split between what a model may draft and what a human must own. The artifact is deliberately small so you can read it in one sitting and adapt the state table to your own review budget.

The review trigger is usually the wrong signal

Most teams re-review drafted docs when someone notices a problem, when a release happens, or when a quarterly audit lands. All three triggers are lagging indicators, and none of them distinguish between a section whose underlying source changed and a section that merely got reflowed by a formatter. That ambiguity is what makes drafted documentation expensive to maintain at scale.

A digest comparison gives you a leading indicator instead. If the pinned source file changed and the section text did not, the section is mechanically re-draftable, and the human only has to spot-check the result. If both the source and the prose changed, two authors have touched the same claim and a human has to reconcile them before anything ships.

That distinction is the whole value of the workflow. It converts an unbounded "review the docs" task into a bounded queue where each item carries a known owner and a known cost.

Step 1 — Declare the pins in one TOML file

Pins live in a single file at the repository root, so the mapping between a drafted section and its sources is reviewable in the same pull request as the prose. A section without a pin is not necessarily wrong, but it is unverifiable, and the workflow should treat it that way.

# docpins.toml — one block per drafted section that paraphrases a real source.
[[pin]]
doc     = "docs/api/authentication.md"
section = "Rate limits"
sources = ["openapi/auth.yaml", "src/middleware/rate_limit.ts"]

[[pin]]
doc     = "docs/api/authentication.md"
section = "Retry semantics"
sources = ["src/middleware/retry.ts"]

[[pin]]
doc     = "docs/guides/install.md"
section = "Install with uv"
sources = ["pyproject.toml"]

Two rules keep this file useful over time. First, a section may have several sources, because a single paragraph often summarizes a schema and the code that enforces it. Second, list only files the draft actually paraphrases; pinning the whole src/ tree produces constant false alarms and trains reviewers to ignore the gate.

Step 2 — Snapshot digests when a human approves the section

The lock file below is written only after a human has read the section, never by the drafting step. Treat it like a lockfile for prose: it is the record of what a person actually signed off on, not what a model produced.

#!/usr/bin/env python3
"""doc_pins.py — fail CI when a pinned source moves behind a reviewed draft.

    python3 tools/doc_pins.py            # report and exit 1 if anything moved
    python3 tools/doc_pins.py --json     # machine-readable scope for re-drafting
    python3 tools/doc_pins.py --relock   # human action after review
"""
import hashlib
import json
import pathlib
import sys

try:
    import tomllib as toml_reader          # Python 3.11+
except ModuleNotFoundError:                 # Python 3.10 and older
    import tomli as toml_reader

ROOT = pathlib.Path(__file__).resolve().parent.parent
PINS = ROOT / "docpins.toml"
LOCK = ROOT / "docpins.lock.json"


def normalize(text):
    """Drop trailing whitespace and blank-line runs so formatting churn is ignored."""
    lines = [line.rstrip() for line in text.strip().splitlines()]
    return "\n".join(line for line in lines if line)


def digest(text):
    return "sha256:" + hashlib.sha256(text.encode("utf-8")).hexdigest()[:16]


def split_sections(markdown):
    """Return {level-2 heading -> normalized body} for a Markdown file."""
    sections, heading, buffer = {}, None, []
    for line in markdown.splitlines():
        if line.startswith("## "):
            if heading is not None:
                sections[heading] = normalize("\n".join(buffer))
            heading, buffer = line[3:].strip(), []
        elif heading is not None:
            buffer.append(line)
    if heading is not None:
        sections[heading] = normalize("\n".join(buffer))
    return sections


def source_digest(paths):
    blob = []
    for relative in sorted(paths):
        blob.append(relative)
        blob.append(normalize((ROOT / relative).read_text("utf-8")))
    return digest("\n".join(blob))


def classify(moved_source, moved_doc):
    if moved_source and moved_doc:
        return "stale-review"    # two authors touched one claim: human reconciles
    if moved_source:
        return "stale-draft"     # only the source moved: safe to re-draft
    if moved_doc:
        return "unpinned-edit"   # prose edited without moving the pin
    return "valid"

The normalize function matters more than it looks. Without it, a formatter run that strips trailing spaces marks every section dirty, and a gate that fires on every commit gets disabled within a week.

Step 3 — Compute the four-state table in CI

The main loop reads the lock file, hashes the current state of both sides, and classifies each section. Missing sections are a hard error rather than a warning, because a renamed heading silently detaches the pin from the prose.

def main(argv):
    with PINS.open("rb") as handle:
        pins = toml_reader.load(handle)["pin"]

    lock = json.loads(LOCK.read_text("utf-8")) if LOCK.exists() else {}
    rows, fresh = [], {}

    for pin in pins:
        key = f"{pin['doc']}#{pin['section']}"
        body = split_sections((ROOT / pin["doc"]).read_text("utf-8")).get(pin["section"])
        if body is None:
            print(f"missing section: {key}", file=sys.stderr)
            return 2

        current_source = source_digest(pin["sources"])
        current_doc = digest(body)
        recorded = lock.get(key, {})

        state = classify(
            moved_source=current_source != recorded.get("source_digest"),
            moved_doc=current_doc != recorded.get("section_digest"),
        )
        rows.append({"key": key, "state": state})
        fresh[key] = {"source_digest": current_source, "section_digest": current_doc}

    if "--relock" in argv:
        LOCK.write_text(json.dumps(fresh, indent=2, sort_keys=True) + "\n", "utf-8")
        print(f"relocked {len(rows)} sections")
        return 0

    if "--json" in argv:
        print(json.dumps(rows, indent=2))
    else:
        width = max(len(row["key"]) for row in rows)
        for row in rows:
            print(f"{row['key']:<{width}}  {row['state']}")

    failures = sum(row["state"] != "valid" for row in rows)
    print(f"\n{failures} of {len(rows)} sections need attention")
    return 1 if failures else 0


if __name__ == "__main__":
    sys.exit(main(sys.argv[1:]))

On a repository with no lock file yet, every section classifies as stale-review, which is the safe default. The first human pass establishes the baseline with --relock, and normal operation starts on the next commit.

$ python3 tools/doc_pins.py
docs/api/authentication.md#Rate limits      valid
docs/api/authentication.md#Retry semantics  stale-draft
docs/guides/install.md#Install with uv      unpinned-edit

2 of 3 sections need attention
$ echo $?
1

The state names are the API of this tool. Everything downstream keys off them, so pick names you can explain to a reviewer in one sentence each.

State	What moved	Who acts	Typical budget	Auto-merge
valid	nothing	nobody	0 min	n/a
stale-draft	source only	model re-draft, then spot-check	5 min	no
stale-review	source and prose	section owner	20–30 min	no
unpinned-edit	prose only	reviewer plus pin update	10 min	never

Step 4 — Re-draft the cheap state, review the expensive one

Only stale-draft sections enter the automatic queue, and the queue is derived from the tool's own JSON output rather than from a hand-maintained list. That constraint is what stops the automation from quietly rewriting paragraphs a human already argued about.

# 1. Scope: sections where only the pinned source moved.
python3 tools/doc_pins.py --json \
  | jq -r '.[] | select(.state == "stale-draft") | .key' \
  > /tmp/redraft.txt

# 2. Re-draft exactly those sections, passing the pinned sources as context.
while read -r key; do
  doc="${key%%#*}"; section="${key##*#}"
  python3 tools/redraft.py --doc "$doc" --section "$section" \
      --sources-from docpins.toml --out "${doc%.md}.redraft.md"
done < /tmp/redraft.txt

# 3. A human reads the diff, then re-locks so the next run starts clean.
git diff --stat
python3 tools/doc_pins.py --relock

This is where MonkeyCode's free model tier is genuinely useful rather than decorative: re-drafting a bounded, already-verified section is exactly the kind of low-stakes call you want to run on free model access instead of spending paid capacity. As of this writing the operator states that MonkeyCode offers a free model tier and a free server option, so the nightly --json scan and the re-draft pass can run on that server without provisioning a separate paid runner. I have not benchmarked either offering, and I make no claims about quotas, model names, or throughput; verify the current terms yourself before depending on them.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The free server is also the natural home for the scheduled scan, because the checker is read-only and cheap. Run it nightly against main, post the non-valid rows to the owning team's channel, and let the count per state be your maintenance metric.

What the model may draft, and what a human has to own

The split below is the part that survives a model upgrade, a prompt rewrite, or a change of vendor. Drafting rights are attached to claim types, not to sections or files.

Claim type	Model may draft	Human must own
Prose transitions and summaries	yes	no
Request and response examples derived from a pinned spec	yes, then checked	final approval
Numeric limits, defaults, and timeouts	no	always
Error semantics, status codes, retry rules	no	always
Security and compliance statements	no	always
Deprecation and version timelines	no	always

The rows in the lower half are the ones that get a team into trouble in public, because they look like ordinary prose and read perfectly well when they are wrong. A drafted sentence saying a token expires in an hour is indistinguishable, on the page, from a verified one.

Reproduce this on a throwaway branch

1. Create a branch and add three Markdown files with level-2 headings, plus one config file the prose paraphrases.
2. Write docpins.toml with three pins, then run the checker once to confirm every section reports stale-review.
3. Run --relock, commit both files, and confirm a clean run reports zero non-valid sections.
4. Edit only the config file, and confirm the pinned section flips to stale-draft while its neighbors stay valid.
5. Edit only a section body without touching its source, and confirm it reports unpinned-edit rather than passing silently.

Step five is the one worth demonstrating to your team, because it is the case a naive freshness check misses entirely. A section whose prose was edited outside the pin is the most likely place for an unverified claim to enter the repository.

Limitations and who should skip this

Digests are line-sensitive, so a reformat of a pinned source file creates a false stale-draft even when the meaning is unchanged. Semantic drift in the opposite direction is worse: a source can change meaning while producing a coincidentally identical digest only in theory, but a source can change meaning and still produce a small, easily missed diff in practice. The gate tells you where to look, not what the answer is.

Sections with no single upstream source cannot be pinned, and forcing a pin onto an editorial page produces a lock file that nobody trusts. Single-author documentation sites and repositories that generate reference pages entirely at build time from a schema should skip this; there is no prose to expire, and the added file is pure overhead.

Teams whose pinned sources change hourly should also reconsider the granularity. In that case, pin the schema rather than the implementation file, and accept that the drafted paragraph will sometimes describe last week's behavior.

If you try this, the number worth reporting back is the share of sections sitting in stale-draft on a typical week. That ratio tells you how much of your documentation maintenance is genuinely mechanical, and how much of it still needs a person who knows why the original sentence was written that way.