Provenance Gates for Docs: Fail the Build When a Model Last Touched a Human-Owned Span

Content classification tells you which paragraphs a model may draft; git provenance tells you who is accountable for the paragraph that actually shipped. This post adds a second, cheaper guard to a generated-docs pipeline: a check that fails when a non-human commit last touched a span you marked as human-owned. The span table and the script below are a proposed design written for this article rather than a system I have operated in production, so treat the code as a reference implementation and run it on a scratch branch before you trust it.

1. The failure mode: a well-formed sentence replaces a reviewed one

Generated documentation fails loudly when it invents a parameter name and quietly when it weakens a consequence. A bulk regeneration pass can swap a reviewed deprecation warning for a fluent paraphrase that reads better, keeps the same heading, and promises less than the original. Content-level review rarely catches this, because the replacement is grammatical, on-topic, and consistent with every neighbouring paragraph. Provenance catches it for almost free: the reviewed sentence was last modified by a named human, and the replacement was not.

The two checks answer different questions. Claim classification answers "is this sentence the kind of claim a model may write?" Provenance answers "did anyone with a name agree to this exact wording?" You want both, but provenance needs far less maintenance, because it depends on commit metadata rather than on a taxonomy that drifts every release.

2. Split spans by consequence, not by topic

Topic-based splits fail because the same subject can be harmless in one place and contractual in another. "Requests time out after 30 seconds" is a reference detail in a parameter table and a support commitment in a limits page. Classify by what a wrong sentence costs the reader: money, data, or trust.

Span	Model may draft	Commit identity required	Reason
Quickstart commands, parameter tables	Yes	Any	Mechanically checkable against source
Reference tables compiled from schema or enums	Yes, via generation	Any	Derivable and diffable
Migration and upgrade steps	Draft, human rewrites	human/*	Wrong ordering destroys data
Breaking-change and deprecation notices	No	human/*	Dated commitments to users
Limits, quotas, retention statements	No	human/*	Product and legal claims that shift outside the repo
Security and data-handling text	No	human/*	Regulated language with external reviewers
Troubleshooting workarounds	Draft	human/*	Describes undocumented behaviour with support cost

The working rule is short: a span is human-owned when being wrong costs a reader something irreversible, not when it is difficult to write.

3. Mark human-owned spans inside the source file

1. Choose HTML comment markers that survive your Markdown renderer, for example <!-- own:human --> and <!-- /own:human -->.
2. Wrap only the sentences that carry the commitment, not the whole page, so the gate stays cheap and the diff stays reviewable.
3. Never nest spans; an unclosed marker should be a hard error, because a silent misparse turns the gate into decoration.

## Upgrading from 2.x

<!-- own:human -->
Upgrading skips queued jobs unless `--drain` is passed. Run the drain step before
rotating credentials, otherwise in-flight jobs are dropped without a retry record.
<!-- /own:human -->

Run `migrate --dry-run` first to see the plan.

4. Gate the spans on commit trailers

1. Make every drafting commit carry an explicit identity in a trailer, for example Generated-By: monkeycode-free/draft-4127 for a model-drafted pass and Generated-By: human/avery-lin for a person.
2. Resolve the last commit for each line of a marked span with git blame --line-porcelain.
3. Reject any human-owned span whose last non-unknown commit does not carry a human/* trailer.
4. Run in report-only mode first, printing findings while always exiting zero, then flip to enforcing.

If you draft reference prose with MonkeyCode's free model access, the drafting commit carries a non-human identity, and the promotion commit is where a person takes ownership. Disclosure: This article was prepared as part of MonkeyCode's product outreach.

#!/usr/bin/env python3
"""provenance_gate.py — fail when a non-human commit last touched an own:human span."""
from __future__ import annotations
import argparse, re, subprocess, sys

OPEN_MARK, CLOSE_MARK = "<!-- own:human -->", "<!-- /own:human -->"
BLAME_SHA = re.compile(r"^([0-9a-f]{40})\s", re.M)


def git(*args: str) -> str:
    return subprocess.run(("git", *args), check=True, capture_output=True, text=True).stdout


def spans(text: str) -> list[tuple[int, int]]:
    found, open_at = [], None
    for lineno, line in enumerate(text.splitlines(), start=1):
        if OPEN_MARK in line:
            open_at = lineno + 1
        elif CLOSE_MARK in line and open_at is not None:
            found.append((open_at, lineno - 1))
            open_at = None
    if open_at is not None:
        raise ValueError("unclosed own:human span")
    return found


def trailer_identity(sha: str) -> str:
    value = git("log", "-1", "--format=%(trailers:key=Generated-By,valueonly)", sha).strip()
    return value or "unknown"


def check(path: str, allow_unknown: bool) -> list[str]:
    findings: list[str] = []
    with open(path, encoding="utf-8") as handle:
        marked = spans(handle.read())
    for first, last in marked:
        porcelain = git("blame", "--line-porcelain", "-L", f"{first},{last}", "--", path)
        for sha in sorted(set(BLAME_SHA.findall(porcelain))):
            who = trailer_identity(sha)
            if who == "unknown" and allow_unknown:
                continue
            if not who.startswith("human/"):
                findings.append(f"{path}:{first}-{last} last touched by {who} ({sha[:12]})")
    return findings


def main() -> int:
    parser = argparse.ArgumentParser()
    parser.add_argument("paths", nargs="+")
    parser.add_argument("--strict", action="store_true",
                        help="also fail on commits with no Generated-By trailer")
    args = parser.parse_args()

    findings = [item for path in args.paths
                for item in check(path, allow_unknown=not args.strict)]
    if findings:
        print("provenance gate failed:\n" + "\n".join(findings), file=sys.stderr)
        return 1
    print(f"provenance gate passed for {len(args.paths)} file(s)")
    return 0


if __name__ == "__main__":
    raise SystemExit(main())

A failing run looks like this, and the message is deliberately specific enough to send back to the drafting session.

provenance gate failed:
docs/upgrade.md:41-43 last touched by monkeycode-free/draft-4127 (9c1f0ab3d2e4)

5. Run the gate where the history is complete

1. Fetch full history in CI, because git blame on a shallow clone silently misattributes lines to the boundary commit.
2. Run the gate only on documentation files changed in the pull request, which keeps the blame cost proportional to the diff.
3. Keep the job independent of your build job, so a docs-provenance failure never blocks an unrelated deploy.
4. Treat exit code 1 as a review request, not as a verdict on the text.

The gate is cheap glue: roughly fifty lines of Python and one history fetch. If you want it to run next to a docs preview rather than inside a heavy build matrix, MonkeyCode's free server option is a plausible host for exactly this kind of small always-on check, though you should confirm current free-model and free-server terms in its documentation before designing a pipeline around them, since availability and limits change.

# illustrative step; adapt the runner to your platform
steps:
  - run: git fetch --unshallow || true
  - run: python3 tools/provenance_gate.py docs/upgrade.md docs/limits.md --strict

6. Known limits and false signals

The gate proves that a person pressed commit on a span, not that a person wrote or understood it. A reviewer who commits model text verbatim passes the check while adding no accountability, and no commit metadata can detect that. Formatting commits from Prettier or a Markdown linter rewrite blame attribution, so run them through git blame --ignore-revs-file or keep them out of documentation directories entirely.

History rewriting is the second weak point: squash merges, rebases, and cherry-picks can move a span's last-touch commit onto a bot identity that carries no trailer. Trailers are also forgeable by anyone with commit access, so the gate enforces team discipline rather than cryptographic authorship. Expect to widen --strict gradually instead of switching it on for a decade of existing history.

Skip this approach if documentation is fully generated from source with no human prose, if a single maintainer owns the whole repo and trailers are pure ceremony, or if your docs live outside version control. The gate earns its keep only where at least two people write the same pages over time.

7. A rollout order that stays small

1. Parse the markers in report-only mode for one release cycle and count how many spans you actually marked.
2. Freeze the highest-consequence spans first: breaking changes, limits, and security text.
3. Add the Generated-By trailer to your drafting workflow before you enforce anything, or every historical span will fail at once.
4. Enforce on changed files in pull requests, and revisit the span table each quarter, since spans that stop carrying risk should stop blocking merges.

Ownership of generated documentation is a policy question that a script can only enforce, and the script is the easy half. If you build the span table before the gate, you will already know which sentences a model should never be allowed to touch on its own, which is the part that survives tooling changes.