DEV Community: Avery Lin

Extract a Command Ledger From Make and CI; Hand-Sign Every Tutorial Shell Block

Avery Lin — Mon, 21 Sep 2026 05:43:22 +0000

Getting-started pages should compile from commands the repository already executes, not from invented install stories. Tutorial prose may be drafted by a model, but fenced shell blocks are not draftable text. Any command that asserts an install path, a version pin, or a network side effect needs a human signature before publish. The workflow extracts a command ledger from Make, package scripts, and CI YAML, then fails unsigned README blocks.

This article is a proposed toolchain, not a field report from a named production org. The scripts below are labeled examples you must review, adapt, and run in your own repository. They do not claim coverage of every Make dialect, every CI vendor, or every shell quoting rule.

Why tutorials drift even when prose looks fine

Most tutorial drift is a command-set problem rather than a grammar problem. A README still tells readers to run pip install . while CI has used a lockfile installer for months. Generated drafts amplify that gap because they complete a familiar narrative instead of quoting automation files that already exist.

A useful split is therefore mechanical rather than stylistic. Extractable facts live in Makefiles, package.json scripts, and CI run: steps. Draftable glue is the connecting paragraph that explains order, purpose, and expected output shape. Owned claims are version pins, supported OS lists, secret handling, and any command that reaches a network.

If a sentence cannot be traced to an extracted argv array or to a named human signature, it does not belong in a getting-started page. That rule is stricter than ordinary editorial review, and that strictness is the point. Tutorials fail in install steps long before they fail in tone.

Three claim classes for getting-started pages

Treat every tutorial unit as one of three classes before a model is allowed to write anything.

Extract — target names, script keys, and argv arrays observed in Make, npm, or CI.
Draft — section order, motivation sentences, and warnings that do not assert a runtime.
Sign — every fenced bash or sh block, every version pin, and every “works on” matrix cell.

The ledger is the extract class serialized to JSON. The model may propose draft-class paragraphs that mention ledger ids. The human signs sign-class blocks by attaching a reviewer id and a UTC date, never by asking the model to initial them.

Artifact: a command ledger plus two checks

The original artifact is a JSON ledger with a deterministic extractor and a README linter. Keep the ledger generated. Hand-edit only a sibling signatures file that maps block hashes to reviewers.

Proposed ledger shape:

{
  "generated_at": "REPLACE_WITH_CI_CLOCK",
  "sources": ["Makefile", "package.json", ".github/workflows/ci.yml"],
  "commands": [
    {
      "id": "make.test",
      "origin": "Makefile:12",
      "argv": ["pytest", "-q"],
      "network": false
    }
  ]
}

Proposed signatures shape, which is never generated by a model:

{
  "blocks": [
    {
      "sha256": "REPLACE_AFTER_HASHING_THE_FENCE_BODY",
      "ledger_id": "make.test",
      "reviewer": "REPLACE_WITH_HUMAN_ID",
      "signed_at": "REPLACE_WITH_ISO8601",
      "allows_network": false
    }
  ]
}

The two files together are the publish gate. CI regenerates the ledger, hashes each README fence, and rejects unknown argv, missing signatures, or network flags that the signer did not allow.

Step 1 — Extract commands with a conservative parser

The extractor below is a heuristic, not a Make implementation. It records recipe lines that look like argv, package.json script strings, and GitHub Actions run: lines. It refuses Make functions, shell control flow, and interpolated CI expressions, because those are not tutorial commands.

# extract_command_ledger.py — example, review before use
from __future__ import annotations

import json, re, sys
from pathlib import Path

MAKE_TARGET = re.compile(r"^([A-Za-z0-9][^:#=\n]*):")
SAFE_ARGV = re.compile(r"^[A-Za-z0-9./_=+-]+")
INTERP = re.compile(r"\$\(|`|\$\{\{")

def argv_of(line: str) -> list[str] | None:
    line = line.strip()
    if not line or line.startswith("#") or INTERP.search(line):
        return None
    if line.startswith("@"):
        line = line[1:]
    parts = line.split()
    if not parts or not SAFE_ARGV.match(parts[0]):
        return None
    if any(p in {"&&", "||", "|", ";", "if", "for"} for p in parts):
        return None
    return parts

def from_makefile(path: Path) -> list[dict]:
    rows, current = [], None
    for i, raw in enumerate(path.read_text(encoding="utf-8").splitlines(), 1):
        m = MAKE_TARGET.match(raw)
        if m and not raw.startswith("\t"):
            current = m.group(1).split()[0]
            continue
        if current and raw.startswith("\t"):
            argv = argv_of(raw[1:])
            if argv:
                rows.append({
                    "id": f"make.{current}",
                    "origin": f"{path}:{i}",
                    "argv": argv,
                    "network": argv[0] in {"curl", "wget", "npm", "pip", "docker"},
                })
    return rows

def from_package_json(path: Path) -> list[dict]:
    data = json.loads(path.read_text(encoding="utf-8"))
    rows = []
    for name, cmd in (data.get("scripts") or {}).items():
        argv = argv_of(cmd)
        if argv:
            rows.append({
                "id": f"npm.{name}",
                "origin": f"{path}:scripts.{name}",
                "argv": argv,
                "network": argv[0] in {"npm", "npx", "yarn", "pnpm"},
            })
    return rows

def from_gha(path: Path) -> list[dict]:
    rows, in_run, buf, start = [], False, [], 0
    for i, raw in enumerate(path.read_text(encoding="utf-8").splitlines(), 1):
        if re.match(r"^\s*run:\s*\|", raw):
            in_run, buf, start = True, [], i
            continue
        if in_run and re.match(r"^\s{4,}\S", raw):
            buf.append(raw.strip())
            continue
        if in_run:
            argv = argv_of(" ".join(buf))
            if argv:
                rows.append({
                    "id": f"gha.{path.stem}.{start}",
                    "origin": f"{path}:{start}",
                    "argv": argv,
                    "network": argv[0] in {"curl", "wget", "npm", "pip", "docker"},
                })
            in_run = False
    return rows

def main() -> None:
    root = Path(sys.argv[1] if len(sys.argv) > 1 else ".")
    rows = []
    for mf in root.glob("Makefile"):
        rows.extend(from_makefile(mf))
    pkg = root / "package.json"
    if pkg.exists():
        rows.extend(from_package_json(pkg))
    for wf in (root / ".github" / "workflows").glob("*.yml"):
        rows.extend(from_gha(wf))
    sys.stdout.write(json.dumps({"sources": sorted({r["origin"].split(":")[0] for r in rows}), "commands": rows}, indent=2))
    sys.stdout.write("\n")

if __name__ == "__main__":
    main()

Run it as a proposed local check, then inspect the JSON before any draft step.

python extract_command_ledger.py . > commands.ledger.json
python -m json.tool commands.ledger.json >/dev/null

Commands rejected by argv_of are a feature. A tutorial that needs curl | sh or a Make function is a sign-class problem, not an extract-class problem, and must not enter the ledger automatically.

Step 2 — Lint README fences against the ledger

The linter hashes each fenced shell body, requires a signature row, and checks that the first argv token matches a ledger command. It also fails when a signed block sets allows_network false while the ledger marked the command as network-touching.

# lint_tutorial_fences.py — example, review before use
from __future__ import annotations

import hashlib, json, re, sys
from pathlib import Path

FENCE = re.compile(r"```

(?:bash|sh)\n(.*?)\n

```", re.S)

def norm(body: str) -> str:
    lines = [ln.rstrip() for ln in body.splitlines() if ln.strip() and not ln.strip().startswith("#")]
    return "\n".join(lines)

def main() -> int:
    readme = Path(sys.argv[1]).read_text(encoding="utf-8")
    ledger = {tuple(c["argv"]): c for c in json.loads(Path(sys.argv[2]).read_text())["commands"]}
    sigs = {b["sha256"]: b for b in json.loads(Path(sys.argv[3]).read_text())["blocks"]}
    failed = 0
    for body in FENCE.findall(readme):
        text = norm(body)
        digest = hashlib.sha256(text.encode()).hexdigest()
        first = text.split()[0] if text else ""
        match = next((c for argv, c in ledger.items() if argv and argv[0] == first), None)
        sig = sigs.get(digest)
        if match is None:
            print(f"UNKNOWN_ARGV {first!r} hash={digest[:12]}")
            failed += 1
            continue
        if sig is None:
            print(f"UNSIGNED_BLOCK ledger_id={match['id']} hash={digest[:12]}")
            failed += 1
            continue
        if match["network"] and not sig.get("allows_network"):
            print(f"NETWORK_NOT_ALLOWED ledger_id={match['id']}")
            failed += 1
    return 1 if failed else 0

if __name__ == "__main__":
    raise SystemExit(main())

Wire both scripts as ordinary CI steps so a regenerated ledger cannot silently accept last week’s fences.

# proposed fragment for .github/workflows/docs-ledger.yml
name: docs-ledger
on: [pull_request]
jobs:
  lint-tutorial-fences:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: "3.12"
      - run: python extract_command_ledger.py . > commands.ledger.json
      - run: python lint_tutorial_fences.py README.md commands.ledger.json tutorial.signatures.json

actions/checkout@v4 and actions/setup-python@v5 are current major tags as of this draft date, 2026-09-21. Pin commit SHAs in repositories that require immutable Actions references.

Step 3 — What a model may draft, and what it must not touch

After the ledger exists, a model may draft narrative glue that cites command ids. It may reorder sections, write motivation paragraphs, and propose headings that map onto extract-class ids. It must not emit a fenced shell block, a version pin, an OS matrix, or a promise that install succeeds on a named platform.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode’s free model access and free server option are relevant only as a place to run that draft-narrative step against the ledger JSON, without treating the model as a signer. Do not send signatures files, credentials, or unpublished install tokens into any draft prompt.

A prompt that respects the split looks like the following example. Replace the ledger excerpt with generated JSON, never with hoped-for commands.

You receive commands.ledger.json as the only source of install and test verbs.
Draft README prose in English that explains order and purpose.
Cite command ids in backticks, for example `make.test`.
Do not output fenced bash or sh blocks.
Do not invent argv, version numbers, OS support, or network flags.
If a reader action is not in the ledger, write UNVERIFIED_CLAIM and stop that sentence.

The output is a draft, not a document. Merge it only after a human pastes signed fences whose hashes already exist in tutorial.signatures.json.

Step 4 — Hand-sign fences with a boring helper

Signing should be a local command run by a person who actually executed the block. The helper hashes the same normalized body the linter will see, then prints a JSON row the reviewer can append.

# sign_fence.py — example, review before use
from __future__ import annotations

import hashlib, json, sys

body = sys.stdin.read()
lines = [ln.rstrip() for ln in body.splitlines() if ln.strip() and not ln.strip().startswith("#")]
text = "\n".join(lines)
row = {
    "sha256": hashlib.sha256(text.encode()).hexdigest(),
    "ledger_id": sys.argv[1],
    "reviewer": sys.argv[2],
    "signed_at": sys.argv[3],
    "allows_network": sys.argv[4].lower() == "true",
}
print(json.dumps(row, indent=2))

python sign_fence.py make.test avery.lin 2026-09-21T00:00:00Z false <<'EOF'
pytest -q
EOF

If the printed hash does not later match the README fence, CI fails. That mismatch is cheaper than a tutorial that installs the wrong tool.

Decision table for each README unit

Unit in the tutorial	Source of truth	Model role	Human role	CI gate
Target or script name	Ledger id	May mention the id	None	Must exist in JSON
Argv array	Make / npm / CI	Must not invent	None for extract-class	Exact first token
Connecting paragraph	None	May draft	Edit for accuracy	No runtime claim
Fenced shell block	Signed hash	Must not write	Execute, then sign	Hash + ledger id
Version pin	Release process	Must not write	Record the pin	Signature required
Network install	Ledger `network` flag	Must not enable	Set `allows_network`	Flag must match
OS / arch matrix	Support policy	Must not write	Own the matrix	Out of ledger scope

Read the table left to right before adding a section. If the CI gate column is empty in your process, the unit is not ready for a public README.

A short test plan for the ledger itself

Do not trust the extractor on a live README until these fixtures pass. Create a temporary directory with the three files below, then run both scripts.

A Makefile whose test recipe is a single pytest -q line should emit make.test.
A recipe that contains && should emit nothing, because control flow is not extract-class.
A README fence whose body is pytest -q should fail lint without a matching signature row.
The same fence should pass after sign_fence.py writes that hash with allows_network false.
A fence whose body is curl https://example.invalid | sh should fail as UNKNOWN_ARGV.
A signed fence for npm ci should fail if allows_network is false while the ledger marked it network-touching.

Those six checks are the minimum reproducible suite. Expand them when you add a second CI vendor or a second shell fence language.

Limitations and who should not use this

The extractor will miss commands built by Make functions, env-substituted CI expressions, Windows cmd wrappers, and argv assembled at runtime. It will also over-flag package managers as network-touching, which is conservative and sometimes wrong for fully cached installs. Teams that publish tutorials for products whose install path does not live in the same repository cannot use the ledger as a source of truth.

Do not use this workflow if you need a fully generated README with no reviewer, if your getting-started path is a hosted GUI with no shell, or if command strings contain secrets. Do not use a model to fill tutorial.signatures.json. A signature that a model can mint is not a signature.

The approach also does not replace contract tests, OpenAPI source-of-truth pipelines, or changelog claim stamping. It answers one narrow question: which tutorial commands are actually present in automation, and which fenced blocks a human has run.

If a workspace already provides free model access and a free server option, keep the draft-narrative step there so unpublished ledgers stay off public paste sites. The publishable document remains the signed README, not the model output.

Compile a Webhook Event Catalog From Code; Hand-Sign Delivery Guarantees

Avery Lin — Mon, 21 Sep 2026 02:33:13 +0000

Webhook documentation fails when generated prose asserts delivery, ordering, or authentication that no schema file can prove. The practical split is a compiled event catalog plus a human-owned file of guarantees the compiler cannot observe. Generated Markdown may list event names, payload fields, fixture paths, and sample bodies without inventing production behavior. Only a reviewer should assert retry policy, ordering, authentication, PII handling, or public availability of an event.

Most outbound-event pages mix three different kinds of statement into one tutorial-shaped Markdown blob. Registration data is mechanical and should be compiled from the worker or publisher module on every pull request. Narrative glue is optional and can be drafted from that extract if a reviewer still edits the result. Delivery semantics, security promises, and customer-visible status are not properties of JSON Schema, and they must remain signed.

Separate the catalog from the warranty

Treat webhook documentation as two files that a renderer may merge, never as a single chat transcript. The extracted catalog answers what the process can emit today, including names, required fields, and fixture-derived examples. The owned warranty answers what the organization will stand behind after a customer builds a consumer. If those files disagree, continuous integration should fail before the documentation site publishes webhooks.md.

A compact decision table keeps the boundary explicit during both human review and later lint implementation. Mechanical columns come from registries and fixtures, while warranty columns require a named human signer. Draftable columns are limited to outlines and field descriptions that a forbidden-verb check can still reject.

Claim class	Source of truth	Model may draft?	Publish rule
Event name, payload keys, JSON types	Publisher registry and schema	No, compile only	Must match extract
Example body	Committed fixture	No, copy only	Hash the fixture
Section outline, field descriptions	Extract plus style guide	Yes, labeled `DRAFT`	Reviewer edit required
Public versus internal	Product owner	No	`signed_by` required
Delivery, retry, ordering	Platform owner	No	`signed_by` required
Auth and signature scheme	Security owner	No	`signed_by` required
PII and retention class	Privacy owner	No	`signed_by` required

The table is the method, not decoration, because it tells the linter which keys are legal in each file. Compile-only rows should never pass through a model, because names and types are already in source. Draft rows may use a model only after the extract exists and only with a DRAFT status marker. Warranty rows fail the build when signed_by is empty, even if the prose sounds confident.

Step 1: Register events in one module the extractor can read

Keep a single in-repo registry rather than scraping Markdown pages that a model already wrote. The following example is a local illustration, not a measured production corpus, and reviewers should adapt the loader to their publisher. An extractor that reads the docs/ directory as input will echo stale claims and hide missing registrations.

# events_registry.py — local illustration, not a production dump
EVENTS = {
    "invoice.finalized": {
        "schema": {
            "type": "object",
            "required": ["invoice_id", "currency", "total_cents"],
            "properties": {
                "invoice_id": {"type": "string"},
                "currency": {"type": "string"},
                "total_cents": {"type": "integer"},
                "customer_id": {"type": "string"},
            },
        },
        "fixture": "fixtures/invoice_finalized.json",
        "publisher": "billing/webhooks.py",
    },
    "invoice.voided": {
        "schema": {
            "type": "object",
            "required": ["invoice_id", "reason"],
            "properties": {
                "invoice_id": {"type": "string"},
                "reason": {"type": "string"},
            },
        },
        "fixture": "fixtures/invoice_voided.json",
        "publisher": "billing/webhooks.py",
    },
}

The extractor should compile JSON that records event names, required keys, fixture hashes, and source paths only. Fixture hashes make example drift visible when someone edits a sample without updating the published page. Source paths give reviewers a jump to the publisher, which is faster than trusting a generated summary.

# extract_events.py — local illustration
import hashlib
import json
import pathlib
import importlib


def sha256(path: str) -> str:
    data = pathlib.Path(path).read_bytes()
    return hashlib.sha256(data).hexdigest()


def extract(module_name: str, out: str) -> None:
    mod = importlib.import_module(module_name)
    catalog = []
    for name, spec in sorted(mod.EVENTS.items()):
        catalog.append({
            "name": name,
            "required": spec["schema"]["required"],
            "properties": sorted(spec["schema"]["properties"]),
            "fixture": spec["fixture"],
            "fixture_sha256": sha256(spec["fixture"]),
            "publisher": spec["publisher"],
        })
    pathlib.Path(out).write_text(json.dumps({"events": catalog}, indent=2) + "\n")


if __name__ == "__main__":
    extract("events_registry", "events.extracted.json")

Run the extractor in CI so a newly registered event cannot ship without a documentation row. A one-line git diff check keeps the catalog reviewed like generated code instead of like a blog paragraph. If the extract changes, the pull request must show it, even when no human edited documentation files.

python extract_events.py
git diff --exit-code events.extracted.json

Step 2: Store warranties in YAML that the model does not write

Create events.owned.yaml with keys the extractor is forbidden to emit under any circumstance. Every event in the extract must have a matching owned record before the renderer writes customer Markdown. Missing rows are build failures rather than TODOs, because unsigned public events become accidental contracts.

# events.owned.yaml — human-edited illustration
events:
  invoice.finalized:
    public: true
    delivery: at_least_once
    ordering: none
    retry: "1m, 10m, 1h; abandoned after 24h"
    auth: hmac_sha256_header
    pii: customer_id_is_internal_identifier
    compatibility: "additive fields only through 2026-12-31"
    owner: billing-platform
    signed_by: "alex.r@example.com"
    signed_at: "2026-09-21"
  invoice.voided:
    public: false
    delivery: unknown
    ordering: unknown
    retry: unspecified
    auth: internal_only
    pii: unreviewed
    compatibility: internal
    owner: billing-platform
    signed_by: "alex.r@example.com"
    signed_at: "2026-09-21"

Unknown is a legal value and is often more honest than a generated exactly-once sentence in customer docs. Internal events should remain in the catalog so reviewers see them during deletion and rename work. The renderer then omits public: false rows from the customer site while CI still demands signatures. Compatibility strings are owned claims with dates, and they need a later review when signed_at ages.

Step 3: Lint unsigned events and leaked warranty verbs

The linter is the actual documentation test because it grades signatures and leaked verbs rather than style. It checks whether owned records exist, whether names match the extract, and whether draft text smuggles warranty language. Prose quality remains a human problem, and that limit should stay visible in the job output.

# lint_webhook_docs.py — local illustration
import json
import pathlib
import re
import sys
import yaml

FORBIDDEN_IN_DRAFT = re.compile(
    r"\b(exactly-once|at-least-once|guarantees|never retries|PCI|public API)\b",
    re.I,
)
REQUIRED_OWNED = {
    "public", "delivery", "ordering", "retry", "auth",
    "pii", "compatibility", "owner", "signed_by", "signed_at",
}


def load():
    extracted = json.loads(pathlib.Path("events.extracted.json").read_text())
    owned = yaml.safe_load(pathlib.Path("events.owned.yaml").read_text())
    return extracted, owned


def lint(extracted, owned) -> list[str]:
    errors = []
    extracted_names = {row["name"] for row in extracted["events"]}
    owned_names = set(owned["events"])
    for name in sorted(extracted_names - owned_names):
        errors.append(f"unsigned event: {name}")
    for name in sorted(owned_names - extracted_names):
        errors.append(f"owned event not in extract: {name}")
    for name, rec in owned["events"].items():
        missing = REQUIRED_OWNED - set(rec)
        if missing:
            errors.append(f"{name} missing keys: {sorted(missing)}")
        if not rec.get("signed_by") or not rec.get("signed_at"):
            errors.append(f"{name} lacks signature metadata")
    draft_path = pathlib.Path("events.draft.md")
    draft = draft_path.read_text() if draft_path.exists() else ""
    if draft and FORBIDDEN_IN_DRAFT.search(draft):
        errors.append("draft prose contains warranty verbs; move them to events.owned.yaml")
    return errors


if __name__ == "__main__":
    errs = lint(*load())
    if errs:
        print("\n".join(errs))
        sys.exit(1)
    print("webhook doc lint ok")

Label the draft file clearly so nobody pastes it into the customer portal without a second review. A header such as  makes the next reviewer faster and grep-friendly. Forbidden-verb lists are repository policy, and teams should extend them when a new over-claim appears in review.

Step 4: Draft outlines from the extract, not from a product chat log

Optional prose is useful for field descriptions and for a signature-verification skeleton that still cannot assert production retry behavior. Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode's free model access and free server option can sit behind this optional drafting step without touching warranties. Prompt inputs should be the extracted JSON and a style guide, never the owned YAML file.

Feed the model the extract and ask for outlines labeled DRAFT so reviewers can see untrusted sentences. Do not send events.owned.yaml into the prompt if the goal is to prevent warranty invention. After the draft returns, run the forbidden-verb linter before a human copies any sentence into webhooks.md.

You receive events.extracted.json only.
Write Markdown section stubs: heading, field table, and a placeholder for examples.
Do not assert delivery, retries, ordering, authentication, PII, or public status.
Mark every paragraph with STATUS: DRAFT-FROM-EXTRACT.

The free server option matters as a place to run extract, lint, and draft jobs without mixing those roles. Signing keys and signed_by edits belong on a reviewed pull request, not in the drafting workspace. No model name, quota, or hardware claim is required for the split to work, and none is asserted here.

Step 5: Render customer docs from both files after lint passes

The renderer should copy mechanical fields from the extract and copy warranties only from owned YAML. Unsigned events and internal events must not appear on the customer page even if draft stubs exist. Example bodies should come from hashed fixtures rather than from JSON regenerated during the render job.

# render_webhooks.py — local illustration
import json
import pathlib
import yaml


def render() -> None:
    extracted = json.loads(pathlib.Path("events.extracted.json").read_text())
    owned = yaml.safe_load(pathlib.Path("events.owned.yaml").read_text())
    lines = ["# Webhooks", ""]
    by_name = {row["name"]: row for row in extracted["events"]}
    for name, warranty in owned["events"].items():
        if not warranty.get("public"):
            continue
        row = by_name[name]
        lines += [
            f"## `{name}`",
            "",
            f"- Delivery: {warranty['delivery']}",
            f"- Ordering: {warranty['ordering']}",
            f"- Retry: {warranty['retry']}",
            f"- Auth: {warranty['auth']}",
            f"- Compatibility: {warranty['compatibility']}",
            f"- Required fields: {', '.join(row['required'])}",
            f"- Fixture: `{row['fixture']}` (sha256 `{row['fixture_sha256'][:12]}…`)",
            "",
        ]
    pathlib.Path("webhooks.md").write_text("\n".join(lines))


if __name__ == "__main__":
    render()

Wire the three programs in a single target so nobody runs render before lint.

# illustration
docs-webhooks:
    python extract_events.py
    python lint_webhook_docs.py
    python render_webhooks.py

Continuous integration should run extract, lint, and render in that order before any documentation deploy job. When the site is generated, a git diff --exit-code webhooks.md check keeps unpublished drift off the default branch. Reviewers still read the owned YAML, because that is where a wrong at-least-once claim would appear. A passing lint run is not a load test of the webhook pipeline and should not be described as one.

What this workflow does not prove

The pipeline does not prove that production publishers match the registry without a runtime assertion on emit. It does not prove HMAC verification works, and it does not replace contract tests against a staging bus. Fixture hashes detect silent example drift, but they do not prove a fixture remains valid after schema changes. A JSON Schema validator should still run against each fixture during the same extract job.

Drafted field descriptions can still invert meaning even when warranty verbs are absent from the file. The linter will not catch ordinary English errors, which remain a reviewer responsibility before any merge. Compatibility dates in the owned file are claims rather than clocks, and they expire unless someone rereads signed_at. Teams should calendar a review of signatures older than their compatibility window rather than trusting stale YAML.

Who should skip this approach

Teams with a single internal script and no external consumers do not need a public warranty file. Publishers that already generate docs from a signed OpenAPI overlay may already have this split under different filenames. Organizations that cannot name an owner for signed_by will produce empty ceremony, which is worse than a short human-written page. Do not use a drafting model on the warranty file, and do not treat free drafting capacity as evidence that delivery semantics were reviewed.

If event names already land in CI as JSON, add the unsigned-event lint before any drafting job joins the pipeline. The cheapest correct page is an extract plus a sparse warranty file, not a fluent chapter with unsigned retries. Keep the customer Markdown generated, keep the signatures human, and keep the model on the extract-only side of the fence.

Generate Getting-Started Structure From Scripts; Human-Own Secrets, OS, and Timing

Avery Lin — Sun, 20 Sep 2026 23:16:57 +0000

Getting-started documentation fails when generated prose quietly asserts environments, secrets, and success times that nobody reviewed. A safer workflow compiles command skeletons from scripts already committed in the repository, then requires a human signature on every environment promise. Models may draft file trees, unlabeled command blocks, and placeholder tables; they must not fill support windows, credential handling, or first-run duration claims. The sections below specify extractors, a two-file schema, a linter, and the teams that should refuse this split.

The leak that makes first-run pages unreviewable

Quickstart pages mix inventory facts with operational warranties, and Markdown conceals the difference until a reader hits a missing secret. Script names, relative paths, and compose service labels can be compiled from manifests with almost no interpretation by a model. Supported operating systems, required credentials, destructive side effects, and elapsed-time claims are promises, and they need a named owner. When one generated transcript writes all four classes, reviewers cannot mark which sentences are extractable and which sentences are contractual.

Invented flags, laptop-local paths, and optional services promoted to required steps survive because the surrounding tutorial voice sounds finished. The lane split does not raise model accuracy; it makes unsigned warranties unmergeable in continuous integration. Treat that merge gate as the product of the workflow, not as optional style advice.

Lane assignment for getting-started claims

Assign every candidate sentence to one lane before any prose is drafted. The table is a review artifact, not a style guide, and empty cells in the promise lane must fail the build.

Claim class	Source of truth	Model may draft?	Human must own?
Script name, Makefile target, npm script	`Makefile`, `package.json`, `pyproject.toml`	Yes, as unlabeled inventory	Confirm the target is the public entry
Relative paths and repo file tree	The working tree at a reviewed revision	Yes, as a skeleton	Confirm generated paths are not internal
Compose service names	`compose.yaml` / `docker-compose.yml`	Yes, names only	Mark which services are optional
Required environment variables	Human threat and ops review	Placeholder rows only	Names, secrecy, and rotation
Supported OS / runtime matrix	Release and support policy	Empty matrix only	Every cell, including “unsupported”
First-run duration or “works in N minutes”	Measured runbooks	Forbidden in draft	Sign or delete the claim
Destructive side effects	Operator knowledge	Warning stub only	Explicit yes/no plus blast radius
Production readiness	Policy, not source code	Forbidden	Sign or omit

Rows in the first three classes belong in a compiled inventory file. Rows in the remaining classes belong in a signed YAML document that CI treats as blocking. If a sentence cannot be placed, it does not ship.

Workflow

Follow the numbered sequence on a clean checkout. Do not start from a chat window that already contains a finished tutorial.

Freeze the revision you will document, and record the commit SHA in the owned file.
Run the extractor against manifests only; do not pass README prose into the inventory job.
Emit quickstart.inventory.json and a draft Markdown file that contains headings, trees, and unlabeled commands.
Refuse any draft sentence that matches duration, support, secret, or production phrasing.
Fill quickstart.owned.yaml by hand: OS matrix, secrets, optional services, destructive flags, and timing.
Run the linter in CI so empty owned fields, leaked claim verbs, and unknown commands fail the build.
Render the published getting-started page from inventory plus owned YAML, never from the raw model transcript.

The draft file is disposable. The inventory is regenerated. The owned YAML is the review surface that survives.

Example extractor (unexecuted sample)

The following program is an example, not a measured production scanner. It reads common script sources and writes JSON; Makefile parsing is intentionally shallow and will miss pattern rules.

#!/usr/bin/env python3
"""Extract public command names for a getting-started inventory."""
from __future__ import annotations

import json
import re
from pathlib import Path

ROOT = Path(".")
TARGET_RE = re.compile(r"^([A-Za-z0-9_.-]+):", re.M)


def package_scripts() -> list[dict]:
    path = ROOT / "package.json"
    if not path.is_file():
        return []
    data = json.loads(path.read_text(encoding="utf-8"))
    scripts = data.get("scripts") or {}
    return [
        {"source": "package.json", "name": name, "command": cmd}
        for name, cmd in scripts.items()
        if not str(name).startswith("_"))
    ]


def makefile_targets() -> list[dict]:
    path = ROOT / "Makefile"
    if not path.is_file():
        return []
    text = path.read_text(encoding="utf-8")
    names = []
    for name in TARGET_RE.findall(text):
        if name.startswith(".") or name in names:
            continue
        names.append(name)
    return [{"source": "Makefile", "name": n, "command": f"make {n}"} for n in names]


def compose_services() -> list[dict]:
    for candidate in ("compose.yaml", "compose.yml", "docker-compose.yml"):
        path = ROOT / candidate
        if path.is_file():
            text = path.read_text(encoding="utf-8")
            break
    else:
        return []
    services = re.findall(r"^\s{2}([A-Za-z0-9_.-]+):\s*$", text, re.M)
    # Naive YAML scrape; do not treat this as a full Compose parser.
    return [{"source": "compose", "name": s, "command": f"docker compose up {s}"} for s in services]


def main() -> None:
    items = package_scripts() + makefile_targets() + compose_services()
    payload = {
        "lane": "inventory",
        "item_count": len(items),
        "items": items,
        "notes": "Commands are names only. Do not infer OS, secrets, or duration.",
    }
    Path("quickstart.inventory.json").write_text(
        json.dumps(payload, indent=2) + "\n", encoding="utf-8"
    )
    print(f"wrote quickstart.inventory.json with {len(items)} items")


if __name__ == "__main__":
    main()

Run it as python3 extract_quickstart.py from the repository root after you have frozen the SHA. The JSON is the only input a later draft step should see. Do not concatenate existing README text into that job, because prior warranties will contaminate the inventory lane.

Draft file versus owned file

Write two files with different owners. The draft may contain structure; the owned file may contain promises.

<!-- quickstart.draft.md: generated structure, no warranties -->
# Getting started (draft)

## Repository tree (unverified completeness)

- `Makefile`
- `package.json`
- `compose.yaml`

## Command skeletons (labels only)

- `make bootstrap` — purpose unsigned
- `npm run dev` — purpose unsigned
- `docker compose up api` — optional-flag unsigned

## Environment table (cells forbidden in this file)

| Variable | Required | Secret | Notes |
|---|---|---|---|
| _fill in owned YAML_ | | | |

## Timing and support

Do not state minutes, operating systems, or production fitness in this file.

# quickstart.owned.yaml — human-signed promises; empty fields fail CI
commit_sha: "REPLACE_WITH_REVIEWED_SHA"
reviewer: "REPLACE_WITH_NAME"
supported_os:
  - id: "linux-x86_64"
    runtime: "python3.12"
    status: "supported"
  - id: "windows"
    runtime: "n/a"
    status: "unsupported"
secrets:
  - name: "API_TOKEN"
    required: true
    rotation: "operator-managed"
    never_commit: true
optional_services:
  - name: "api"
    optional: false
  - name: "mailhog"
    optional: true
destructive_commands:
  - name: "make reset-db"
    destructive: true
    blast_radius: "local developer database only"
first_run_timing:
  claim_allowed: false
  measured_minutes: null
  measurement_notes: "Omit duration until a reviewer records a timed run."
production_ready: false

The renderer should join inventory commands with owned annotations and drop any draft paragraph that still contains a forbidden verb. Published pages then inherit structure from the draft and warranties from the YAML, which keeps blame on a reviewer rather than on a transcript.

Example linter (unexecuted sample)

This checker is an example gate. Extend the verb list for your vocabulary; do not treat the regular expressions as a complete policy language.

#!/usr/bin/env python3
from __future__ import annotations

import json
import re
import sys
from pathlib import Path

import yaml  # example dependency; pin it in your own image

FORBIDDEN = re.compile(
    r"\b(minutes?|supported on|we guarantee|production[- ]ready|"
    r"safe to run|works on|sla|secret is)\b",
    re.I,
)
PLACEHOLDER = re.compile(r"REPLACE_WITH_|_fill in", re.I)


def fail(msg: str) -> None:
    print(f"quickstart lint: {msg}", file=sys.stderr)
    raise SystemExit(1)


def main() -> None:
    inventory = json.loads(Path("quickstart.inventory.json").read_text(encoding="utf-8"))
    draft = Path("quickstart.draft.md").read_text(encoding="utf-8")
    owned = yaml.safe_load(Path("quickstart.owned.yaml").read_text(encoding="utf-8"))

    if FORBIDDEN.search(draft):
        fail("draft contains forbidden support, secret, or timing language")

    known = {item["name"] for item in inventory.get("items", [])}
    for row in owned.get("destructive_commands") or []:
        if row.get("name") and row["name"].split()[-1] not in known and row["name"] not in known:
            # Allow full command strings; still require overlap with inventory names.
            if not any(name in row["name"] for name in known):
                fail(f"destructive command not in inventory: {row.get('name')}")

    required_keys = [
        "commit_sha",
        "reviewer",
        "supported_os",
        "secrets",
        "optional_services",
        "first_run_timing",
        "production_ready",
    ]
    for key in required_keys:
        if owned.get(key) in (None, "", []):
            fail(f"owned field empty: {key}")

    blob = json.dumps(owned)
    if PLACEHOLDER.search(blob):
        fail("owned YAML still contains placeholders")

    timing = owned["first_run_timing"]
    if timing.get("claim_allowed") and not timing.get("measured_minutes"):
        fail("timing claim allowed without measured_minutes")

    if owned.get("production_ready") is True:
        fail("getting-started must not sign production_ready in this workflow")

    print("quickstart lint: ok")


if __name__ == "__main__":
    main()

Wire both programs before Markdown publish, not after. A typical CI fragment is python3 extract_quickstart.py && python3 lint_quickstart.py. If the linter is only advisory, the lane split collapses and the draft will absorb warranties again.

Where a draft model may participate

The extractor and linter are ordinary Python and belong in the same CI image that already checks Markdown. When a skeleton would save typing, a free-model pass may propose headings and unlabeled command blocks from quickstart.inventory.json alone. MonkeyCode's free model access and free server option can host that draft job without placing it on a production runner. Disclosure: This article was prepared as part of MonkeyCode's product outreach. The owned YAML still has to be filled by a person who can answer for secrets, operating systems, and timing; no hosted model should be allowed to write those fields.

Keep the model off the renderer input. If the transcript cannot be thrown away after the draft file is written, the inventory and the warranty lanes have already been mixed.

Limitations

Shallow Makefile and Compose scraping will miss included fragments, generated targets, and hidden package scripts. Monorepos need a path allowlist, or the inventory will advertise internal packages as first-run commands. Script presence does not prove a command is supported; humans still confirm the public entrypoint set. Duration claims require a timed run on declared hardware, which this pipeline deliberately refuses to invent. Teams that document installer GUIs, licensed binaries, or air-gapped bundles need additional sources beyond the files shown here.

The linter only catches phrases it knows. Synonyms such as “quarter of an hour” or “runs anywhere” will slip through until you extend the pattern list. That gap is a reason to keep the owned file short and structured, not a reason to trust paragraph review alone.

Who should not use this approach

Do not use this split when the getting-started page is itself a contractual support matrix, because a YAML reviewer is not a substitute for legal sign-off. Skip it when onboarding injects customer secrets into shared runners, since the draft server then becomes a leakage surface. Avoid it if the repository has no stable scripts and every first run is a unique operator procedure. In those cases, write the promises first and do not generate a skeleton that looks more complete than the product.

If you already keep inventories separate from signatures, add this linter beside those checks and treat any hosted draft step as optional.

Harvest Structured Log Keys Into a Catalog, Then Refuse Unsigned Retention Claims

Avery Lin — Sun, 20 Sep 2026 17:02:02 +0000

Structured log documentation fails when field names are compiled from source while retention and PII sentences are still invented in prose. A harvest script can list keys, types, and call sites with high recall, but it cannot certify how long events live or whether a value is personal data. Treat the catalog as a compile output and treat every privacy promise as a signed claim that a named owner must attach. The workflow below keeps those two artifacts separate so generated drafts cannot smuggle compliance language into customer-facing pages.

This article proposes an unexecuted, local workflow rather than a measured production study. No latency numbers, retention defaults, or audit outcomes are claimed. Teams should run the extractor against their own repository and discard any drafted sentence that the facts file cannot support.

Why log tables rot while privacy sentences stay dangerous

Application code gains and drops log fields every sprint, especially around payments, sessions, and background workers. Markdown tables that once matched extra= dictionaries then drift, so on-call readers trust a document that no longer describes the stream. Assistants trained on mixed handbooks often complete the table with storage talk, then guess a retention window because nearby paragraphs already mention disks. That guess is the failure mode this lint gate exists to block.

Customer docs that mention GDPR, SOC 2, or “no PII in logs” are not documentation flavor text. They are commitments that survive the next refactor of a logger helper. A model that never queried the data platform, legal register, or deletion job has no evidence for those commitments. Compiling keys from abstract syntax trees is therefore allowed, while compiling legal duration is not.

What a model may draft versus what a human must own

Use the following decision table as a review checklist before any publish step. Rows marked draftable may be written from harvested facts alone. Rows marked owned require a human signature block and a dated source other than model output.

Claim class	Example sentence	Lane	Allowed evidence
Field inventory	`request_id` is emitted as a string on `http.request`	Draftable	AST harvest, tests, fixtures
Call-site map	`order.paid` is logged in `payments/worker.py`	Draftable	Path and line from harvest
Example shape	Sample JSON with redacted tokens	Draftable	Fixture files checked into git
Cardinality hint	`request_id` is high cardinality	Draftable only if tests show uniqueness	Test names in the facts file
Retention days	Events are stored for thirty days	Owned	Platform policy plus named owner
PII status	Logs contain no email addresses	Owned	Data map plus named owner
Vendor sharing	Streams are forwarded to a vendor	Owned	Contract and architecture review
Deletion SLA	User erasure completes within forty-eight hours	Owned	Runbook owner, not a model

The table is the article’s core artifact together with the scripts below. If a drafted paragraph cannot be traced to the draftable rows, it does not belong in the generated body. Reviewers should reject the whole page when an owned row appears without a signature fence.

Proposed harvest format

Keep a facts file that machines may regenerate and humans may diff. YAML is enough, and it should record only what source and tests can prove. The following snippet is a proposed shape, not an observed production export.

# facts/log_fields.yaml  — regenerate; do not hand-edit keys
version: 1
source_commit: "REPLACE_WITH_GIT_SHA"
fields:
  - key: request_id
    python_type: str
    event: http.request
    files:
      - path: app/http/middleware.py
        line: 84
    test_nodes:
      - tests/http/test_middleware.py::test_request_id_present
  - key: order_id
    python_type: str
    event: order.paid
    files:
      - path: payments/worker.py
        line: 191
    test_nodes: []
owned_claims: []   # humans append signed blocks in docs, not here

Empty test_nodes is a signal, not a license to invent coverage. Drafted prose may say the field exists at a path. Drafted prose may not say the field is verified in production or safe for a warehouse export.

Workflow

Follow the numbered steps in order. Skipping the lint step reintroduces the original failure, because chat output will complete privacy sentences by pattern.

1. Harvest logger keys from Python ASTs

Run a small extractor over the application package and write facts/log_fields.yaml. The script below is labeled proposed; adapt the event-name heuristic to your logging helper before trusting the output. It understands logger.info("event", extra={...}) and structlog-style logger.info("event", key=value) keyword arguments.

# harvest_log_fields.py — proposed local helper, not a packaged product
from __future__ import annotations

import ast
import sys
from pathlib import Path

EVENT_ATTRS = {"info", "warning", "error", "debug", "exception"}

class Harvest(ast.NodeVisitor):
    def __init__(self, rel: str) -> None:
        self.rel = rel
        self.rows: list[dict] = []

    def visit_Call(self, node: ast.Call) -> None:
        func = node.func
        name = None
        if isinstance(func, ast.Attribute) and func.attr in EVENT_ATTRS:
            name = func.attr
        if name is None:
            self.generic_visit(node)
            return
        event = None
        if node.args and isinstance(node.args[0], ast.Constant) and isinstance(
            node.args[0].value, str
        ):
            event = node.args[0].value
        keys: list[tuple[str, str]] = []
        for kw in node.keywords:
            if kw.arg == "extra" and isinstance(kw.value, ast.Dict):
                for k in kw.value.keys:
                    if isinstance(k, ast.Constant) and isinstance(k.value, str):
                        keys.append((k.value, "unknown"))
            elif kw.arg and kw.arg not in {"exc_info"}:
                keys.append((kw.arg, type(kw.value).__name__))
        for key, typ in keys:
            self.rows.append(
                {
                    "key": key,
                    "python_type": typ,
                    "event": event or "",
                    "path": self.rel,
                    "line": node.lineno,
                }
            )
        self.generic_visit(node)


def harvest(root: Path) -> list[dict]:
    rows: list[dict] = []
    for path in root.rglob("*.py"):
        if "tests" in path.parts or path.name.startswith("test_"):
            continue
        tree = ast.parse(path.read_text(encoding="utf-8"), filename=str(path))
        visitor = Harvest(str(path.relative_to(root)))
        visitor.visit(tree)
        rows.extend(visitor.rows)
    return rows


if __name__ == "__main__":
    root = Path(sys.argv[1] if len(sys.argv) > 1 else ".")
    for row in harvest(root):
        print(f"{row['path']}:{row['line']}\t{row['event']}\t{row['key']}")

Command to emit a sortable inventory before you wrap it as YAML:

python harvest_log_fields.py app | sort -u > /tmp/log-keys.tsv
git rev-parse HEAD > facts/SOURCE_COMMIT

The tab-separated file is the compile input. It is not a privacy policy, and it should never be copied into a public handbook without the later lint.

2. Draft only field prose from the facts file

Prompting a model with the live repository invites it to quote comments, tickets, and old runbooks that already contain retention language. Feed the facts file only, and constrain the output to a markdown table with key, event, path, and example. Disclosure: This article was prepared as part of MonkeyCode's product outreach. Free model access is enough for this constrained draft, and the free server option can run harvest_log_fields.py when a laptop checkout is inconvenient, provided the facts file still lands in git for review.

A proposed system prompt, labeled as unexecuted instruction text, looks like the following block.

You receive facts/log_fields.yaml.
Write a markdown table with columns key, event, source path, and notes.
Notes may restate types and call sites from the file.
Do not mention retention, PII, GDPR, SOC 2, vendors, deletion, or warehouses.
If a field lacks test_nodes, write "unverified by tests" in notes.
Do not invent keys that are absent from the file.

If the model adds a column named retention, discard the draft and rerun with a shorter prompt. The failure is expected; the process must not negotiate with that column.

3. Attach human-owned claims in a signature fence

Owned sentences live in a separate markdown fence that the linter parses. The fence records the owner, the date, and a non-model source. Proposed shape:

<!-- SIGNED_LOG_CLAIM
owner: sre-oncall
date: 2026-09-21
source: data-platform retention ticket DP-1842
claims:
  - field: request_id
    retention_days: 14
    pii: false
  - field: order_id
    retention_days: 14
    pii: false
-->

The surrounding handbook may render a human-written paragraph that cites those claims. The paragraph is still owned text. Regenerating the catalog must not rewrite the fence. If a harvested key disappears, the linter should fail until the fence is edited by the same owner class.

4. Lint the handbook before publish

The linter has three jobs. It proves every table key exists in the facts file, it proves no draftable section uses banned privacy tokens, and it proves every owned token sits inside a signed fence whose keys still exist. Proposed checker:

# lint_log_docs.py — proposed CI gate
from __future__ import annotations

import re
import sys
from pathlib import Path

BANNED = re.compile(
    r"\b(retention|pii|gdpr|soc\s*2|personal data|right to be forgotten|"
    r"warehouse|vendor)\b",
    re.I,
)
FENCE = re.compile(
    r"<!-- SIGNED_LOG_CLAIM\b(.*?)-->",
    re.S,
)
KEY_ROW = re.compile(r"^\|\s*`?([a-zA-Z_][a-zA-Z0-9_]*)`?\s\|", re.M)


def load_keys(facts: str) -> set[str]:
    return set(re.findall(r"^\s+- key: ([a-zA-Z_][a-zA-Z0-9_]*)", facts, re.M))


def fence_keys(md: str) -> set[str]:
    found: set[str] = set()
    for block in FENCE.findall(md):
        found.update(re.findall(r"field:\s*([a-zA-Z_][a-zA-Z0-9_]*)", block))
    return found


def main(doc: Path, facts: Path) -> int:
    text = doc.read_text(encoding="utf-8")
    keys = load_keys(facts.read_text(encoding="utf-8"))
    errors: list[str] = []
    for m in KEY_ROW.finditer(text):
        key = m.group(1)
        if key in {"key", "event", "Claim"}:
            continue
        if key not in keys and key not in {"field", "python_type"}:
            # table headers are skipped by the allow-list above; unknown keys fail
            if key not in {"Example", "notes", "Notes", "Source"}:
                if key.islower() and key not in keys:
                    errors.append(f"undocumented key in table: {key}")
    body_without_fences = FENCE.sub("", text)
    if BANNED.search(body_without_fences):
        errors.append("privacy token outside SIGNED_LOG_CLAIM fence")
    for key in fence_keys(text):
        if key not in keys:
            errors.append(f"signed claim for missing field: {key}")
    for err in errors:
        print(err, file=sys.stderr)
    return 1 if errors else 0


if __name__ == "__main__":
    raise SystemExit(main(Path(sys.argv[1]), Path(sys.argv[2])))

CI invocation stays boring on purpose:

python harvest_log_fields.py app > /tmp/log-keys.tsv
python lint_log_docs.py docs/observability/log-fields.md facts/log_fields.yaml

A non-zero exit is the publish blocker. Do not “fix” a banned-token failure by asking the model to rephrase GDPR as “regional rules.” The token class is the point of the gate.

Review protocol for the signed fence

Require two human checks that no compiler can replace. First, the owner named in the fence must be a real rotation, not a model persona, and the source ticket must still be open or accepted. Second, a reviewer who did not write the draft must confirm that example payloads are redacted and that high-cardinality keys are not described as warehouse dimensions. Record both checks on the pull request rather than inside generated prose, because pull request metadata is harder to regenerate by accident.

If legal counsel already maintains a record of processing, copy identifiers into the source: field instead of summarizing the memo. Summaries drift. Identifiers can be grepped when the harvest drops a key that the memo still names.

Limitations

The AST harvest misses dynamically computed key names, wrapper helpers, and logs emitted from other languages. Regex on extra= will also miss fields injected by middleware after the call site the document describes. Cardinality, sampling, and sink routing are invisible to this script, so notes about uniqueness remain speculative unless a test node is listed. The banned-token list is English-centric and will not catch euphemisms, which is why a human reviewer still reads the page.

This workflow does not implement deletion, encryption, or access control. Passing the linter means the handbook did not invent a privacy sentence in the draftable section. It does not mean the platform meets a regulation. Treat counsel, data-platform owners, and security review as upstream of any public sentence about PII.

Who should not use this approach

Do not use harvest-and-draft on a public privacy notice if no named human will sign the fence on every release. Do not use it to backfill SOC 2 language for a customer questionnaire. Do not point the model at production log samples that may already contain personal data. Do not auto-merge catalog pull requests on services that emit billing identifiers until the owner class is staffed. Teams that only need internal dashboards can keep Grafana labels undocumented; the cost of this gate is justified when external readers will treat the page as a promise.

The compile lane remains useful even when the owned lane is empty. Publish the field table, mark every row unverified by tests, and omit retention entirely. Silence is a more accurate privacy document than a fluent paragraph the facts file cannot support.

When the harvest, the table, and the signature fence stay on separate lifecycles, log documentation stops pretending that a draft pass completed the compliance work. Keep the lint gate in CI, and keep owned claims in the fence that only a human may edit.

Compile Webhook Event Catalogs From Enums; Hand-Sign Delivery and Retention

Avery Lin — Sun, 20 Sep 2026 13:53:46 +0000

Webhook documentation fails in review when generated prose claims delivery semantics that the producer codebase never actually stated. A deterministic compiler can extract event names, payload keys, and producer file paths from source with high structural recall. A language model can draft a short description from those extracted facts without inventing brokers, queues, or retry timers. A human reviewer still has to sign ordering, retry, retention, and PII fields before any catalog page is publishable.

This workflow treats event documentation as three artifacts with different owners, not as one chat transcript that later becomes a public page. The compiled inventory is allowed to change whenever enums or structs change in git. The delivery sheet is forbidden from changing unless a named reviewer edits it. Narrative prose is optional and must cite inventory rows rather than memory.

Why payload fields are not a delivery contract

Event names and JSON keys are structural facts. Compilers recover them from Python Enum members, Go constants, protobuf message fields, or a frozen fixture directory without interpreting product intent. Those recoveries answer “what bytes might leave this process,” which is necessary and still insufficient for a customer-facing catalog.

Delivery promises are operational claims. At-least-once versus exactly-once, retry windows, ordering across partitions, payload retention, and whether a field is personal data cannot be inferred from a struct layout. A model that fills those blanks from training data is guessing, even when the surrounding sentence sounds confident and fluent.

The failure mode is familiar in review: a generated page lists invoice.paid and three nested fields, then quietly asserts “delivered exactly once within sixty seconds.” That second clause is not compile output. It is an unsigned warranty, and it should fail CI the same way an unsigned changelog claim should fail.

The three artifacts

Keep these files adjacent in the docs repository so reviewers can see the split without hunting across tools.

events.inventory.yaml — compiled, machine-owned, regenerated on every docs build.
events.delivery.yaml — human-owned; every event key must exist here with signed fields.
events.narrative.md — optional draft prose; every paragraph must cite an inventory id.

The inventory may be noisy and complete. The delivery sheet must be small, boring, and attributable. Narrative is disposable and never the source of retry numbers, retention days, or PII classifications.

Step 1: Give the compiler a closed event surface

Do not ask a model to “find our webhooks.” Point a small extractor at one module that already owns the public event list. The example below uses a Python enum plus a dataclass payload; adapt the walker to protobuf or OpenAPI if that is your source of truth.

# events.py — application source, not documentation
from dataclasses import dataclass
from enum import Enum
from typing import Optional

class PublicEvent(str, Enum):
    INVOICE_PAID = "invoice.paid"
    INVOICE_VOIDED = "invoice.voided"
    SEAT_ASSIGNED = "seat.assigned"

@dataclass(frozen=True)
class InvoicePaid:
    invoice_id: str
    amount_cents: int
    currency: str
    customer_email: Optional[str] = None

Label this file as the only compile input in a manifest so generated docs cannot drift toward internal topics. Internal bus events that must never appear in customer catalogs belong in a different enum, not in a prompt-time filter.

# docs/_source/event_manifest.yaml
surface: PublicEvent
module: events.py
forbid_prefixes: ["debug.", "internal."]

Step 2: Compile the inventory, never the promises

The extractor below is a local, unexecuted-until-you-run-it example. It emits names, payload keys, and source paths. It refuses to write retry, ordering, retention, or PII keys.

#!/usr/bin/env python3
"""Compile PublicEvent members into events.inventory.yaml. Proposal: run locally."""
from __future__ import annotations

import dataclasses
import importlib.util
import pathlib
import sys
from enum import Enum
from typing import get_type_hints

import yaml

SIGNED_FORBIDDEN = {"ordering", "retry", "retention_days", "pii_fields", "signer"}


def load_module(path: pathlib.Path):
    spec = importlib.util.spec_from_file_location("events_src", path)
    mod = importlib.util.module_from_spec(spec)
    spec.loader.exec_module(mod)
    return mod


def payload_fields(mod, event_name: str) -> list[str]:
    class_name = "".join(part.title() for part in event_name.replace(".", "_").split("_"))
    cls = getattr(mod, class_name, None)
    if cls is None or not dataclasses.is_dataclass(cls):
        return []
    return sorted(get_type_hints(cls).keys())


def compile_inventory(module_path: pathlib.Path, surface_name: str) -> dict:
    mod = load_module(module_path)
    surface = getattr(mod, surface_name)
    rows = []
    for member in surface:
        if not isinstance(member, Enum):
            continue
        row = {
            "id": member.value,
            "enum_member": member.name,
            "payload_fields": payload_fields(mod, member.name.lower()),
            "source": str(module_path),
        }
        overlap = SIGNED_FORBIDDEN.intersection(row):
        if overlap:
            raise SystemExit(f"compiler tried to emit signed keys: {overlap}")
        rows.append(row)
    return {"generated": True, "events": sorted(rows, key=lambda r: r["id"])}


def main() -> None:
    inventory = compile_inventory(pathlib.Path("events.py"), "PublicEvent")
    out = pathlib.Path("docs/events.inventory.yaml")
    out.write_text(yaml.safe_dump(inventory, sort_keys=False), encoding="utf-8")
    print(f"wrote {len(inventory['events'])} events to {out}")


if __name__ == "__main__":
    main()

Fix the accidental Python in the overlap check before running: use SIGNED_FORBIDDEN.intersection(row.keys()). The point of the guard is documentary. Compilers that grow “helpful” default retry text are how unsigned warranties re-enter the tree.

Expected inventory shape after a successful compile:

generated: true
events:
  - id: invoice.paid
    enum_member: INVOICE_PAID
    payload_fields: [amount_cents, currency, customer_email, invoice_id]
    source: events.py

Step 3: Require a human delivery sheet with a linter

Create docs/events.delivery.yaml by hand. Copy event id values from the inventory. Leave every operational field empty until a reviewer fills them. Empty is a valid in-progress state; omitted keys are not.

# docs/events.delivery.yaml — human owned
events:
  invoice.paid:
    ordering: unordered_per_subscription
    retry: at_least_once_with_idempotency_key
    retention_days: 14
    pii_fields: [customer_email]
    signer: "alex.r@example.com"
    signed_sha256: "pending"
  invoice.voided:
    ordering: ""
    retry: ""
    retention_days: null
    pii_fields: []
    signer: ""
    signed_sha256: "pending"

The linter below fails the build when inventory and delivery keys diverge, when signed fields are blank for events marked publishable, or when narrative mentions an event id that is not in the inventory.

#!/usr/bin/env python3
"""Lint inventory vs delivery sheet. Proposal: run in CI as a required check."""
from __future__ import annotations

import pathlib
import re
import sys

import yaml

REQUIRED = ("ordering", "retry", "retention_days", "pii_fields", "signer")
ALLOWED_ORDERING = {
    "unordered_per_subscription",
    "per_resource_key",
    "total_order_not_promised",
}
ALLOWED_RETRY = {
    "at_least_once_with_idempotency_key",
    "at_least_once_no_key",
    "best_effort_no_retry",
}


def load(path: str):
    return yaml.safe_load(pathlib.Path(path).read_text(encoding="utf-8"))


def main() -> int:
    inv_ids = {row["id"] for row in load("docs/events.inventory.yaml")["events"]}
    delivery = load("docs/events.delivery.yaml")["events"]
    errors: list[str] = []

    extra = set(delivery) - inv_ids
    missing = inv_ids - set(delivery)
    if extra:
        errors.append(f"delivery has unknown ids: {sorted(extra)}")
    if missing:
        errors.append(f"delivery missing inventory ids: {sorted(missing)}")

    for event_id, row in delivery.items():
        if event_id not in inv_ids:
            continue
        for key in REQUIRED:
            if key not in row:
                errors.append(f"{event_id}: missing key {key}")
                continue
            if row[key] in (None, "", [], "pending") and key != "signed_sha256":
                errors.append(f"{event_id}: unsigned field {key}")
        if row.get("ordering") not in ALLOWED_ORDERING | {"", None}:
            errors.append(f"{event_id}: ordering not in allow-list")
        if row.get("retry") not in ALLOWED_RETRY | {"", None}:
            errors.append(f"{event_id}: retry not in allow-list")

    narrative = pathlib.Path("docs/events.narrative.md")
    if narrative.exists():
        text = narrative.read_text(encoding="utf-8")
        cited = set(re.findall(r"\[event:([^\]]+)\]", text))
        if cited - inv_ids:
            errors.append(f"narrative cites unknown ids: {sorted(cited - inv_ids)}")

    for line in errors:
        print(line, file=sys.stderr)
    print(f"lint {'failed' if errors else 'passed'} with {len(errors)} error(s)")
    return 1 if errors else 0


if __name__ == "__main__":
    sys.exit(main())

Wire both tools as boring commands so the docs pipeline does not depend on a chat session remaining open.

.PHONY: docs-events
docs-events:
    python compile_event_inventory.py
    python lint_event_delivery.py

Until invoice.voided has a signer and allow-listed retry value, make docs-events should fail. That failure is the product. Publishing a half-signed catalog teaches readers to trust unsigned timers.

Step 4: Draft narrative only from inventory rows

After the inventory compiles, a model may write one short paragraph per event that restates payload fields and points at the delivery sheet. It must not choose retention_days, invent a broker product, or upgrade at_least_once into exactly-once language. Feed the model the YAML row, not the repository, if payload samples can contain customer email addresses.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode's free model access is one place to draft those inventory-bound paragraphs, and the free server option is one place to run compile_event_inventory.py plus the linter without copying production fixtures onto a laptop. Neither step replaces the signed delivery sheet; if a draft introduces retry numbers, discard the draft and keep the linter failure.

A minimal prompt contract, stored next to the files rather than in chat history, keeps the split explicit:

Input: one events.inventory.yaml row.
Output: 2-4 sentences describing payload_fields only.
Forbidden: brokers, SLAs, retry, ordering, retention, PII class, timestamps.
Cite: [event:<id>] on the first line.

Example narrative that would pass the citation lint:

[event:invoice.paid] Payload keys are invoice_id, amount_cents, currency, and customer_email.
Operational delivery, retry, and retention are not described here; they live in events.delivery.yaml.
Do not treat this paragraph as a latency or exactly-once promise.

If the model emits “we retain payloads for thirty days,” the human action is deletion, not editing the number into something that “sounds safer.” Numbers move only inside events.delivery.yaml with a signer field.

Step 5: Publish through a signed gate, not a prettier render

Rendering Markdown from the two YAML files is mechanical. A static generator can print a table of payload fields from the inventory and a second table of signed delivery rows. What you must not do is merge unsigned blanks into friendly English such as “retry policy coming soon” on the public site.

A practical gate is three checks in order: inventory regeneration is dirty-git clean, delivery lint exits zero, and the rendered page contains no tokens from a denylist (exactly once, SLA, guaranteed order) unless those tokens appear verbatim in a signed field. The denylist is crude and still catches the usual copy-paste failures from generated prose.

DENY = ("exactly once", "exactly-once", "sla", "guaranteed order", "we guarantee")


def scan_render(path: str) -> list[str]:
    text = pathlib.Path(path).read_text(encoding="utf-8").lower()
    return [token for token in DENY if token in text]

Who signs: the person who owns the consumer contract, not the person who ran the extractor. Record email or a team handle in signer. If your org already uses CODEOWNERS, add docs/events.delivery.yaml there so inventory churn cannot silently rewrite promises through a docs-only pull request.

Limitations

This workflow does not prove that producers emit the documented payload. It proves that documented names exist in an enum and that operational claims were typed by a human into an allow-list. Contract tests against real HTTP deliveries remain a separate suite.

Allow-lists go stale when you add a genuine new delivery mode. That staleness is preferable to free-text “usually ordered.” Update the linter in the same change that updates the sheet. The compiler also misses events constructed by string concatenation; those events should not be public until they join the enum.

Generated narrative can still leak PII if you paste production payloads into the prompt. Keep samples synthetic. The inventory should list field names, not example addresses.

Who should not use this approach

Do not use this split if you have no public event contract and only an internal debug bus. Compiling that bus into a catalog creates a support surface you did not intend to own. Do not use it if legal retention rules live only in a wiki that engineers cannot sign in git.

Teams that already generate webhook docs from a reviewed OpenAPI file with human-edited x-delivery extensions may not need a second YAML sheet. Adopt this pattern when chat output is currently the first draft of retry language. Skip remote drafting entirely when event payloads are themselves regulated data.

The core conclusion does not change with prettier models. Enums compile. Delivery promises are signatures. Prose that cannot cite an inventory row does not belong on the catalog page.

Map CI Bootstrap Jobs to a Getting-Started Graph, Then Gate Unsigned Duration Claims

Avery Lin — Sun, 20 Sep 2026 12:07:09 +0000

Getting-started docs fail when a model invents duration, accounts, and copy-paste safety from CI YAML that never stated those promises. Compile a bootstrap command graph from workflows and scripts, then keep every timing and secret claim in a human-signed block. Unsigned phrases such as five minutes or no extra account should fail a linter before the tutorial ships. The rest of this article is a concrete extractor, a facts file, and a claim gate you can run locally.

First-run prose is a contract, not a summary

API reference can tolerate a missing example, but first-run tutorials cannot, because readers execute them as a contract. When that contract is generated from chat, the failure mode is confident setup advice that CI never validated. The damage shows up later as support tickets about missing tokens, slow image pulls, and privileged ports. This workflow treats those tickets as unsigned claims that should have been blocked in review.

Continuous integration already encodes the real bootstrap order, including checkout, toolchain install, service boot, and test invocation. A tutorial that ignores that graph will reorder steps, drop a login, or hide a Docker daemon requirement behind casual language. The useful split is therefore mechanical: the model may draft headings and command text from extracted nodes, while a human owns elapsed time, third-party accounts, and side effects. Nothing in the signed lane should be inferred from fluent prose.

What the model may draft versus what a human must own

The table below is a review gate, not a style guide. If a sentence asserts a fact from the owned column, the facts file needs a matching signed record before publish. Draftable text can still be wrong, so the extractor is an input, not an approval. Owned claims remain blocked even when the draft looks complete.

Claim class	Model may draft from repo artifacts	Human must own before publish
Step order	Makefile targets, `package.json` scripts, workflow `run` steps	That a reader can skip CI-only caching or matrix jobs
Command text	Literal shell lines and working directories	That the line is copy-paste safe on a laptop
Files created	Paths written by scripts or compose files	That those paths are safe to delete after the tutorial
Time to first success	Nothing; CI duration is not reader duration	Minutes, hours, and “quick” language
Accounts and secrets	Env var names and login subcommands	Which SaaS accounts are required, never secret values
Supported OS	Runner labels such as `ubuntu-latest`	Desktop OS support, Docker Desktop, and nested virtualization
Network and privileges	Ports in compose files, `sudo` in recipes	Bind to 443, inbound firewall, and rootless constraints
Side effects	`dropdb`, `terraform apply`, image pushes	Data loss, cost, and production-adjacent credentials

Artifact: bootstrap graph, facts file, and claim linter

The following Python module is a proposal you can run against a real repository. It does not execute commands, call a network, or parse a complete YAML grammar. It only builds a conservative inventory from Makefiles, npm scripts, and GitHub Actions run: lines, then fails a tutorial that asserts unsigned duration or account language.

#!/usr/bin/env python3
"""first_run_gate.py — proposal: inventory bootstrap commands, lint unsigned claims."""
from __future__ import annotations

import json, re, sys
from pathlib import Path

ACCOUNT_HINTS = re.compile(
    r"aws |gcloud |az |docker login|npm login|gh auth|kubectl |terraform |"
    r"ansible |ssh |sudo |vault ",
    re.I,
)
UNSIGNED = [
    (re.compile(r"\b(\d+)\s*(minute|min|hour|hr)s?\b", re.I), "duration"),
    (re.compile(r"\b(quick|simply|just|easy)\b", re.I), "ease"),
    (re.compile(r"no extra accounts?|without (an? )?account|no sign-?up", re.I), "account"),
    (re.compile(r"copy[- ]paste (and )?(it )?works|works on (any|every) (os|machine)", re.I), "portability"),
    (re.compile(r"production[- ]safe|no side effects|will not delete", re.I), "side_effect"),
]

def makefile_nodes(text: str, source: str) -> list[dict]:
    nodes = []
    target = None
    for line in text.splitlines():
        if re.match(r"^[A-Za-z0-9_.-]+:", line) and not line.startswith("\t"):
            target = line.split(":", 1)[0]
            continue
        if target and line.startswith("\t"):
            cmd = line.strip()
            nodes.append({"source": source, "id": target, "cmd": cmd,
                          "account_hint": bool(ACCOUNT_HINTS.search(cmd))})
    return nodes

def npm_nodes(text: str, source: str) -> list[dict]:
    try:
        scripts = json.loads(text).get("scripts") or {}
    except json.JSONDecodeError:
        return []
    nodes = []
    for name, cmd in scripts.items():
        nodes.append({"source": source, "id": f"npm:{name}", "cmd": cmd,
                      "account_hint": bool(ACCOUNT_HINTS.search(str(cmd)))})
    return nodes

def workflow_nodes(text: str, source: str) -> list[dict]:
    nodes = []
    step = "unnamed"
    for line in text.splitlines():
        m = re.match(r"\s*-\s*name:\s*(.+)", line)
        if m:
            step = m.group(1).strip().strip('"')
        m = re.match(r"\s*run:\s*\|?\s*(.*)$", line)
        if m and m.group(1).strip():
            cmd = m.group(1).strip()
            nodes.append({"source": source, "id": step, "cmd": cmd,
                          "account_hint": bool(ACCOUNT_HINTS.search(cmd))})
    return nodes

def collect(root: Path) -> list[dict]:
    nodes = []
    mf = root / "Makefile"
    if mf.exists():
        nodes += makefile_nodes(mf.read_text(encoding="utf-8"), str(mf))
    pkg = root / "package.json"
    if pkg.exists():
        nodes += npm_nodes(pkg.read_text(encoding="utf-8"), str(pkg))
    wf = root / ".github" / "workflows"
    if wf.is_dir():
        for yml in sorted(wf.glob("*.y*ml")):
            nodes += workflow_nodes(yml.read_text(encoding="utf-8"), str(yml))
    return nodes

def lint(md: str, signed: set[str]) -> list[str]:
    hits = []
    for i, line in enumerate(md.splitlines(), 1):
        if line.strip().startswith("> signed:"):
            continue
        for rx, kind in UNSIGNED:
            if rx.search(line) and kind not in signed:
                hits.append(f"L{i} unsigned:{kind}: {line.strip()}")
    return hits

def load_signed(path: Path) -> set[str]:
    if not path.exists():
        return set()
    kinds = set()
    for line in path.read_text(encoding="utf-8").splitlines():
        if line.startswith("signed_kinds:"):
            continue
        if line.strip().startswith("- "):
            kinds.add(line.strip()[2:].split("#", 1)[0].strip())
    return kinds

def main(argv: list[str]) -> int:
    root = Path(argv[1] if len(argv) > 1 else ".")
    nodes = collect(root)
    (root / "first_run_graph.json").write_text(
        json.dumps(nodes, indent=2), encoding="utf-8")
    print(f"wrote {len(nodes)} bootstrap nodes to first_run_graph.json")
    md_path = root / "getting-started.draft.md"
    facts = root / "first_run_signed.yml"
    if md_path.exists():
        hits = lint(md_path.read_text(encoding="utf-8"), load_signed(facts))
        for hit in hits:
            print(hit)
        return 1 if hits else 0
    return 0

if __name__ == "__main__":
    raise SystemExit(main(sys.argv))

Pair the graph with a tiny signed facts file that only a maintainer should edit. Secret values never belong in that file, only names, account types, and duration ranges measured on a named machine class. The linter treats missing kinds as failures, which is the point of the gate.

# first_run_signed.yml — human-owned; do not generate the signed_kinds list.
reader_os: ["macos-14", "ubuntu-22.04"]
required_accounts: ["GitHub", "container registry"]
secret_names: ["GH_TOKEN", "REGISTRY_PASSWORD"]
measured_first_success: "18-35 minutes on a warm DSL link after image cache"
side_effects: ["local Postgres volume created", "binds 5432"]
copy_paste_safe: false
signed_kinds:
  - duration
  - account
  - portability
  - side_effect
  # ease is intentionally omitted until someone deletes "simply" from the draft

A draft tutorial can keep command blocks that match graph nodes, but duration language must sit behind an explicit signed marker or a signed kind. The example below would fail until ease is added, because “simply” is still present.

# Getting started (draft)

> signed: duration, account, portability, side_effect

Clone the repository, then run the `bootstrap` Makefile target from the graph.
First success took 18-35 minutes on a warm DSL link after the image cache filled.
You need a GitHub account and a container registry identity; do not paste tokens here.
Do not simply skip Docker Desktop on macOS; nested virtualization is an owned claim.

Numbered workflow

Extract the graph in a clean worktree. Run python3 first_run_gate.py . on the commit you intend to document, not on an unsaved editor buffer. Commit first_run_graph.json beside the tutorial if reviewers need the same node list. Treat missing Makefiles as a missing inventory, not as permission to improvise steps.
Cluster nodes into reader-visible phases. Group toolchain install, service boot, credential prompts, and verification commands, and drop CI-only cache keys. Record skipped nodes in the facts file so a reviewer can see what the tutorial deliberately omitted. Do not let the model reinsert actions/cache as a laptop step.
Draft structure from nodes only. Headings, fenced commands, and file paths may come from a model that receives the JSON graph and nothing else. Deny the model the right to fill minutes, OS support, or “no account needed” gaps. If a node has account_hint: true, the draft must leave a placeholder instead of a reassurance.
Sign the owned lane on a measured machine. Fill first_run_signed.yml after someone actually ran the reader path, including image pulls and first-time compiler caches. Write ranges, not marketing integers, and name the OS plus network class used for the measurement. Refuse to sign copy_paste_safe when any command still interpolates a secret.
Gate publishing on the linter exit code. Run the script in CI against getting-started.draft.md and fail the job on any unsigned: line. After the facts file is signed, render the public page from the draft plus the signed block, not from a fresh chat transcript. Keep the graph and the signed file in review, because either one can rot independently.

Where free model access belongs in this loop

The drafting step needs a model that can rewrite JSON nodes into Markdown without touching the signed lane. Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode’s free model access and free server option are enough to run that drafting pass on the graph file, while the extractor and linter stay local and deterministic. Keep secret values off the server, and send only node identifiers, command strings already in git, and placeholder names from the facts file. If the model adds a minute count or an account waiver, the linter still blocks the merge, which is the control that matters.

Limitations

The parser is line-oriented, so composite GitHub Actions, reusable workflows, and dynamic matrices will be under-counted. Makefile macros, included fragments, and generated scripts will also disappear from the graph until you expand them in a pre-step. CI duration is a poor predictor of laptop duration, which is why the facts file requires a human measurement instead of a copied job timestamp. The unsigned-language list is English-centric and will miss cheerful synonyms unless you extend the patterns. This method does not prove that a command succeeded; it only proves that forbidden claims were either signed or removed.

Who should not use this approach

Skip the gate if your product setup is a GUI installer with no scripted bootstrap, because the graph will be empty and the linter will only nag adjectives. Skip it if legal or security teams already require a dedicated review of every time and data-handling sentence, and you would be duplicating that process with a weaker regex. Skip it for air-gapped readers whose first-run path cannot be measured on a machine that resembles CI. Do not use a drafting model on graphs that still contain production credentials, private registry passwords, or customer identifiers. Teams that want a single generated README with no signed file should not adopt this workflow, because the whole point is the split.

Run the extractor on one getting-started page this week, fail it on purpose with a fake five-minute claim, then sign only the kinds you actually measured. The free model access is optional for the skeleton; the signed facts file is not.

Compile Exception Inventories for Docs; Hand-Sign Impact and Retry Claims

Avery Lin — Sun, 20 Sep 2026 10:49:46 +0000

Generated troubleshooting pages fail in review when models invent recovery claims that no maintainer has actually signed. Extract exception classes and stable error codes from source, then restrict generation to headings, related symbols, and explicit placeholders. Humans must own user-visible impact, data mutation, and retry safety before any troubleshooting page is allowed to merge.

Why error catalogs are a different documentation class

Front-page README files attract review because they sit on the default repository landing path for every visitor. Troubleshooting pages sit one click deeper and often ship as generated filler after a release freeze. A model can list exception class names from an AST walk without knowing whether retrying the call duplicates a billed side effect. That gap is not a tone problem; it is an ownership split between compile facts and production claims.

Teams that treat the whole page as model output mix two evidence classes that do not share a reviewer. Structure can be compiled from the tree with a deterministic script and a stable identifier scheme. Claims about money, data loss, and retry safety cannot be compiled from syntax or message strings alone. Mixing those classes produces pages that look complete while remaining unsigned by anyone on the on-call rotation.

This workflow keeps the split mechanical by generating the inventory file and requiring humans to edit the ownership file. Continuous integration refuses the merge when those files disagree, or when draft tokens remain in a human cell. Reviewers then read a claims diff instead of a wall of generated prose that hides the unsigned fields.

The ownership matrix the checker enforces

The artifact for this workflow is a two-file contract rather than a longer generation prompt that hides ownership. The errors.inventory.json file is compiled from the application tree and must remain machine-owned after every commit. The claims.ownership.yml file is edited only by humans and is the only source for production sentences.

Field	Source	Model may draft?	Merge if unsigned?
`compile_id`	inventory script	no	no
`symbol`	inventory script	no	no
`message_template`	inventory script	no	no
`related_symbols`	model skeleton	yes, structure only	yes, as links
`user_visible_impact`	human ownership file	no	no
`mutates_data`	human ownership file	no	no
`safe_to_retry`	human ownership file	no	no
`recovery_command`	human ownership file	no	no
`signed_by` / `signed_at`	human ownership file	no	no

Keep the matrix in the repository beside the two files so reviewers do not renegotiate the split during each pull request. The checker below encodes the same rules, which keeps the table from drifting into wiki folklore. If a new field appears in generated prose, add a matrix row before teaching the model another heading style.

Step 1 — Compile the exception inventory from source

Do not ask a model to discover the error surface of a library or a production service. Walk annotated exception classes with a small script and emit stable identifiers that survive file moves only when you choose that rule. The script below is a labeled proposal for trees that store error_code as a class-level constant.

# compile_error_inventory.py
# Proposal: unexecuted example for a Python tree that tags errors with error_code.
from __future__ import annotations

import ast
import json
from pathlib import Path


class ErrorVisitor(ast.NodeVisitor):
    def __init__(self, relpath: str) -> None:
        self.relpath = relpath
        self.rows: list[dict] = []

    def visit_ClassDef(self, node: ast.ClassDef) -> None:
        bases = [ast.unparse(b) for b in node.bases]
        if not any("Error" in b or "Exception" in b for b in bases):
            self.generic_visit(node)
            return
        code = None
        template = None
        for stmt in node.body:
            if not isinstance(stmt, ast.Assign):
                continue
            for target in stmt.targets:
                if isinstance(target, ast.Name) and target.id == "error_code":
                    if isinstance(stmt.value, ast.Constant):
                        code = stmt.value.value
                if isinstance(target, ast.Name) and target.id == "message_template":
                    if isinstance(stmt.value, ast.Constant):
                        template = stmt.value.value
        if code:
            self.rows.append(
                {
                    "compile_id": f"{self.relpath}:{node.name}:{code}",
                    "symbol": node.name,
                    "error_code": code,
                    "message_template": template or "",
                    "path": self.relpath,
                    "lineno": node.lineno,
                }
            )
        self.generic_visit(node)


def compile_inventory(root: Path) -> list[dict]:
    rows: list[dict] = []
    for path in sorted(root.rglob("*.py")):
        tree = ast.parse(path.read_text(encoding="utf-8"))
        visitor = ErrorVisitor(str(path.relative_to(root)))
        visitor.visit(tree)
        rows.extend(visitor.rows)
    rows.sort(key=lambda r: r["compile_id"])
    return rows


if __name__ == "__main__":
    import argparse

    parser = argparse.ArgumentParser()
    parser.add_argument("--root", type=Path, required=True)
    parser.add_argument("--out", type=Path, required=True)
    args = parser.parse_args()
    payload = {"errors": compile_inventory(args.root)}
    args.out.parent.mkdir(parents=True, exist_ok=True)
    args.out.write_text(json.dumps(payload, indent=2) + "\n", encoding="utf-8")

Run the compiler before any generation step so the model never becomes the source of identifiers.

python compile_error_inventory.py --root src --out docs/_generated/errors.inventory.json
git diff -- docs/_generated/errors.inventory.json

The inventory may include names, file paths, line numbers, and literal message templates copied from constants. Compiled inventory rows must omit severity labels, blast radius, billed side effects, and any safe-to-retry boolean flag. Those omitted fields are production claims, and production claims do not belong in a compiled inventory file.

Step 2 — Restrict the model to a draftable skeleton

A model may draft the troubleshooting skeleton from the inventory after the compile step has finished. Allowed outputs are H2 headings per compile_id, related-symbol lists, and placeholders that name each human cell. Forbidden outputs include recovery shell commands, payment or deletion warnings, SLA numbers, and any sentence that asserts production behavior.

The following prompt is a labeled proposal, not an executed transcript from a production run.

You are drafting STRUCTURE only for troubleshooting.md.
Read docs/_generated/errors.inventory.json.
For each compile_id, emit:

## {symbol} (`{error_code}`)
Related symbols: (list other inventory symbols that share a path prefix)
{{CLAIM:user_visible_impact:{compile_id}}}
{{CLAIM:mutates_data:{compile_id}}}
{{CLAIM:safe_to_retry:{compile_id}}}
{{CLAIM:recovery_command:{compile_id}}}

Rules:
- Do not invent error codes.
- Do not fill CLAIM tokens.
- Do not write shell commands.
- Do not assert that an operation is safe, billed, or destructive.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode's free model access and free server option can host this draft step when a maintainer wants the skeleton produced off-laptop. The ownership file and the checker still run in the same repository hooks you already use for tests. Do not treat model output as a signed claim even when the troubleshooting draft looks fluent and complete.

After the draft step, the model should write only the structure file and must leave every CLAIM token untouched. Reviewers should reject a skeleton that contains a verb like retry, delete, refund, or safe outside a claim token. A related-symbol list is structure; a recommended shell pipeline is a claim and belongs in the ownership file.

Step 3 — Hand-write the five human claim fields

For every compile_id in the compiled inventory, a maintainer writes five fields into docs/claims.ownership.yml by hand. None of these fields may be pasted from chat output, because the checker cannot prove origin beyond the signed_by value you type. Paste detection is a review habit here, not a cryptographic proof of authorship for the signed row.

# docs/claims.ownership.yml
# Human-owned. Do not generate this file.
version: 1
claims:
  - compile_id: "payments/errors.py:ChargeConflictError:PAY_CONFLICT"
    user_visible_impact: "The customer may see a pending charge without an order confirmation."
    mutates_data: true
    safe_to_retry: false
    recovery_command: "bin/payments reconcile --charge-id <id> --dry-run"
    signed_by: "maintainer-id"
    signed_at: "2026-09-20"
  - compile_id: "payments/errors.py:IdempotencyReplayError:PAY_IDEMPOTENT"
    user_visible_impact: "The API returns the original receipt; the customer should not be double charged."
    mutates_data: false
    safe_to_retry: true
    recovery_command: "true  # no-op; return stored receipt"
    signed_by: "maintainer-id"
    signed_at: "2026-09-20"

Numbered rules for the human pass:

Set mutates_data from storage code and side-effect logs, not from the exception class name alone during review.
Set safe_to_retry only when a replay cannot create a second durable write or a second bill.
Put the recovery command in a form an on-call engineer can paste, including a dry-run flag when writes exist.
Refuse to sign a row that still contains TODO, a claim token, or wording copied from the skeleton file.

Unsigned rows are not partial progress toward a merge; they are missing evidence and must block the renderer. A true value for mutates_data without a dry-run recovery command should fail human review even if the checker only looks for nonempty strings. Date stamps use the calendar day of the signature, not the day the skeleton was generated.

Step 4 — Render only after both files agree

A renderer concatenates the structure file and the ownership file into the published troubleshooting page in git. It must exit nonzero if a claim token remains or if an inventory id lacks a matching ownership row. It must also exit when an ownership row points at a compile_id that the inventory no longer emits.

# render_troubleshooting.py
# Proposal: unexecuted example.
from __future__ import annotations

import json
import re
import sys
from pathlib import Path

import yaml

TOKEN = re.compile(r"\{\{CLAIM:([a-z_]+):([^}]+)\}\}")


def load_claims(path: Path) -> dict[str, dict]:
    data = yaml.safe_load(path.read_text(encoding="utf-8"))
    return {row["compile_id"]: row for row in data["claims"]}


def render(structure: str, claims: dict[str, dict], inventory_ids: set[str]) -> str:
    missing = inventory_ids - set(claims)
    extra = set(claims) - inventory_ids
    if missing or extra:
        raise SystemExit(
            f"claim/inventory mismatch missing={sorted(missing)} extra={sorted(extra)}"
        )

    def repl(match: re.Match[str]) -> str:
        field, compile_id = match.group(1), match.group(2)
        row = claims[compile_id]
        if field not in row or row[field] in (None, "", "TODO"):
            raise SystemExit(f"unsigned field {field} for {compile_id}")
        value = row[field]
        if field == "mutates_data":
            return "Mutates durable data: yes" if value else "Mutates durable data: no"
        if field == "safe_to_retry":
            return "Safe to retry: yes" if value else "Safe to retry: no"
        return str(value)

    rendered = TOKEN.sub(repl, structure)
    if "{{CLAIM:" in rendered:
        raise SystemExit("unresolved claim token")
    if not all(row.get("signed_by") and row.get("signed_at") for row in claims.values()):
        raise SystemExit("every claim row needs signed_by and signed_at")
    return rendered


if __name__ == "__main__":
    inventory = json.loads(Path("docs/_generated/errors.inventory.json").read_text())
    ids = {row["compile_id"] for row in inventory["errors"]}
    structure = Path("docs/_generated/troubleshooting.structure.md").read_text()
    claims = load_claims(Path("docs/claims.ownership.yml"))
    out = render(structure, claims, ids)
    Path("docs/troubleshooting.md").write_text(out, encoding="utf-8")
    print("rendered docs/troubleshooting.md", file=sys.stderr)

python compile_error_inventory.py --root src --out docs/_generated/errors.inventory.json
python render_troubleshooting.py
python - <<'PY'
from pathlib import Path
text = Path("docs/troubleshooting.md").read_text()
assert "{{CLAIM:" not in text
assert "Safe to retry:" in text
print("ownership gate passed")
PY

Wire the compile and render commands in the same job so a skeleton cannot land without signed claims. The final Python assertion is a second check for leftover tokens after a successful render in CI. Failing this job in CI is cheaper than publishing an unsigned recovery command during an incident.

Step 5 — Read the gate output as a review checklist

When the checker fails, treat the output as a review queue rather than as a writing task for the model. A missing compile_id usually means a new exception shipped into production without an on-call sentence attached. An extra claim row means a deleted error still has recovery text that will mislead operators during an incident. An unsigned field means the skeleton is still a draft and is not yet an operator document.

Reviewers should inspect diffs of claims.ownership.yml with the same attention they already give to migration SQL. A flipped safe_to_retry bit is a behavior change for anyone following the page during an incident. Do not accept a claims diff that only updates signed_at without a corresponding inventory or product change.

Limitations

This method assumes exception classes carry a stable error_code constant that the AST visitor can read. Dynamic error factories, stringly-typed codes, and errors raised from generated protobuf stubs will not appear until you extend the compiler. The checker proves presence and signature metadata, and it does not prove that mutates_data matches production. Wrong human claims will still merge if a maintainer types a name under signed_by without reading the storage path.

Message templates that interpolate untrusted input can leak into the inventory file during the compile step. Keep templates as format strings, and strip interpolated values before compile output is committed to git. The renderer does not sanitize recovery commands, so a signed malicious command is still a malicious command in the page.

Free model drafts can drift across runs even when the inventory is unchanged between those runs. Commit the skeleton only when the heading set matches compile_id values from the current inventory file. Ignore wording churn in related-symbol lists unless a reviewer explicitly asked for that cross-link change.

Who should not use this approach

Do not use this workflow on repositories that have no durable side effects and no operator recovery path, because the ownership file becomes ceremony. Do not use it as a substitute for runbooks that live in an incident tool with paging and ownership rotations. Do not point the model at production logs to fill in claim fields, because logs are evidence for humans. Mutation flags belong in the ownership file after a human reads those logs, not in the draft prompt.

Skip the generator entirely when the error surface is smaller than roughly a dozen codes and a maintainer already hand-writes the page. The compile step still helps, but the skeleton model adds a moving file without reducing review load. Teams that cannot name a signed_by identity should not enable the merge gate, because the signature field would become a rubber stamp.

The useful split in this documentation workflow stays narrow and mechanical across both libraries and services. Models may draft page structure from compiled identifiers, including headings, related symbols, and empty claim tokens. Humans own impact, mutation, retry safety, and the recovery command that operators will paste during an incident. If you already keep documentation checks in CI, add the ownership files next to them before expanding generation.

Extract a Migration Inventory From Git Diffs, Then Lint Breaking-Change Claims

Avery Lin — Sun, 20 Sep 2026 07:29:27 +0000

A documentation generator may draft inventories from diffs, but it cannot own breaking-change labels or support duration. Treat every compatibility sentence as a human-signed claim, not as autocomplete output from a model. The workflow below extracts a symbol inventory, drafts a migration skeleton, then lints claim language before publication. Teams that skip the last step publish promises the repository cannot actually keep.

The failure mode this workflow targets

Migration guides fail when generated prose restates a diff as a guarantee about downstream builds. A removed export is an observable fact; “this release is backward compatible” is a product decision with support cost. Models compress those two classes into the same paragraph because both look like English. Reviewers then argue about tone instead of about the claim that will trap a release manager.

Git history remains a useful source for inventories because it is cheap, local, and reproducible without a network. It is a poor source for severity, because a one-line rename can break more callers than a large internal rewrite. The method therefore splits work into extract, draft, classify, and lint, with different owners at each stage. Keep those stages in separate files so a fluent paragraph cannot hide an unsigned guarantee.

What a model may draft

A model may rearrange extracted rows into readable sections, propose heading order, and expand a symbol name into a short description. It may also suggest a checklist of caller updates when the inventory already lists the old and new identifiers. None of those drafts should include dates of support, “safe to upgrade” language, or security impact. If the draft needs a verb that implies a promise, replace it with a placeholder token before anyone reads the Markdown as a guide.

Keep the draft input narrow: a YAML inventory, a version pair, and a list of files that changed. Do not paste changelog marketing, issue-tracker sentiment, or unreviewed chat summaries into that prompt. Wider context increases fluency while also increasing unauthorized claims about compatibility. Label the prompt as a proposal, and store the exact inventory hash beside the draft so reviewers can see drift.

What a human must own

A human must label each inventory row as breaking, behavior, deprecation, or docs-only using the public contract, not the size of the diff. A human must write the upgrade command that the project will actually support, including the rollback path if the upgrade fails. A human must set the support window for the old symbol, or explicitly state that no window exists. Those three fields are the release, not decorative metadata around generated headings.

Security, licensing, and data-retention sentences are never draftable under this method. If the model emits them, the linter must fail the build rather than ask a reviewer to notice. Compatibility with named downstream frameworks is also human-owned, because it requires evidence from a test matrix the generator did not run. Unsigned sentences in those classes should block the tag, not wait for a documentation follow-up issue.

Workflow

1. Freeze the version pair and the public surface

Record the previous tag and the candidate revision before any prose is written. Export the public surface from tests or from an explicit __all__ list so the inventory cannot drift into private helpers. Store both facts in a small directory that CI can hash on every documentation job.

mkdir -p .migration
git rev-parse v1.4.0 > .migration/from.rev
git rev-parse HEAD    > .migration/to.rev
python -c "import json, mypkg; print(json.dumps(sorted(mypkg.__all__)))" \
  > .migration/public_surface.json
sha256sum .migration/* > .migration/inputs.sha256

That trio is the only allowed source for later extraction. If a symbol is absent from public_surface.json, it does not belong in the customer migration guide. Private helpers can still change; they simply do not earn a compatibility sentence.

2. Extract a symbol inventory from the diff

Run a deterministic extractor against the two revisions and write YAML, not Markdown. Markdown invites early narration; YAML keeps rows classifiable later. The script below is a starting point for Python packages that export names from a single surface file. Treat it as a labeled example, not as a parser for every language in a monorepo.

# extract_migration_inventory.py — proposal for a single Python export surface
import ast, json, subprocess, sys, yaml
from pathlib import Path

SURFACE = Path(".migration/public_surface.json")
FROM_REV = Path(".migration/from.rev").read_text().strip()
TO_REV = Path(".migration/to.rev").read_text().strip()
TARGET = "src/mypkg/__init__.py"

def names_at(rev: str) -> set[str]:
    blob = subprocess.check_output(["git", "show", f"{rev}:{TARGET}"], text=True)
    tree = ast.parse(blob)
    public = set(json.loads(SURFACE.read_text()))
    found = set()
    for node in tree.body:
        if isinstance(node, ast.Assign):
            for t in node.targets:
                if isinstance(t, ast.Name) and t.id in public:
                    found.add(t.id)
        if isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef, ast.ClassDef)):
            if node.name in public:
                found.add(node.name)
    return found

old, new = names_at(FROM_REV), names_at(TO_REV)
rows = []
for name in sorted(old | new):
    if name in old and name not in new:
        kind = "removed"
    elif name not in old and name in new:
        kind = "added"
    else:
        kind = "unchanged"
    if kind == "unchanged":
        continue
    rows.append({
        "symbol": name,
        "change": kind,
        "class": None,          # human-owned
        "support_window": None, # human-owned
        "rollback": None,       # human-owned
        "owned_by": None,
    })
Path(".migration/inventory.yaml").write_text(yaml.safe_dump(rows, sort_keys=False))
print(f"wrote {len(rows)} rows", file=sys.stderr)

Run it from a clean worktree so uncommitted edits cannot leak into the inventory. Commit inventory.yaml with empty human fields rather than with guessed classes. Empty fields are a visible queue; guessed classes look finished and skip review.

python extract_migration_inventory.py
git add .migration/inventory.yaml

3. Draft only the narrative scaffolding

Hand the YAML to a drafting model with a hard instruction: no compatibility verbs, no version support dates, no “users should be safe.” The allowed output is headings, bullet restatements of rows, and placeholder tokens such as {{SUPPORT_WINDOW}}. Placeholder tokens make missing human work visible in review, which is more useful than a polished paragraph that hides a missing owner.

Proposal prompt, not an executed production prompt:

Using only inventory.yaml, write MIGRATION.draft.md.
Restate each row as a bullet. Do not classify breakingness.
Do not write support dates, rollback advice, or security impact.
Insert {{CLASS}}, {{SUPPORT_WINDOW}}, and {{ROLLBACK}} tokens
wherever a human decision is required. Quote symbol names verbatim.

If you use a hosted editor for that draft pass, keep the inventory file as the only attachment. Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode's free model access and free server option can generate that scaffolding from the YAML inventory without requiring a paid seat for the draft step. The classification table and the linter still run locally in CI, which is the point of splitting ownership.

4. Classify every row with a decision table

Fill class, support_window, rollback, and owned_by by hand. Do not accept model-suggested classes unless a test in the repository demonstrates the behavior change. The table below is the entire policy for this workflow, and it should live next to the inventory rather than in a style guide nobody opens during a release.

Evidence in the two revisions	Required class	Who may write the sentence	Block merge if missing
Public symbol removed or renamed	`breaking`	Human	Support window and rollback
Same signature, default or error mapping changed, proven by a test	`behavior`	Human	Test path cited in the row
Symbol still present, documented as going away later	`deprecation`	Human	Removal version or “no date”
Docs, examples, or heading text only	`docs-only`	Model draft allowed	None
Auth, crypto, privacy, license, or personal data	never draftable	Human only	Entire section
Named downstream framework compatibility	never from a diff	Human only	CI matrix job name

After classification, copy tokens out of the draft and into a second file named MIGRATION.md. Keep MIGRATION.draft.md unshipped so generated scaffolding cannot be installed as the user guide. The owned file is what packaging, GitHub releases, and the documentation site may render.

# .migration/inventory.yaml — human fields filled after the draft pass
- symbol: load_client
  change: removed
  class: breaking
  support_window: "none; removed in 2.0.0"
  rollback: "pin mypkg==1.4.x and import load_client"
  owned_by: "release-owner@example.com"
- symbol: Client.from_config
  change: added
  class: docs-only
  support_window: n/a
  rollback: n/a
  owned_by: "release-owner@example.com"

5. Lint claim language before merge

Scan the owned Markdown for claim classes that require a signature. Fail CI when a sentence matches and the adjacent HTML comment lacks owned-by. This is mechanical, not stylistic, and it should run on the same job that builds the documentation site.

# lint_migration_claims.py
import re, sys
from pathlib import Path

CLAIM = re.compile(
    r"\b(backward compatible|backwards-compatible|safe to upgrade|"
    r"no breaking changes|always|never|guaranteed|supported until|"
    r"security|personally identifiable|GDPR|SOC 2)\b",
    re.I,
)
OWNED = re.compile(r"<!-- owned-by: .+ -->")

text = Path("MIGRATION.md").read_text()
failures = []
for i, line in enumerate(text.splitlines(), 1):
    if CLAIM.search(line) and not OWNED.search(line):
        failures.append(f"L{i}: unsigned claim: {line.strip()}")

if failures:
    print("\n".join(failures))
    sys.exit(1)
print("migration claims signed")

python lint_migration_claims.py
# wire the same command into documentation CI, not into a laptop alias

A signed line looks like the following example, which CI can grep without parsing English. Unsigned synonyms will still slip through, so treat the regex as a net, not as a contract lawyer.

`load_client` is removed in 2.0.0 with no support window.
<!-- owned-by: release-owner@example.com -->

A compact owned guide, after lint

The shipped file should read as inventory plus signed decisions, not as a narrative of the sprint. The example below is short on purpose, because extra adjectives are where unsigned claims hide. Expand only after the linter is green.

# Migration 1.4.x to 2.0.0

## Breaking

- Replace `load_client(...)` with `Client.from_config(...)`.
  Support window: none; removed in 2.0.0.
  Rollback: pin `mypkg==1.4.x`.
  <!-- owned-by: release-owner@example.com -->

## Added

- `Client.from_config` is the supported constructor for file-based setup.

Limitations

The extractor sees names, not behavior. A function that keeps its signature while changing a default timeout will not appear as breaking. The linter sees words, not legal meaning, so a careful paraphrase can evade the regex. Neither tool replaces a contract test suite that installs the previous client against the new server.

Git diffs also miss generated code and submodule pins unless those paths are included explicitly. Teams with multiple public languages need one inventory per language, because a Python rename does not describe the REST resource. The method assumes a tagged history; trunk-only repositories must invent a comparable freeze, or the version pair is fiction.

Who should not use this approach

Do not use this workflow to generate security advisories, SLA text, or statements about personal data. Do not use it when the public contract lives only in tribal knowledge and there is no __all__, OpenAPI file, or export map. Do not use it as evidence that a release is compatible because the linter passed.

Internal tools with a single caller may skip the prose draft entirely and ship the YAML inventory. Regulated products should treat every migration sentence as human-owned and should not introduce a model into the path at all. If the team cannot name an owner email for breaking rows, stop before the draft step.

Closing

Migration documentation stays honest when inventories are compiled and claims are signed as separate artifacts. Keep the generator on headings and restatements, and keep humans on breaking labels, support windows, and rollback. If a workspace already drafts the inventory sections, store the ownership table and the linter in the repository rather than in a chat transcript.

Treat Troubleshooting Docs as Two Files: an Extracted Fault Catalog and a Signed Recovery Matrix

Avery Lin — Sun, 20 Sep 2026 04:23:32 +0000

Generated troubleshooting pages fail in review when draft descriptions and user-facing outcomes occupy the same markdown file. A catalog of exception types, status codes, and log event names can be extracted, then drafted, without promising any user outcome. Recovery language, retry guidance, and data-loss statements remain a human-owned matrix that continuous integration can refuse to publish unsigned. The remainder of this article specifies a two-file pipeline, a YAML schema, and a test that enforces the split.

Mixed ownership, not missing prose, is the failure mode

Most teams regenerate troubleshooting sections from a single prompt over the repository or from unreviewed chat output. That process mixes three statement kinds that do not share a source of truth: mechanical inventory, plausible description, and operational promise. Inventory can be compiled from raise sites and error handlers, while description can be drafted and later edited. Operational promise cannot be inferred from a stack trace, an HTTP mapping, or a short model summary.

When those three statement kinds share one file, reviewers cannot tell derived sentences from product warranties. A regenerated paragraph can change retry advice into a rollback claim without any test noticing the swap. The published page then disagrees with the transaction boundary that the running service actually implements. Splitting the catalog from the recovery matrix turns that disagreement into a build failure instead of a reading exercise.

Ownership matrix for error-path documentation

Use the following table as the publishing contract for every troubleshooting page that will ship to users. Rows a model may draft still require a human to accept or rewrite the sentence before merge. Rows marked human-owned must never be emitted into the generated catalog file at all. Cheap CI checks cover only a subset; refunds and availability claims still need a named owner.

Statement class	Source of truth	Model may draft?	Human must own	Publish gate
Exception class inventory	AST or `raise` sites	No; extract only	Confirm coverage	Fail if source fault is unsigned
HTTP status mapping	Framework error handlers	No; extract only	Confirm mapping	Fail if a mapped status is missing
One-sentence mechanical description	Extracted name plus handler	Yes; draft only	Edit for accuracy	Fail if `reviewed` is not true
Retryable or idempotent?	Transaction and queue code	No	Yes	Fail if the field is missing
Data committed or discarded	Persistence boundary	No	Yes	Fail if the field is missing
User-visible recovery steps	Product and support policy	No	Yes	Fail if the field is missing
Support window or SLA language	Contract, not code	No	Yes	Fail if present in generated files

The table is the policy artifact, and the scripts below enforce only those checks that remain cheap in CI. Dynamic exceptions, multi-language services, and dependency faults will not appear until extractors are extended. Banned-verb filters are not proof of safety; they only stop the most obvious outcome language in drafts.

Workflow: extract, draft, sign, then render

Keep the four stages serial so a draft can never publish itself as a recovery instruction. Each stage writes or updates one file, and later stages refuse to run when earlier files are incomplete. The commands assume a Python service tree; adapt the extractor if handlers live in another language.

1. Extract a fault catalog from source, not from chat

Run a deterministic extractor over the service package and write faults.generated.yaml before any model runs. Do not ask a model to list errors, because listing is a compile step with a verifiable source. Keep the generated file out of the published site until every description is reviewed and every fault has a signed recovery row.

# extract_faults.py — proposed extractor, not a production-grade scanner
import ast
import pathlib
import sys
import yaml

ROOT = pathlib.Path(sys.argv[1])
rows = []

class RaiseVisitor(ast.NodeVisitor):
    def __init__(self, path):
        self.path = path
        self.found = []

    def visit_Raise(self, node):
        name = None
        if isinstance(node.exc, ast.Call) and isinstance(node.exc.func, ast.Name):
            name = node.exc.func.id
        elif isinstance(node.exc, ast.Name):
            name = node.exc.id
        if name:
            self.found.append({
                "fault_id": name,
                "path": str(self.path),
                "lineno": node.lineno,
                "draft_description": "",
                "reviewed": False,
            })
        self.generic_visit(node)

for py in ROOT.rglob("*.py"):
    if "tests" in py.parts or py.name.startswith("test_"):
        continue
    tree = ast.parse(py.read_text(encoding="utf-8"), filename=str(py))
    visitor = RaiseVisitor(py)
    visitor.visit(tree)
    rows.extend(visitor.found)

seen = set()
unique = []
for row in rows:
    if row["fault_id"] in seen:
        continue
    seen.add(row["fault_id"])
    unique.append(row)

yaml.safe_dump({"faults": unique}, open("faults.generated.yaml", "w"), sort_keys=False)

python extract_faults.py ./src

The extractor records locations and empty description fields; it does not invent HTTP mappings, retry policy, or user guidance. Label extra heuristics, such as status-code guesses from nearby integers, as proposals until they are checked against real handlers. If your service raises faults from a shared errors package, point the extractor at that package rather than at every caller.

2. Draft descriptions on a disposable lane

Feed only the generated catalog, with secret-bearing source removed, into a drafting job that fills draft_description. Limit each draft to one mechanical sentence that restates the fault name and the raising module. The job must not write retry guidance, rollback claims, support hours, or any sentence that tells a user what happened to their data.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. A drafting job of this kind can run against MonkeyCode's free model access on the free server option. That placement keeps the catalog off a workstation GPU and off a billed API key. Do not send production connection strings, customer traces, or unsigned legal text into that lane.

After the job returns, a reviewer sets reviewed: true only when the sentence matches the raise site. Unreviewed descriptions stay unpublished even if the YAML file is committed for inspection. Treat the lane as a draft printer, not as an owner of user outcomes, and do not store customer incidents there.

3. Maintain a human-owned recovery matrix

Create recovery.signed.yaml by hand and keep it out of any regenerate script. Every fault_id in the generated catalog must have a matching signed row before a troubleshooting page may render. The dates below are sample signatures for this article, not production metrics or measured incident rates.

# recovery.signed.yaml — human-owned; never regenerated
version: 1
rows:
  - fault_id: PaymentCaptureError
    retryable: false
    data_outcome: "capture request not committed; wallet balance unchanged"
    user_recovery: "Re-open checkout and submit once; do not retry in a loop."
    signer: "payments-oncall"
    signed_at: "2026-09-20"
  - fault_id: InventoryReservationTimeout
    retryable: true
    data_outcome: "reservation lease expired; no stock decrement"
    user_recovery: "Retry after 5 seconds; abandon after three attempts."
    signer: "inventory-oncall"
    signed_at: "2026-09-20"

Replace signers with the team that owns the transaction boundary, not with the person who ran the extractor. If a fault has no owner, it has no published recovery language and the page should omit it. Adding a plausible recovery sentence without a signer is the failure mode this pipeline exists to block.

4. Gate publish with a coverage test, then render without a model

The tests below fail when source contains an extracted fault that lacks a signed recovery row, or when a generated description is still unreviewed. They also fail when the generated file contains banned outcome verbs that belong only in the signed matrix.

# test_fault_docs.py — proposed CI gate
import pathlib
import re

import pytest
import yaml

BANNED_IN_GENERATED = re.compile(
    r"\b(retry|roll(?:ed)? back|data loss|SLA|uptime|never lose)\b",
    re.I,
)


def test_every_extracted_fault_has_a_signed_recovery():
    generated = yaml.safe_load(pathlib.Path("faults.generated.yaml").read_text())
    signed = yaml.safe_load(pathlib.Path("recovery.signed.yaml").read_text())
    signed_ids = {row["fault_id"] for row in signed["rows"]}
    missing = [
        f["fault_id"]
        for f in generated["faults"]
        if f["fault_id"] not in signed_ids
    ]
    assert missing == [], f"unsigned faults: {missing}"


def test_generated_descriptions_are_reviewed_and_non_promising():
    generated = yaml.safe_load(pathlib.Path("faults.generated.yaml").read_text())
    for fault in generated["faults"]:
        assert fault.get("reviewed") is True, fault["fault_id"]
        text = fault.get("draft_description") or ""
        assert not BANNED_IN_GENERATED.search(text), fault["fault_id"]


def test_signed_rows_name_outcome_and_recovery():
    signed = yaml.safe_load(pathlib.Path("recovery.signed.yaml").read_text())
    for row in signed["rows"]:
        assert row.get("data_outcome"), row["fault_id"]
        assert row.get("user_recovery"), row["fault_id"]
        assert row.get("signer"), row["fault_id"]
        assert isinstance(row.get("retryable"), bool), row["fault_id"]

python -m pytest test_fault_docs.py -q

Render markdown only after those tests pass, and keep the renderer free of model calls. Concatenate the reviewed description with the signed recovery columns using a small template, then commit the rendered page as a build output rather than as a chat paste.

# proposed render contract — not an executed benchmark
for fault in reviewed_catalog:
    rec = signed_index[fault.fault_id]
    write("## " + fault.fault_id)
    write(fault.draft_description)
    write("- Retryable: " + rec.retryable)
    write("- Data outcome: " + rec.data_outcome)
    write("- What to do: " + rec.user_recovery)

Reviewer checklist before merge

Confirm each fault_id still exists at the recorded path, or delete the catalog row and the signed row together.
Confirm retryable matches the queue or HTTP idempotency rules in the handler, not the draft sentence.
Confirm data_outcome matches the persistence boundary, including partial writes and lease expiry.
Reject any generated description that tells the user to retry, wait on an SLA, or assume a rollback.

A reviewer who cannot name the transaction boundary should leave reviewed false and leave the recovery row unsigned. Unpublished faults are cheaper than a troubleshooting page that invents a rollback the ledger never performed. The checklist is the human counterpart to the pytest file, not a substitute for it.

Limitations and who should not use this

The extractor will miss dynamically constructed exceptions, errors raised in other languages, and faults that exist only inside a dependency. The banned-verb regex is a coarse filter, not a proof that a description is safe to show a customer. The recovery matrix can still be wrong if the signer misunderstands isolation, leases, or outbox delivery.

Do not use this approach for incident reports that include customer data, for security advisories, or for legally binding availability claims. Those documents need counsel and an incident commander, not a draft lane. Teams that lack a named owner per fault should keep troubleshooting pages unpublished rather than auto-filling recovery steps from a model.

Closing check

Before the next release, extract the fault catalog, draft descriptions on a throwaway lane, and refuse to publish any row that lacks a signer. The useful output is not more troubleshooting prose; it is a list of faults the team is not yet willing to stand behind.

Stamp Changelog Claims as Extract, Draft, or Sign Before Publishing Notes

Avery Lin — Sun, 20 Sep 2026 01:15:54 +0000

Generated release notes fail when a model is allowed to classify breakage or promise an upgrade path. A safer workflow extracts public symbols with ordinary git and AST tooling before any prose is drafted. Models may rewrite extracted inventories into readable summaries, but they must not assign compatibility class or rollback language. Humans still own deprecation dates, data-migration steps, and every sentence a user would treat as a promise.

Why generated changelogs smear authority

Release notes mix three kinds of statement that look similar in Markdown, yet they do not share a source of truth. Symbol lists can be compiled from two git trees with a parser, so they are cheap to regenerate after every tag. Narrative summaries of those lists can be drafted by a model because they only restate already extracted rows. Compatibility claims cannot be compiled from a diff, because a removed helper might be public API or an accidental export.

Teams blur those classes when they paste a chat transcript into CHANGELOG.md and ship the tag. Readers then treat a fluent paragraph as a contract, including sentences about zero downtime or lossless rollback. The failure is not that the prose is ugly; the failure is that authority was never stamped on each claim.

The three authorities

This workflow assigns every changelog row exactly one authority before any drafting work starts in the repository. Mixing those authorities in one Markdown heading is what makes generated notes look finished while remaining unprovable. Keep the labels in the working file, not in a reviewer's memory after the tag exists.

EXTRACT rows must be rebuilt by a script from git objects, AST exports, or collected tests, and never from a prompt transcript.
DRAFT rows may be written by a model as restatements of EXTRACT rows, and they remain unpublished until a reviewer accepts the wording.
SIGN rows must be written by a named human owner, covering breakage class, migration, rollback policy, and deprecation calendar language.

A model that fills a SIGN row is not accelerating documentation but forging a signature the repository cannot prove. Treat that event as a broken build rather than as a wording issue for later cleanup.

Artifact: a stamped changelog map

Keep the working document out of CHANGELOG.md until the checker passes against the stamped map. Store an explicit authority map beside the tag, for example docs/changelog/v2.5.0.yaml in the same repository. Each row names its authority, its evidence path, and whether a model was allowed to touch it.

# docs/changelog/v2.5.0.yaml
version: "2.5.0"
compared_to: "2.4.0"
rows:
  - id: symbols.added
    authority: EXTRACT
    evidence: build/symbol_diff.json
    body: []
  - id: symbols.removed
    authority: EXTRACT
    evidence: build/symbol_diff.json
    body: []
  - id: narrative.summary
    authority: DRAFT
    evidence: "symbols.added,symbols.removed"
    drafted_by: null
    body: ""
  - id: compat.class
    authority: SIGN
    owner: ""
    allowed_values: ["compatible", "breaking", "mixed"]
    body: ""
  - id: migration.steps
    authority: SIGN
    owner: ""
    body: ""
  - id: rollback.policy
    authority: SIGN
    owner: ""
    body: ""
  - id: deprecation.timeline
    authority: SIGN
    owner: ""
    body: ""

Empty SIGN bodies are a passing state, while model-filled SIGN bodies are a failing state that blocks the tag. Operators should treat that distinction as the whole gate, not as optional review advice. The map may still publish without migration text when the extractor reports no removed public names.

Step 1: Extract a public-symbol inventory

Label the following script as a worked example, not as a production coverage claim for every language. It compares two annotated tags in a Python tree and records names that look public at module scope. It does not decide whether a deletion is breaking, because that decision needs product intent the parser cannot see.

#!/usr/bin/env python3
"""Extract public function and class names from two git tags. Proposal example."""
from __future__ import annotations

import ast
import json
import subprocess
import sys
from pathlib import Path


def git_show(tag: str, path: str) -> str | None:
    proc = subprocess.run(
        ["git", "show", f"{tag}:{path}"],
        check=False,
        capture_output=True,
        text=True,
    )
    if proc.returncode != 0:
        return None
    return proc.stdout


def public_names(source: str) -> set[str]:
    tree = ast.parse(source)
    names: set[str] = set()
    for node in tree.body:
        if isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef, ast.ClassDef)):
            if not node.name.startswith("_"):
                names.add(node.name)
    return names


def list_py_files(tag: str) -> list[str]:
    proc = subprocess.run(
        ["git", "ls-tree", "-r", "--name-only", tag],
        check=True,
        capture_output=True,
        text=True,
    )
    return [
        p for p in proc.stdout.splitlines()
        if p.endswith(".py") and "/tests/" not in p
    ]


def inventory(tag: str) -> set[str]:
    found: set[str] = set()
    for path in list_py_files(tag):
        src = git_show(tag, path)
        if not src:
            continue
        try:
            found |= {f"{path}::{n}" for n in public_names(src)}
        except SyntaxError:
            continue
    return found


def main() -> int:
    if len(sys.argv) != 3:
        print("usage: extract_symbol_diff.py <old-tag> <new-tag>", file=sys.stderr)
        return 2
    old_tag, new_tag = sys.argv[1], sys.argv[2]
    old, new = inventory(old_tag), inventory(new_tag)
    payload = {
        "old_tag": old_tag,
        "new_tag": new_tag,
        "added": sorted(new - old),
        "removed": sorted(old - new),
        "unchanged_count": len(old & new),
    }
    Path("build").mkdir(exist_ok=True)
    Path("build/symbol_diff.json").write_text(json.dumps(payload, indent=2))
    print(json.dumps(
        {k: (len(v) if isinstance(v, list) else v) for k, v in payload.items()},
        indent=2,
    ))
    return 0


if __name__ == "__main__":
    raise SystemExit(main())

Run the extractor against annotated tags, not against an uncommitted working tree, so later reviewers can rebuild the same evidence object. Point it at the tags users actually install, rather than at a feature branch that never becomes an artifact.

python3 extract_symbol_diff.py v2.4.0 v2.5.0

The JSON file is the only input a later drafting step may read for this workflow. Chat logs, issue titles, and marketing copy stay out of the EXTRACT rows because they are not reconstructible from git objects.

Step 2: Attach authority stamps

Load the YAML map and copy added and removed names into EXTRACT bodies without touching SIGN fields. Leave SIGN bodies blank on purpose after a large diff, because silence is cheaper to review than a generated compatibility story. If a removed symbol exists, the checker later demands a human compatibility class so hidden breakage cannot ship as empty prose.

#!/usr/bin/env python3
"""Stamp EXTRACT bodies from symbol_diff.json. Leaves SIGN rows untouched."""
from __future__ import annotations

import json
from pathlib import Path

import yaml

diff = json.loads(Path("build/symbol_diff.json").read_text())
doc = yaml.safe_load(Path("docs/changelog/v2.5.0.yaml").read_text())
for row in doc["rows"]:
    if row["id"] == "symbols.added":
        row["body"] = diff["added"]
    elif row["id"] == "symbols.removed":
        row["body"] = diff["removed"]
Path("docs/changelog/v2.5.0.yaml").write_text(yaml.safe_dump(doc, sort_keys=False))

Operators who extend the map should follow four mechanical rules rather than inventing extra authorities during a release week.

Add a new EXTRACT row only when a command can rebuild it from git objects, AST input, or another checked-in evidence file.
Add a DRAFT row only when every sentence can cite an EXTRACT identifier already present in the same map.
Add a SIGN row for any claim about users, stored data, calendar dates, money, downtime, or compatibility class.
Never convert a SIGN row into DRAFT because the model sounded confident or because the deadline is close.

Step 3: Let a model draft only restated summaries

The drafting prompt should receive build/symbol_diff.json and the empty narrative.summary row, not the live CHANGELOG.md. Ask for a restatement of added and removed names, grouped by path prefix, with no compatibility adjectives attached. Reject output that contains words such as safe, breaking, migrate, rollback, supported, or guarantee, because those tokens belong to SIGN rows.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode's free model access and free server option can host that constrained drafting step when a team wants the summary written off-laptop. The mention here is only about where a DRAFT row might be produced; it does not change the authority map and does not authorize filling SIGN rows.

After a draft returns, set drafted_by: model on narrative.summary and store the prose in body for review. A reviewer may edit that paragraph, then keep the flag or clear it after a full rewrite that no longer depends on the model. SIGN rows remain empty until a named owner writes them in a later commit that the checker can attribute.

A minimal prompt shape, labeled as a proposal rather than a measured template, looks like the following block.

You may write docs/changelog narrative.summary only.
Use solely the added and removed arrays from symbol_diff.json.
Group names by directory prefix.
Do not classify compatibility.
Do not mention upgrade, rollback, downtime, or data loss.
If a name is ambiguous, quote it and stop.

Step 4: Fail the build when SIGN rows are model-authored

The checker is the artifact that makes the map enforceable instead of ceremonial. It reads the YAML file, confirms EXTRACT bodies match the JSON evidence, and rejects failure classes that show up when generated notes are pasted too early. Wire it to the tag job so the human-readable changelog is rendered only after the gate passes.

#!/usr/bin/env python3
"""Fail CI when changelog authorities are violated. Proposal example."""
from __future__ import annotations

import json
import re
import sys
from pathlib import Path

import yaml

FORBIDDEN_IN_DRAFT = re.compile(
    r"\b(safe|breaking|compatible|migrate|rollback|guarantee|supported|zero downtime)\b",
    re.I,
)
SIGN_IDS = {
    "compat.class",
    "migration.steps",
    "rollback.policy",
    "deprecation.timeline",
}


def fail(msg: str) -> None:
    print(f"authority-check: {msg}", file=sys.stderr)
    raise SystemExit(1)


def main() -> int:
    diff = json.loads(Path("build/symbol_diff.json").read_text())
    doc = yaml.safe_load(Path(sys.argv[1]).read_text())
    rows = {r["id"]: r for r in doc["rows"]}

    if rows["symbols.added"]["body"] != diff["added"]:
        fail("EXTRACT symbols.added does not match symbol_diff.json")
    if rows["symbols.removed"]["body"] != diff["removed"]:
        fail("EXTRACT symbols.removed does not match symbol_diff.json")

    draft = rows["narrative.summary"]
    if draft["authority"] != "DRAFT":
        fail("narrative.summary must remain DRAFT")
    if FORBIDDEN_IN_DRAFT.search(draft.get("body") or ""):
        fail("DRAFT prose used a SIGN vocabulary word")

    removed = diff["removed"]
    compat = rows["compat.class"]
    if removed and not compat.get("body"):
        fail("removed public symbols require a human compat.class signature")
    if compat.get("drafted_by") == "model":
        fail("compat.class cannot be model-authored")
    if compat.get("body") and compat.get("body") not in {
        "compatible",
        "breaking",
        "mixed",
    }:
        fail("compat.class must be one of compatible|breaking|mixed")

    for sid in SIGN_IDS:
        row = rows[sid]
        if row.get("drafted_by") == "model":
            fail(f"{sid} is SIGN and cannot be model-authored")
        if row["authority"] != "SIGN":
            fail(f"{sid} must be SIGN")
        if row.get("body") and not row.get("owner"):
            fail(f"{sid} has body text but no owner")

    print("authority-check: ok")
    return 0


if __name__ == "__main__":
    raise SystemExit(main())

python3 extract_symbol_diff.py v2.4.0 v2.5.0
python3 stamp_extract_bodies.py
python3 check_authority.py docs/changelog/v2.5.0.yaml

Render CHANGELOG.md from the map only after this command exits zero and SIGN owners are present. Until that point, a drafted summary is a restatement of EXTRACT evidence rather than a user-facing release note.

Decision table for draft rights

Claim	Authority	Model may	Human must
Added or removed public names	EXTRACT	Not write this row	Confirm the export surface is intentional
Path-grouped restatement	DRAFT	Write, without compatibility adjectives	Edit for accuracy and keep evidence citations
compatible / breaking / mixed	SIGN	Leave blank	Choose exactly one value
User migration commands	SIGN	Propose only in a scratch file, never in the map	Rewrite, run, and sign
Rollback supported or unsupported	SIGN	Forbidden	State the policy in owner-attributed text
Deprecation end date	SIGN	Forbidden	Set calendar language, or omit the row
Security impact wording	SIGN	Forbidden	Security owner only

Treat any cell marked forbidden as a CI failure, not as a review comment sitting on the pull request. Review comments do not stop a fluent paragraph from reaching users who will treat it as a contract.

Limitations

The extractor only reads top-level function and class names in Python files outside tests/. It misses __all__ re-exports, C extensions, type stubs, CLI entry points, and HTTP routes, so the inventory is incomplete for many libraries. Word-list blocking on DRAFT prose is brittle, because a model can describe breakage without using the blocked tokens. The checker cannot see product intent, so a human can still sign compatible incorrectly after a real behavior change that left names intact.

Git tags must exist and must be the objects users install, or the evidence trail becomes theater. Comparing branches, stash dumps, or generated wheels will stamp rows that release consumers cannot reconstruct from the published tag. Teams that publish notes from issue trackers rather than from tags need a different EXTRACT source, and they should not pretend this script covers that case.

Who should skip this workflow

Skip the map when legal or customer contracts already define changelog language, because a YAML authority field is not a substitute for counsel review. Skip it for security advisories that require coordinated disclosure, since those documents have a different owner and a different embargo clock. Skip it when the project has no public Python surface, or when two tags cannot be fetched inside continuous integration.

Do not use model drafting at all when the release only exists to ship a data-loss fix. In that case there is nothing useful to restate from a symbol list, and the SIGN rows should be written first by the owner who reproduced the defect. Generated summaries add fluency without adding evidence, which is the failure mode this workflow is designed to keep out of user-facing notes.

Treat Config Docs as Two Artifacts: Extracted Inventory and Signed Promises

Avery Lin — Sat, 19 Sep 2026 22:07:44 +0000

Treat configuration documentation as two artifacts, because mixing inventory with promises is how unsigned defaults reach users. A model can list keys and call sites after an extractor walks the syntax tree of a typical codebase. A human still has to sign defaults, secret handling, deprecation windows, and every production-support claim. The rest of this article is a gate that refuses generated copy until those owned fields exist.

Why config pages fail as a single generated document

Most configuration pages rot for a mechanical reason rather than for weak writing or missing screenshots. Keys appear in code, then a chat session invents a default, a unit, and a support story. Reviewers often read the surrounding prose and miss the unsigned policy sitting inside a table cell. Users then treat that table cell as a support contract the maintainers never actually accepted.

Inventory is cheap to regenerate after every merge, and that regeneration is the part generation can help. Promises are expensive because they bind release managers, support staff, and security reviewers to a public statement. A useful workflow therefore lets a model draft structure and descriptions while blocking merge on empty human-owned fields. The ownership matrix below is the contract between those two jobs.

Ownership matrix: draftable inventory versus signed promises

Use the table as a review checklist rather than as a style guide for tone or brand voice. Extracted fields come from syntax and should be rebuilt in CI whenever the tree changes. Signed fields bind the project to users, so they stay UNSET until a named person replaces them. Generated prose that fills promise columns is a defect in this workflow, not a shortcut.

Field	Source of truth	Model may draft?	Human must sign?	Merge if `UNSET`?
Key name	Source identifiers	No; extract only	Confirm spelling	No
Call sites	AST or equivalent inventory	No; extract only	Spot-check paths	No if extractor is empty
Type hint	Annotations and usage	Yes, as a proposal	Confirm	Warning only
Short description	Nearby comments	Yes	Edit for accuracy	Allowed with a draft flag
Default value	Runtime plus deploy overlays	No	Yes	No
Secret / redaction	Threat model	No	Yes	No
Environment scope	Operations policy	Proposal only	Yes	No
Deprecation window	Release policy	No	Yes	No
Production support	Support policy	No	Yes	No
Example value	Sanitized samples	Yes, fake data only	Confirm no live secrets	No if the example looks real

The rule is simple enough to encode in CI, and that encoding is the point of the matrix. Anything recoverable from syntax is inventory and can be rebuilt without scheduling a meeting. Anything that would surprise a customer if wrong is a promise and needs a named signer. That split is what the extractor, the ownership file, and the gate implement in the next sections.

1. Extract inventory from source, never from chat memory

Start from the repository, not from a prompt that asks a model to remember your flags. The following Python example walks an abstract syntax tree for os.getenv and os.environ.get calls. Treat the script as a worked example you can extend to argparse, Click, or YAML loaders. It writes JSON inventory and never writes user-facing defaults, support windows, or secret labels.

#!/usr/bin/env python3
"""Extract configuration key inventory from Python sources. Worked example."""
from __future__ import annotations

import ast
import json
import sys
from pathlib import Path


def call_accessor(func: ast.AST) -> str | None:
    if isinstance(func, ast.Attribute) and isinstance(func.value, ast.Name):
        if func.value.id == "os" and func.attr == "getenv":
            return "os.getenv"
    if (
        isinstance(func, ast.Attribute)
        and func.attr == "get"
        and isinstance(func.value, ast.Attribute)
        and func.value.attr == "environ"
        and isinstance(func.value.value, ast.Name)
        and func.value.value.id == "os"
    ):
        return "os.environ.get"
    return None


def const_str(node: ast.AST | None) -> str | None:
    if isinstance(node, ast.Constant) and isinstance(node.value, str):
        return node.value
    return None


class EnvKeyVisitor(ast.NodeVisitor):
    def __init__(self, relpath: str) -> None:
        self.relpath = relpath
        self.rows: list[dict] = []

    def visit_Call(self, node: ast.Call) -> None:
        accessor = call_accessor(node.func)
        if accessor and node.args:
            key = const_str(node.args[0])
            source_default = const_str(node.args[1]) if len(node.args) > 1 else None
            if key:
                self.rows.append(
                    {
                        "key": key,
                        "accessor": accessor,
                        "literal_default_in_source": source_default,
                        "path": self.relpath,
                        "line": node.lineno,
                    }
                )
        self.generic_visit(node)


def extract(root: Path) -> list[dict]:
    rows: list[dict] = []
    skip = {".venv", "venv", "node_modules", ".git"}
    for path in root.rglob("*.py"):
        if any(part in skip for part in path.parts):
            continue
        rel = str(path.relative_to(root))
        try:
            tree = ast.parse(path.read_text(encoding="utf-8"), filename=rel)
        except (SyntaxError, UnicodeDecodeError):
            continue
        visitor = EnvKeyVisitor(rel)
        visitor.visit(tree)
        rows.extend(visitor.rows)
    rows.sort(key=lambda row: (row["key"], row["path"], row["line"]))
    return rows


if __name__ == "__main__":
    root = Path(sys.argv[1] if len(sys.argv) > 1 else ".")
    json.dump(extract(root), sys.stdout, indent=2)
    sys.stdout.write("\n")

Run the script against a checkout and store the JSON as a build artifact rather than as published documentation. A literal default found in source remains inventory, because deploy overlays can still replace it at runtime. Do not copy that literal into a user-facing Default column without a named human signer. Regenerate the inventory file in CI so missing keys fail the build instead of rotting on a wiki page.

mkdir -p build
python3 extract_config_keys.py . > build/config-inventory.json

2. Keep a human-owned promise file beside the inventory

Create docs/config-ownership.json and treat that file as the only promise artifact in this workflow. Generated description text may land in draft_description after a skim, which is a weaker bar than signing defaults. Every policy field starts as UNSET and remains invalid for merge until a person replaces the sentinel. The LOG_LEVEL row below is complete; the DATABASE_URL row must not merge with UNSET policy fields.

{
  "version": 1,
  "keys": {
    "DATABASE_URL": {
      "draft_description": "SQLAlchemy connection string used by the API process.",
      "default": "UNSET",
      "secret": "UNSET",
      "environments": "UNSET",
      "deprecation_window": "UNSET",
      "production_support": "UNSET",
      "example": "postgresql://USER:REDACTED@localhost:5432/app",
      "signer": "UNSET",
      "signed_at": "UNSET"
    },
    "LOG_LEVEL": {
      "draft_description": "Process log verbosity for the API worker.",
      "default": "INFO",
      "secret": "no",
      "environments": "all",
      "deprecation_window": "none",
      "production_support": "supported",
      "example": "INFO",
      "signer": "release-owners",
      "signed_at": "2026-09-20"
    }
  }
}

Date stamps belong to the signer, not to the model that proposed the sentence. A completed row is a review record, not proof that production currently matches the cell. Teams that prefer YAML can store the same shape in YAML, because the gate below cares about sentinels rather than about encoding. Keep the promise file in the same repository as the code that introduced the keys.

3. Gate the rendered reference page in CI

The gate has three jobs, and none of those jobs involve judging writing quality or tone. It fails when inventory keys are missing from the ownership file or extra rows lack extractor evidence. It fails when promise fields still contain the UNSET sentinel instead of a signed value. It also fails when a rendered example for a secret key looks like a live connection string.

#!/usr/bin/env python3
"""Fail CI when config promises are unsigned. Worked example, stdlib only."""
from __future__ import annotations

import json
import sys
from pathlib import Path

PROMISE_FIELDS = (
    "default",
    "secret",
    "environments",
    "deprecation_window",
    "production_support",
    "signer",
    "signed_at",
)


def main() -> int:
    inventory = json.loads(Path("build/config-inventory.json").read_text(encoding="utf-8"))
    ownership = json.loads(Path("docs/config-ownership.json").read_text(encoding="utf-8"))
    owned = ownership.get("keys") or {}
    errors: list[str] = []

    inv_keys = {row["key"] for row in inventory}
    for key in sorted(inv_keys):
        if key not in owned:
            errors.append(f"missing ownership row for {key}")
            continue
        row = owned[key]
        for field in PROMISE_FIELDS:
            value = row.get(field, "UNSET")
            if value is None or value == "UNSET" or str(value).strip() == "":
                errors.append(f"{key}.{field} is unsigned")
        secret = str(row.get("secret", "")).lower()
        example = str(row.get("example", ""))
        if secret in {"yes", "true"} and example:
            if "REDACTED" not in example.upper() and "://" in example:
                errors.append(f"{key}.example looks like a live secret")

    for key in sorted(set(owned) - inv_keys):
        errors.append(f"ownership row {key} has no inventory evidence")

    if errors:
        print("config doc ownership gate failed:", file=sys.stderr)
        for item in errors:
            print(f"- {item}", file=sys.stderr)
        return 1
    print(f"config doc ownership gate passed for {len(inv_keys)} keys")
    return 0


if __name__ == "__main__":
    raise SystemExit(main())

Wire both commands in the same CI job so inventory cannot silently diverge from the signed ownership file. A green gate means every extracted key has a signer, not that the signed default is operationally true. Wrong but signed values still need runtime checks, staging review, and incident follow-up after release. Put the commands in the same job so a partial checkout cannot publish yesterday's promises with today's keys.

python3 extract_config_keys.py . > build/config-inventory.json
python3 check_config_docs.py

4. Let a model draft only the draftable columns

After the inventory file exists, a model may propose draft_description text and sanitized examples for each extracted key. It must not write default, secret, deprecation_window, production_support, signer, or signed_at under any prompt. Feed the model the inventory JSON and the matrix, then paste proposals back into draft_description only.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. Free model access and a free server option can run the extractor and draft those description lines. Keep the ownership file in git rather than in a chat transcript after the draft returns. The product does not replace the signer column, and this workflow does not depend on a named model.

A practical prompt constraint is to require JSON fragments that contain only key and draft_description. Reject any payload that includes promise fields, even when the invented values look operationally plausible. Plausible unsigned defaults are the failure mode this gate exists to catch during review. Paste accepted fragments into draft_description and leave every promise field under human control.

5. Render the public page from both files

Keep the renderer boring so it cannot invent policy when a signed cell is missing from disk. Inventory supplies key names and call-site footnotes, while ownership supplies every signed table cell. Draft descriptions may appear as body copy after a human has skimmed them for obvious factual errors. Do not let the renderer fall back to literal_default_in_source when the signed default is absent.

#!/usr/bin/env python3
"""Render docs/config-reference.md from inventory plus signed ownership. Example only."""
from __future__ import annotations

import json
from collections import defaultdict
from pathlib import Path


def render() -> str:
    inventory = json.loads(Path("build/config-inventory.json").read_text(encoding="utf-8"))
    owned = json.loads(Path("docs/config-ownership.json").read_text(encoding="utf-8"))["keys"]
    sites: dict[str, list[str]] = defaultdict(list)
    for row in inventory:
        sites[row["key"]].append(f"{row['path']}:{row['line']}")

    lines = [
        "# Configuration reference",
        "",
        "Defaults, secret handling, and support windows are human-signed.",
        "Call sites are extracted from source on each build.",
        "",
    ]
    for key in sorted(owned):
        row = owned[key]
        lines.extend(
            [
                f"## `{key}`",
                "",
                f"- Default: `{row['default']}`",
                f"- Secret: {row['secret']}",
                f"- Environments: {row['environments']}",
                f"- Deprecation window: {row['deprecation_window']}",
                f"- Production support: {row['production_support']}",
                f"- Example: `{row['example']}`",
                f"- Signer: {row['signer']} on {row['signed_at']}",
                "",
                str(row.get("draft_description", "")),
                "",
            ]
        )
        if key in sites:
            joined = ", ".join(f"`{item}`" for item in sites[key][:8])
            lines.append(f"Call sites: {joined}")
            lines.append("")
    return "\n".join(lines).rstrip() + "\n"


if __name__ == "__main__":
    Path("docs/config-reference.md").write_text(render(), encoding="utf-8")

A missing promise should fail the gate in the previous step, not degrade into an invented contract during render. If you need a markdown table instead of definition lists, build it only after the gate has already passed. Rendering unsigned cells as empty strings is still a publication bug, because readers will fill the silence with whatever default they hope exists.

Test plan for the gate

Label the following cases as a test plan you can run without production traffic or customer data. The cases distinguish inventory drift from unsigned policy, which ordinary markdown snapshot tests rarely catch. If the team only diffs final HTML, unsigned defaults can hide inside an otherwise stable reference page. Run them on every change to the extractor, the gate, or the ownership schema itself.

Add a new os.getenv("NEW_FLAG") call and confirm CI fails for a missing ownership row.
Add the row with every promise field set to UNSET and confirm the gate still fails.
Sign the promise fields, rerun the job, and confirm the rendered page contains the signed default.
Put a live-looking DATABASE_URL example without REDACTED and confirm the secret check fails.
Add an ownership key that the extractor cannot find and confirm the extra-row check fails.
Change only draft_description and confirm the gate stays green, because copy edits are not promises.

These six cases are more informative than a single golden-file test of docs/config-reference.md. A golden file can freeze an unsigned default simply because nobody edited the surrounding headings. The gate asserts the ownership invariant even when the rendered markdown looks unchanged to a casual diff.

Limitations

This extractor covers a narrow Python pattern and will miss dynamically composed key names at runtime. Teams that load configuration from remote stores need another inventory source, or the gate will be incomplete. Literal defaults in source can disagree with Helm values, and the ownership file is where that conflict is resolved. Do not treat a green CI job as proof that production currently uses the signed default.

The gate cannot prove that a signed default is operationally true in every deployed environment. It only proves that a named signer accepted the cell before the reference page was rendered. Drafted descriptions can still be wrong in quieter ways, so they need a human pass before publication. Do not publish inventory call sites if those paths reveal internal hostnames your threat model treats as sensitive.

Who should not use this approach

Skip this workflow if a single operator owns every config key and already writes the reference by hand. Skip it if your configuration surface is already published through a reviewed schema such as OpenAPI or JSON Schema. Skip it if you cannot name a signer, because the ownership file then becomes theater instead of a gate. Those teams will pay the ceremony cost without reducing unsigned-promise risk in the published page.

Security-sensitive secret documentation should not be drafted on a shared server when examples might include real connection strings. Keep those examples synthetic, and keep the ownership file in the same private repository as the code. Teams that want a model to invent support windows should not use this method at all. Inventing support windows is the behavior the gate is designed to reject before merge.

Closing

Configuration reference pages stay honest when inventory is compiled and promises are signed as separate artifacts. A model can draft descriptions after the extractor has listed every key from source. A human still owns defaults, secret flags, environments, deprecation windows, and production support. Keep the ownership file in version control, regenerate inventory in CI, and refuse to merge UNSET cells.

Split Generated Library Docs Into Draftable Structure and Owned Warranties

Avery Lin — Sat, 19 Sep 2026 18:58:45 +0000

Generated documentation usually fails at warranty custody rather than at missing section headings or empty parameter tables. A model can draft structure from public symbols, but it cannot honestly own runtime behavior, compatibility, or support. Treat every sentence that a user might quote as a promise as human-owned, and keep the model inside a draftable lane. The rest of this article specifies a custody ledger, a scanner, and a five-step generation workflow.

Generated prose hides two claim classes

When a generator fills a README from signatures, it often invents outcomes that never appear in tests. Users later copy those generated sentences into support issues and treat them as binding product contracts. The documentation set then contains two kinds of text that look identical inside ordinary Markdown files. One kind only restates exported names, parameter types, and file paths that the current repository already proves.

The other kind asserts behavior under failure, upgrade, concurrency, elapsed time, or mixed version deployments. Maintainers need a mechanical split rather than a style guide that reviewers forget under deadline pressure. Draftable text may be regenerated on every tag because it is a projection of public symbols. Human-owned text must survive regeneration and must fail CI when it appears without a named owner.

The scanner below encodes that split as a custody ledger plus a set of forbidden unowned patterns. Nothing in this workflow measures writing quality, tone, or marketing clarity for an external landing page. It only asks whether a claim is a projection of the tree or a warranty that still needs an owner. If the claim is a warranty, the model must not be the last writer on that sentence.

Custody table: what a model may draft

Use the table as a review contract rather than as a prompt appendix that nobody opens. Rows marked draftable can be rebuilt from the repository without any human affidavit on the branch. Rows marked owned must keep a named owner, a change date, and a pointer to evidence. Cross-links sit in the middle because path existence is checkable while usefulness still needs a person.

Claim class	Model may draft	Human must own	Evidence
Heading tree from public modules	Yes	No	Module walk
Parameter names and annotated types	Yes	No	Signatures
Identifier glossary	Yes	No	Symbol index
Cross-links between doc pages	Yes	Review	Path existence
Defaults copied from AST literals	Yes, with citation	Confirm	AST defaults
Copy-paste commands that imply a run	No	Yes	Owned run stamp
Version compatibility and upgrade paths	No	Yes	Release policy
Failure, retry, and timeout behavior	No	Yes	Tests
Security, privacy, and threat statements	No	Yes	Security review
Latency, complexity, and throughput	No	Yes	Measured notes
Stability labels such as production-ready	No	Yes	Release train
Support windows and deprecation dates	No	Yes	Versioning policy

The table is intentionally conservative about examples, because fenced code reads like a demonstration that somebody executed. A model can propose a command shape from a CLI parser, but it cannot attest that the command printed that output today. Compatibility language is also owned because it binds future releases, not the current abstract syntax tree. Security sentences are owned because they are legal and operational statements rather than identifier restatements.

Artifact: a custody ledger the scanner can read

Keep the human custody ledger in docs/custody.yaml so a regeneration pass cannot silently rewrite owners. The file lists draftable surfaces, owned claim patterns, and the people who may sign affidavits. Patterns are lowercase tokens that usually turn a description into a warranty when they appear in user-facing Markdown. Owners are stable identifiers, not display names, so git blame and CODEOWNERS can stay aligned.

version: 1
draftable_surfaces:
  - headings
  - signature_tables
  - identifier_glossary
  - intra_doc_links
owned_claim_patterns:
  - always
  - never
  - guaranteed
  - compatible
  - supports
  - secure
  - thread-safe
  - zero-downtime
  - backwards-compatible
  - production-ready
  - o(1)
  - latency
  - sla
owners:
  - id: release
    files: ["docs/support.md", "CHANGELOG.md"]
  - id: runtime
    files: ["docs/failures.md"]
  - id: examples
    files: ["docs/getting-started.md"]

Place owned claims in Markdown using an HTML comment that the renderer ignores and the scanner parses. The comment records owner, claim class, and a short evidence pointer such as a test path. Unsigned prose that still contains an owned pattern is a gate failure, even when the surrounding paragraph looks polished. Example fences need a run stamp immediately above the fence, or the scanner treats the block as an implied demonstration.

<!-- custody:owned owner=runtime claim=failure evidence=tests/test_retry.py -->
On retryable transport errors the client waits two seconds, then attempts one retry.

<!-- custody:run owner=examples evidence=tests/test_cli_help.py -->

bash
python -m inventory.cli symbols --format json

python

That HTML comment format is the regeneration boundary between draftable files and owned files. Draftable files may be overwritten by a generator after each public API change lands. Owned comments and the sentences they cover must not be overwritten unless the named owner edits them in the same change. If a generator cannot preserve those comments, it must write to a *.draft.md sibling and leave the owned file untouched.

Scanner: fail CI on unowned warranties

The scanner is a small Python program that maintainers can run locally and inside CI. It loads the ledger, walks Markdown under docs/, and reports unowned warranties with file and line numbers. It also flags fenced blocks that lack a preceding run stamp, because those blocks impersonate executed examples. The script below is a worked example, not a published benchmark and not a claim about any hosted model.

#!/usr/bin/env python3
"""Fail when user-facing docs contain unowned warranty language."""

from __future__ import annotations

import pathlib
import re
import sys

import yaml

OWNED_RE = re.compile(
    r"<!--\s*custody:owned\s+owner=(?P<owner>\S+)\s+"
    r"claim=(?P<claim>\S+)\s+evidence=(?P<evidence>\S+)\s*-->"
)
RUN_RE = re.compile(
    r"<!--\s*custody:run\s+owner=(?P<owner>\S+)\s+evidence=(?P<evidence>\S+)\s*-->"
)
FENCE_RE = re.compile(r"^```

")
TOKEN_RE = re.compile(r"[A-Za-z0-9+()/.^-]+")


def load_ledger(path: pathlib.Path) -> dict:
    data = yaml.safe_load(path.read_text(encoding="utf-8"))
    if not data or "owned_claim_patterns" not in data:
        raise SystemExit(f"invalid custody ledger: {path}")
    patterns = [p.lower() for p in data["owned_claim_patterns"]]
    owners = {row["id"] for row in data.get("owners", [])}
    return {"patterns": patterns, "owners": owners}


def iter_markdown(root: pathlib.Path) -> list[pathlib.Path]:
    return sorted(p for p in root.rglob("*.md") if p.is_file())


def scan_file(path: pathlib.Path, ledger: dict) -> list[str]:
    lines = path.read_text(encoding="utf-8").splitlines()
    errors: list[str] = []
    active_owned: set[str] = set()
    pending_run = False
    for idx, raw in enumerate(lines, start=1):
        owned = OWNED_RE.search(raw)
        if owned:
            if owned.group("owner") not in ledger["owners"]:
                errors.append(f"{path}:{idx}: unknown owner {owned.group('owner')}")
            active_owned.add(owned.group("claim").lower())
            pending_run = False
            continue
        run = RUN_RE.search(raw)
        if run:
            if run.group("owner") not in ledger["owners"]:
                errors.append(f"{path}:{idx}: unknown run owner {run.group('owner')}")
            pending_run = True
            continue
        if FENCE_RE.match(raw):
            if raw.strip() != "

```" and not pending_run:
                errors.append(
                    f"{path}:{idx}: example fence without custody:run stamp"
                )
            pending_run = False
            continue
        if raw.startswith("#") or raw.startswith("|") or raw.startswith("<!--"):
            continue
        tokens = {tok.lower() for tok in TOKEN_RE.findall(raw)}
        hits = [p for p in ledger["patterns"] if p in tokens or p in raw.lower()]
        unsigned = [h for h in hits if h not in active_owned]
        if unsigned:
            errors.append(
                f"{path}:{idx}: unowned claims {unsigned}: {raw.strip()[:80]}"
            )
        active_owned.clear()
    return errors


def main() -> int:
    root = pathlib.Path("docs")
    ledger = load_ledger(pathlib.Path("docs/custody.yaml"))
    failures: list[str] = []
    for path in iter_markdown(root):
        failures.extend(scan_file(path, ledger))
    if failures:
        sys.stderr.write("\n".join(failures) + "\n")
        return 1
    sys.stdout.write(
        f"custody ok: {len(list(iter_markdown(root)))} markdown files\n"
    )
    return 0


if __name__ == "__main__":
    raise SystemExit(main())

The ownership window is one following Markdown line after an owned comment, which keeps affidavits local and reviewable. Example fences reset the pending run flag so a stamp cannot cover a later block on the same page. Heading lines and table rows are skipped because they are draftable surfaces in the ledger. Maintainers should still read tables, because a table cell can hide a compatibility promise; the scanner is a net, not a substitute for review.

Five-step generation workflow

1. Inventory public symbols from the tree

Walk only exported modules, CLI entry points, and checked-in schema files rather than chat history. Record each symbol with its module path, signature, and default literals when those defaults exist in source. Write the inventory to docs/_inventory.json so later steps do not re-parse the tree with a different policy. The symbol inventory is draftable output, and it should be regenerated whenever public symbols change.

python -m inventory.cli symbols --src src --out docs/_inventory.json
git diff --check docs/_inventory.json

That command is a shape, not a signed run, until a maintainer adds a custody run stamp in the getting-started page. The inventory must omit private helpers, because private names invite documentation of unstable call paths. If the language has an explicit export list, trust that list over recursive directory walking alone. Reject files that the published package does not ship, including tests and local helper scripts.

2. Emit a draftable skeleton into sibling files

Generate headings, signature tables, and a glossary into docs draft siblings rather than into the owned pages. Constrain the generator to copy identifiers from _inventory.json and to leave behavior sentences as empty placeholders. Empty placeholders are safer than fluent guesses, because fluent guesses look finished during hurried review. Diff the draft against the previous draft so reviewers see structural drift instead of rewritten warranties.

python tools/emit_doc_skeleton.py \
  --inventory docs/_inventory.json \
  --out-dir docs/drafts \
  --surfaces headings,signature_tables,identifier_glossary

The drafting pass only needs a place to run a constrained prompt against the symbol inventory and emit Markdown skeletons.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

MonkeyCode's free model access and free server option can host that isolated drafting step without extra hardware. Do not send owned affidavits or unpublished security notes to any drafting host, including a free remote shell. Keep the owned ledger and the scanner on the repository's normal CI, because those files are the contract, not the model output.

3. Promote structure, then hand-write owned claims

Copy stable headings and signature tables from the draft sibling into the owned Markdown file only. Do not copy narrative paragraphs, sample outputs, or any sentence that contains an owned pattern. Write failure behavior, support windows, and compatibility notes under custody comments with explicit evidence pointers. If evidence is missing, delete the sentence instead of asking the generator to complete the claim.

## retry()

| Argument | Type | Default |
| --- | --- | --- |
| attempts | int | 2 |

<!-- custody:owned owner=runtime claim=failure evidence=tests/test_retry.py -->
On retryable transport errors the client waits two seconds, then attempts one retry.

Promotion is a three-way merge among the previous owned file, the new draft, and the tests that evidence citations name. If a test path in an evidence field does not exist, the change is incomplete even when the scanner has not yet run. Owners listed in docs/custody.yaml should match CODEOWNERS so review routing and affidavit identity stay the same. Refuse to promote a draft that rewrites an owned paragraph without a matching test-suite change.

4. Stamp examples only after a local run

Run each getting-started command in a clean environment and paste the real output under a run stamp. If the output depends on wall-clock time or generated identifiers, replace those spans with placeholders and say so in the owned sentence. Never let the generator invent exit codes, because exit codes become support contracts the moment they appear. Delete examples that cannot be reproduced from the published package on a clean machine.

python -m inventory.cli symbols --format json > /tmp/symbols.json
python tools/stamp_example.py \
  --cmd "python -m inventory.cli symbols --format json" \
  --out docs/getting-started.md

The stamping helper should store the command, working directory, and package version beside the fence. That metadata is owned evidence, not decorative front matter for a static documentation site. If two examples share a prefix, stamp them separately so a later edit cannot reuse a stale run. Broken examples are documentation bugs, and they should fail the same CI job as unowned warranty tokens.

5. Gate the branch with the custody scanner

Run the scanner after promotion and after example stamping, not before, or it will flag unfinished drafts. Wire it as a required check so generated prose cannot land with always, secure, or production-ready left unsigned. Keep the job cheap: it reads Markdown and a YAML ledger, and it should finish in seconds. Fail closed when the ledger is missing, because a missing policy is not an empty policy.

# .github/workflows/doc-custody.yml
# Adapt action versions to current forge docs for your runners.
name: doc-custody
on: [push, pull_request]
jobs:
  scan:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: "3.12"
      - run: pip install pyyaml pytest
      - run: python tools/scan_doc_custody.py
      - run: PYTHONPATH=tools pytest tests/test_scan_doc_custody.py

The workflow file is a template maintainers must adapt to their forge, runner labels, and Python version. Pinning ubuntu-latest without reading the forge changelog is a separate operations risk this article does not claim to solve. The important property is that custody failure blocks merge the same way a unit-test failure blocks merge. Documentation generation that cannot pass this gate is still a draft, regardless of how complete it reads.

Tests for the scanner itself

A custody gate that nobody tests will rot into a linter people skip. Keep a tiny fixture tree under tests/fixtures/docs with one clean page and one dirty page. The clean fixture page has one owned failure claim and one stamped command example. The dirty fixture page uses backwards-compatible in unsigned prose and an unstamped code fence.

from pathlib import Path

from scan_doc_custody import load_ledger, scan_file


def test_clean_page_has_no_errors(tmp_path: Path) -> None:
    docs = tmp_path / "docs"
    docs.mkdir()
    (docs / "custody.yaml").write_text(
        "owned_claim_patterns: [always]\nowners: [{id: runtime, files: []}]\n",
        encoding="utf-8",
    )
    page = docs / "ok.md"
    page.write_text(
        "<!-- custody:owned owner=runtime claim=always evidence=tests/t.py -->\n"
        "The mapper always emits UTF-8.\n",
        encoding="utf-8",
    )
    ledger = load_ledger(docs / "custody.yaml")
    assert scan_file(page, ledger) == []


def test_unsigned_warranty_is_reported(tmp_path: Path) -> None:
    docs = tmp_path / "docs"
    docs.mkdir()
    (docs / "custody.yaml").write_text(
        "owned_claim_patterns: [secure]\nowners: [{id: runtime, files: []}]\n",
        encoding="utf-8",
    )
    page = docs / "bad.md"
    page.write_text("The proxy is secure by default.\n", encoding="utf-8")
    ledger = load_ledger(docs / "custody.yaml")
    errors = scan_file(page, ledger)
    assert errors and "unowned claims" in errors[0]

Those tests lock the intended split: draftable restatement stays silent, and unsigned warranties stay noisy. They do not prove that a generator will obey the sibling-file rule, which remains a process control. If a team later adds HTML docs or generated sites, they must extend the walker rather than assuming Markdown is the only surface. The custody idea is independent of the file format; the scanner in this article is not.

Limitations and who should skip this

This workflow is a documentation control plane, not a writing assistant and not a substitute for product legal review. Teams without tests cannot fill evidence pointers, and empty evidence fields make the affidavits decorative. Single-file scripts with no public API surface will spend more time maintaining the ledger than they will save. Security advisories, license grants, and privacy statements need specialist owners sitting outside this scanner.

The pattern list will both under-match and over-match, because English warranties are not a closed token set. supports in "the parser supports UTF-8" may be an owned compatibility claim or a restatement of a codec name. Human review still decides promotion, and the scanner only catches the obvious unsigned tokens after promotion. Do not treat a green custody job as proof that the docs are true, only that owned language was claimed by a named person.

Generated headings can still be wrong when the inventory policy includes private modules or omits shipped entry points. Wrong structure is cheaper to fix than wrong warranties, which is why this article puts the model on the structure side. If a maintainer cannot name an owner for a sentence, the sentence does not belong in the published set. Delete it, move it to an internal design note, or wait for a test that can serve as evidence.

What this changes in review

Reviewers can stop arguing about whether generated prose sounds confident enough for a 1.0 README. They can ask a narrower question: which sentences are projections of public symbols, and which sentences are warranties. Warranties need owners, dates of a kind encoded in git, and evidence paths that exist on the branch. Everything else can be regenerated without extra ceremony when the public API surface moves.

The custody ledger also makes multi-maintainer documentation less tribal across rotating reviewers on a library. A new contributor can add a signature table without inheriting responsibility for retry timing they have never run. The runtime owner remains responsible for failure prose, even when the generator refreshed the heading tree in the same pull request. That separation of draftable structure and owned behavior is the entire method of this workflow.