Compile Exception Inventories for Docs; Hand-Sign Impact and Retry Claims

Generated troubleshooting pages fail in review when models invent recovery claims that no maintainer has actually signed. Extract exception classes and stable error codes from source, then restrict generation to headings, related symbols, and explicit placeholders. Humans must own user-visible impact, data mutation, and retry safety before any troubleshooting page is allowed to merge.

Why error catalogs are a different documentation class

Front-page README files attract review because they sit on the default repository landing path for every visitor. Troubleshooting pages sit one click deeper and often ship as generated filler after a release freeze. A model can list exception class names from an AST walk without knowing whether retrying the call duplicates a billed side effect. That gap is not a tone problem; it is an ownership split between compile facts and production claims.

Teams that treat the whole page as model output mix two evidence classes that do not share a reviewer. Structure can be compiled from the tree with a deterministic script and a stable identifier scheme. Claims about money, data loss, and retry safety cannot be compiled from syntax or message strings alone. Mixing those classes produces pages that look complete while remaining unsigned by anyone on the on-call rotation.

This workflow keeps the split mechanical by generating the inventory file and requiring humans to edit the ownership file. Continuous integration refuses the merge when those files disagree, or when draft tokens remain in a human cell. Reviewers then read a claims diff instead of a wall of generated prose that hides the unsigned fields.

The ownership matrix the checker enforces

The artifact for this workflow is a two-file contract rather than a longer generation prompt that hides ownership. The errors.inventory.json file is compiled from the application tree and must remain machine-owned after every commit. The claims.ownership.yml file is edited only by humans and is the only source for production sentences.

Field	Source	Model may draft?	Merge if unsigned?
compile_id	inventory script	no	no
symbol	inventory script	no	no
message_template	inventory script	no	no
related_symbols	model skeleton	yes, structure only	yes, as links
user_visible_impact	human ownership file	no	no
mutates_data	human ownership file	no	no
safe_to_retry	human ownership file	no	no
recovery_command	human ownership file	no	no
signed_by / signed_at	human ownership file	no	no

Keep the matrix in the repository beside the two files so reviewers do not renegotiate the split during each pull request. The checker below encodes the same rules, which keeps the table from drifting into wiki folklore. If a new field appears in generated prose, add a matrix row before teaching the model another heading style.

Step 1 — Compile the exception inventory from source

Do not ask a model to discover the error surface of a library or a production service. Walk annotated exception classes with a small script and emit stable identifiers that survive file moves only when you choose that rule. The script below is a labeled proposal for trees that store error_code as a class-level constant.

# compile_error_inventory.py
# Proposal: unexecuted example for a Python tree that tags errors with error_code.
from __future__ import annotations

import ast
import json
from pathlib import Path


class ErrorVisitor(ast.NodeVisitor):
    def __init__(self, relpath: str) -> None:
        self.relpath = relpath
        self.rows: list[dict] = []

    def visit_ClassDef(self, node: ast.ClassDef) -> None:
        bases = [ast.unparse(b) for b in node.bases]
        if not any("Error" in b or "Exception" in b for b in bases):
            self.generic_visit(node)
            return
        code = None
        template = None
        for stmt in node.body:
            if not isinstance(stmt, ast.Assign):
                continue
            for target in stmt.targets:
                if isinstance(target, ast.Name) and target.id == "error_code":
                    if isinstance(stmt.value, ast.Constant):
                        code = stmt.value.value
                if isinstance(target, ast.Name) and target.id == "message_template":
                    if isinstance(stmt.value, ast.Constant):
                        template = stmt.value.value
        if code:
            self.rows.append(
                {
                    "compile_id": f"{self.relpath}:{node.name}:{code}",
                    "symbol": node.name,
                    "error_code": code,
                    "message_template": template or "",
                    "path": self.relpath,
                    "lineno": node.lineno,
                }
            )
        self.generic_visit(node)


def compile_inventory(root: Path) -> list[dict]:
    rows: list[dict] = []
    for path in sorted(root.rglob("*.py")):
        tree = ast.parse(path.read_text(encoding="utf-8"))
        visitor = ErrorVisitor(str(path.relative_to(root)))
        visitor.visit(tree)
        rows.extend(visitor.rows)
    rows.sort(key=lambda r: r["compile_id"])
    return rows


if __name__ == "__main__":
    import argparse

    parser = argparse.ArgumentParser()
    parser.add_argument("--root", type=Path, required=True)
    parser.add_argument("--out", type=Path, required=True)
    args = parser.parse_args()
    payload = {"errors": compile_inventory(args.root)}
    args.out.parent.mkdir(parents=True, exist_ok=True)
    args.out.write_text(json.dumps(payload, indent=2) + "\n", encoding="utf-8")

Run the compiler before any generation step so the model never becomes the source of identifiers.

python compile_error_inventory.py --root src --out docs/_generated/errors.inventory.json
git diff -- docs/_generated/errors.inventory.json

The inventory may include names, file paths, line numbers, and literal message templates copied from constants. Compiled inventory rows must omit severity labels, blast radius, billed side effects, and any safe-to-retry boolean flag. Those omitted fields are production claims, and production claims do not belong in a compiled inventory file.

Step 2 — Restrict the model to a draftable skeleton

A model may draft the troubleshooting skeleton from the inventory after the compile step has finished. Allowed outputs are H2 headings per compile_id, related-symbol lists, and placeholders that name each human cell. Forbidden outputs include recovery shell commands, payment or deletion warnings, SLA numbers, and any sentence that asserts production behavior.

The following prompt is a labeled proposal, not an executed transcript from a production run.

You are drafting STRUCTURE only for troubleshooting.md.
Read docs/_generated/errors.inventory.json.
For each compile_id, emit:

## {symbol} (`{error_code}`)
Related symbols: (list other inventory symbols that share a path prefix)
{{CLAIM:user_visible_impact:{compile_id}}}
{{CLAIM:mutates_data:{compile_id}}}
{{CLAIM:safe_to_retry:{compile_id}}}
{{CLAIM:recovery_command:{compile_id}}}

Rules:
- Do not invent error codes.
- Do not fill CLAIM tokens.
- Do not write shell commands.
- Do not assert that an operation is safe, billed, or destructive.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode's free model access and free server option can host this draft step when a maintainer wants the skeleton produced off-laptop. The ownership file and the checker still run in the same repository hooks you already use for tests. Do not treat model output as a signed claim even when the troubleshooting draft looks fluent and complete.

After the draft step, the model should write only the structure file and must leave every CLAIM token untouched. Reviewers should reject a skeleton that contains a verb like retry, delete, refund, or safe outside a claim token. A related-symbol list is structure; a recommended shell pipeline is a claim and belongs in the ownership file.

Step 3 — Hand-write the five human claim fields

For every compile_id in the compiled inventory, a maintainer writes five fields into docs/claims.ownership.yml by hand. None of these fields may be pasted from chat output, because the checker cannot prove origin beyond the signed_by value you type. Paste detection is a review habit here, not a cryptographic proof of authorship for the signed row.

# docs/claims.ownership.yml
# Human-owned. Do not generate this file.
version: 1
claims:
  - compile_id: "payments/errors.py:ChargeConflictError:PAY_CONFLICT"
    user_visible_impact: "The customer may see a pending charge without an order confirmation."
    mutates_data: true
    safe_to_retry: false
    recovery_command: "bin/payments reconcile --charge-id <id> --dry-run"
    signed_by: "maintainer-id"
    signed_at: "2026-09-20"
  - compile_id: "payments/errors.py:IdempotencyReplayError:PAY_IDEMPOTENT"
    user_visible_impact: "The API returns the original receipt; the customer should not be double charged."
    mutates_data: false
    safe_to_retry: true
    recovery_command: "true  # no-op; return stored receipt"
    signed_by: "maintainer-id"
    signed_at: "2026-09-20"

Numbered rules for the human pass:

1. Set mutates_data from storage code and side-effect logs, not from the exception class name alone during review.
2. Set safe_to_retry only when a replay cannot create a second durable write or a second bill.
3. Put the recovery command in a form an on-call engineer can paste, including a dry-run flag when writes exist.
4. Refuse to sign a row that still contains TODO, a claim token, or wording copied from the skeleton file.

Unsigned rows are not partial progress toward a merge; they are missing evidence and must block the renderer. A true value for mutates_data without a dry-run recovery command should fail human review even if the checker only looks for nonempty strings. Date stamps use the calendar day of the signature, not the day the skeleton was generated.

Step 4 — Render only after both files agree

A renderer concatenates the structure file and the ownership file into the published troubleshooting page in git. It must exit nonzero if a claim token remains or if an inventory id lacks a matching ownership row. It must also exit when an ownership row points at a compile_id that the inventory no longer emits.

# render_troubleshooting.py
# Proposal: unexecuted example.
from __future__ import annotations

import json
import re
import sys
from pathlib import Path

import yaml

TOKEN = re.compile(r"\{\{CLAIM:([a-z_]+):([^}]+)\}\}")


def load_claims(path: Path) -> dict[str, dict]:
    data = yaml.safe_load(path.read_text(encoding="utf-8"))
    return {row["compile_id"]: row for row in data["claims"]}


def render(structure: str, claims: dict[str, dict], inventory_ids: set[str]) -> str:
    missing = inventory_ids - set(claims)
    extra = set(claims) - inventory_ids
    if missing or extra:
        raise SystemExit(
            f"claim/inventory mismatch missing={sorted(missing)} extra={sorted(extra)}"
        )

    def repl(match: re.Match[str]) -> str:
        field, compile_id = match.group(1), match.group(2)
        row = claims[compile_id]
        if field not in row or row[field] in (None, "", "TODO"):
            raise SystemExit(f"unsigned field {field} for {compile_id}")
        value = row[field]
        if field == "mutates_data":
            return "Mutates durable data: yes" if value else "Mutates durable data: no"
        if field == "safe_to_retry":
            return "Safe to retry: yes" if value else "Safe to retry: no"
        return str(value)

    rendered = TOKEN.sub(repl, structure)
    if "{{CLAIM:" in rendered:
        raise SystemExit("unresolved claim token")
    if not all(row.get("signed_by") and row.get("signed_at") for row in claims.values()):
        raise SystemExit("every claim row needs signed_by and signed_at")
    return rendered


if __name__ == "__main__":
    inventory = json.loads(Path("docs/_generated/errors.inventory.json").read_text())
    ids = {row["compile_id"] for row in inventory["errors"]}
    structure = Path("docs/_generated/troubleshooting.structure.md").read_text()
    claims = load_claims(Path("docs/claims.ownership.yml"))
    out = render(structure, claims, ids)
    Path("docs/troubleshooting.md").write_text(out, encoding="utf-8")
    print("rendered docs/troubleshooting.md", file=sys.stderr)

python compile_error_inventory.py --root src --out docs/_generated/errors.inventory.json
python render_troubleshooting.py
python - <<'PY'
from pathlib import Path
text = Path("docs/troubleshooting.md").read_text()
assert "{{CLAIM:" not in text
assert "Safe to retry:" in text
print("ownership gate passed")
PY

Wire the compile and render commands in the same job so a skeleton cannot land without signed claims. The final Python assertion is a second check for leftover tokens after a successful render in CI. Failing this job in CI is cheaper than publishing an unsigned recovery command during an incident.

Step 5 — Read the gate output as a review checklist

When the checker fails, treat the output as a review queue rather than as a writing task for the model. A missing compile_id usually means a new exception shipped into production without an on-call sentence attached. An extra claim row means a deleted error still has recovery text that will mislead operators during an incident. An unsigned field means the skeleton is still a draft and is not yet an operator document.

Reviewers should inspect diffs of claims.ownership.yml with the same attention they already give to migration SQL. A flipped safe_to_retry bit is a behavior change for anyone following the page during an incident. Do not accept a claims diff that only updates signed_at without a corresponding inventory or product change.

Limitations

This method assumes exception classes carry a stable error_code constant that the AST visitor can read. Dynamic error factories, stringly-typed codes, and errors raised from generated protobuf stubs will not appear until you extend the compiler. The checker proves presence and signature metadata, and it does not prove that mutates_data matches production. Wrong human claims will still merge if a maintainer types a name under signed_by without reading the storage path.

Message templates that interpolate untrusted input can leak into the inventory file during the compile step. Keep templates as format strings, and strip interpolated values before compile output is committed to git. The renderer does not sanitize recovery commands, so a signed malicious command is still a malicious command in the page.

Free model drafts can drift across runs even when the inventory is unchanged between those runs. Commit the skeleton only when the heading set matches compile_id values from the current inventory file. Ignore wording churn in related-symbol lists unless a reviewer explicitly asked for that cross-link change.

Who should not use this approach

Do not use this workflow on repositories that have no durable side effects and no operator recovery path, because the ownership file becomes ceremony. Do not use it as a substitute for runbooks that live in an incident tool with paging and ownership rotations. Do not point the model at production logs to fill in claim fields, because logs are evidence for humans. Mutation flags belong in the ownership file after a human reads those logs, not in the draft prompt.

Skip the generator entirely when the error surface is smaller than roughly a dozen codes and a maintainer already hand-writes the page. The compile step still helps, but the skeleton model adds a moving file without reducing review load. Teams that cannot name a signed_by identity should not enable the merge gate, because the signature field would become a rubber stamp.

The useful split in this documentation workflow stays narrow and mechanical across both libraries and services. Models may draft page structure from compiled identifiers, including headings, related symbols, and empty claim tokens. Humans own impact, mutation, retry safety, and the recovery command that operators will paste during an incident. If you already keep documentation checks in CI, add the ownership files next to them before expanding generation.