Compile an Exception Catalog From AST; Sign Retryability Outside the Model

Unsigned operational claims in generated docs are a release defect, not a writing problem. A compiler should own exception names, modules, and raise sites from the current tree. A model may restate those extracted facts in plain language, and it must invent nothing else. A human must sign retryability, HTTP status maps, PII flags, and support language before any render.

This article proposes an unexecuted documentation pipeline for Python services that already raise typed errors. It does not claim production metrics, customer outcomes, or a measured reduction in incidents. The working artifact is a three-lane ledger: compiled cells, draft restatements, and human-signed policy cells that the renderer will refuse to invent.

Why a single-pass doc generator fails here

Exception names and file locations are mechanical facts, so they belong in a compiler rather than a chat transcript. Retryability, public HTTP status, and whether a message may contain identifiers are policy, not syntax. Mixing those lanes produces docs that look complete while quietly promising behavior the runtime never guaranteed. The failure mode is familiar: a generated page lists PaymentConflict, then asserts it is safe to retry, then ships with a 500 mapping that operations never approved.

Trend discussion around calling unreviewed model output "engineering" is adjacent, not the method. The method is narrower. Names come from AST. Prose restatements are optional and bounded. Operational meaning is a signature, stored beside the catalog, never inferred from tone.

Lane split before any model call

Treat every catalog row as three disjoint regions, and reject writes that cross a region boundary.

1. Compile lane. Exception class, bases, module path, constant code strings, and raise-site counts from the current commit.
2. Restatement lane. One sentence that may only restate compile-lane fields; banned tokens include SLA, forever, always retry, never fails, and PII examples.
3. Signature lane. http_status, retryable, user_visible, may_contain_pii, support_window, and owner, all empty until a human writes them.

A restatement that introduces a status code is a failed draft, not a helpful expansion. A signature cell filled by a model is a failed build, not a review comment. Keep those rules in tests, because review fatigue will not enforce them on a long catalog.

Step 1: Extract the catalog from AST

The extractor below is a proposed script, not a measured production service. Point it at a package root, then emit JSON that later stages must treat as read-only facts.

#!/usr/bin/env python3
"""Proposed compiler: exception catalog from AST. Unexecuted example."""
from __future__ import annotations

import ast
import json
import sys
from collections import Counter
from pathlib import Path

class RaiseCounter(ast.NodeVisitor):
    def __init__(self) -> None:
        self.counts: Counter[str] = Counter()

    def visit_Raise(self, node: ast.Raise) -> None:
        exc = node.exc
        if isinstance(exc, ast.Call) and isinstance(exc.func, ast.Name):
            self.counts[exc.func.id] += 1
        elif isinstance(exc, ast.Name):
            self.counts[exc.id] += 1
        self.generic_visit(node)


def exception_classes(tree: ast.AST, module: str) -> list[dict]:
    rows = []
    for node in tree.body:
        if not isinstance(node, ast.ClassDef):
            continue
        bases = [b.id for b in node.bases if isinstance(b, ast.Name)]
        if not any(b.endswith("Error") or b.endswith("Exception") or b == "Exception" for b in bases + [node.name]):
            continue
        code = None
        for stmt in node.body:
            if isinstance(stmt, ast.Assign):
                for t in stmt.targets:
                    if isinstance(t, ast.Name) and t.id == "code" and isinstance(stmt.value, ast.Constant):
                        code = stmt.value.value
        rows.append({
            "name": node.name,
            "module": module,
            "bases": bases,
            "code": code,
            "lineno": node.lineno,
        })
    return rows


def compile_atlas(root: Path) -> dict:
    rows = []
    raises: Counter[str] = Counter()
    for path in root.rglob("*.py"):
        if "tests" in path.parts:
            continue
        source = path.read_text(encoding="utf-8")
        tree = ast.parse(source)
        module = ".".join(path.relative_to(root).with_suffix("").parts)
        rows.extend(exception_classes(tree, module))
        visitor = RaiseCounter()
        visitor.visit(tree)
        raises.update(visitor.counts)
    for row in rows:
        row["raise_sites"] = int(raises.get(row["name"], 0))
        row["restatement"] = ""
        row["http_status"] = None
        row["retryable"] = None
        row["user_visible"] = None
        row["may_contain_pii"] = None
        row["support_window"] = None
        row["owner"] = None
        row["signed"] = False
    return {"schema": "exception-atlas.v1", "package": root.name, "rows": rows}


if __name__ == "__main__":
    atlas = compile_atlas(Path(sys.argv[1]).resolve())
    json.dump(atlas, sys.stdout, indent=2, sort_keys=True)
    sys.stdout.write("\n")

Run it as a deterministic compile step, then store the output beside the docs tree rather than inside a chat log.

python compile_exception_atlas.py ./src > docs/_generated/exception_atlas.json

The compiler may record raise_sites as an integer count, which is a tree fact, not a traffic claim. It must not emit QPS, error budgets, or customer impact. Those fields do not exist in AST, so they cannot be honest compile-lane data.

Step 2: Freeze the row contract

Document the JSON contract in the same change as the compiler, so later drafts cannot grow extra keys quietly. The proposed shape is small on purpose.

exception-atlas.v1 row
  compile: name, module, bases, code, lineno, raise_sites
  draft:   restatement          # model-writable, fact-bounded
  sign:    http_status, retryable, user_visible,
           may_contain_pii, support_window, owner, signed

A merge script should copy compile fields from the latest AST run onto any previously signed row with the same module:name key. If a class disappears, keep the old signature in a removed list instead of deleting history. Humans still decide whether the public catalog may drop that name.

Step 3: Bound the restatement, then discard overreach

If a restatement step is used, feed only compile-lane fields, never prior signatures, never incident tickets, and never customer text. The prompt is a filter, not a research brief. A proposed instruction block follows; treat it as unexecuted.

Restate only these fields in one sentence of at most 28 words:
name, module, bases, code, raise_sites.
Do not mention HTTP, retry, SLA, PII, customers, or time windows.
Do not add examples, URLs, or sample payloads.
If a field is null, say it is absent; do not guess.

Validate the draft with a linter before it touches the ledger. Reject sentences that contain banned tokens, digits that do not already appear in code or raise_sites, or any key that belongs in the signature lane. The restatement is optional documentation sugar. The catalog remains publishable if every restatement is blank and every signature is filled.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode's free model access and free server option can host that restatement job against the frozen compile JSON, which keeps the draft off the signed cells. The product claim here is availability of those two options only; this article does not name models, quotas, hardware, duration, or benchmarks.

Step 4: Sign operational cells by hand

Humans fill a sidecar, not the generated JSON blob, so reviews can diff policy without rereading AST noise. Proposed sidecar name: docs/exception_policy.yaml.

# Human-owned. Models must not write this file.
rows:
  billing.PaymentConflict:
    http_status: 409
    retryable: false
    user_visible: true
    may_contain_pii: false
    support_window: "documented for the current major API"
    owner: "payments-api"
    signed: true
  billing.ProviderTimeout:
    http_status: 503
    retryable: true
    user_visible: true
    may_contain_pii: false
    support_window: "documented for the current major API"
    owner: "payments-api"
    signed: true

Signing retryable: true is a compatibility promise to clients that will back off and replay. Signing may_contain_pii: true is a logging and support-doc constraint, not a prose flourish. If the owner cannot name a window, leave signed: false and fail the docs build. An unsigned row is cheaper than a sentence that invents a sunset date.

Step 5: Render only merged, signed rows

The renderer joins atlas and sidecar on module.name, then writes Markdown from the join, not from chat output. Proposed rules for the join:

1. Drop any atlas row whose sidecar signed is not true.
2. Fail the build if the sidecar names a class the atlas no longer contains, unless it is marked removed.
3. Fail the build if a restatement file tries to set http_status or retryable.
4. Emit a table, then a short paragraph per signed row, using the restatement only when present.

# Proposed join; unexecuted example.
REQUIRED = ("http_status", "retryable", "user_visible", "may_contain_pii", "support_window", "owner")

def merge(atlas_rows, policy):
    published = []
    for row in atlas_rows:
        key = f"{row['module']}.{row['name']}"
        sig = policy.get(key)
        if not sig or not sig.get("signed"):
            continue
        missing = [k for k in REQUIRED if sig.get(k) in (None, "")]
        if missing:
            raise SystemExit(f"unsigned cells on {key}: {missing}")
        published.append({**row, **sig})
    return published

A small pytest file should treat leaked unsigned rows as a broken pipeline, the same way a missing migration is a broken release.

BANNED = ("sla", "always retry", "never fails", "forever", "guarantee")

def test_no_unsigned_rows_in_published_markdown(published_markdown: str, merged_rows):
    assert merged_rows, "renderer published nothing; catalog cannot be empty after sign-off"
    lower = published_markdown.lower()
    for token in BANNED:
        assert token not in lower
    for row in merged_rows:
        assert f"{row['name']}" in published_markdown
        assert str(row["http_status"]) in published_markdown

Decision table: what each lane may claim

Claim	Compile from AST	Model restatement	Human signature
Class name, module, bases	Yes	Restate only	No rewrite
Constant code attribute	Yes if literal	Restate only	No rewrite
Raise-site count in tree	Yes	Restate only	No rewrite
HTTP status to clients	No	No	Yes
Safe to retry	No	No	Yes
Message may include PII	No	No	Yes
Support window / sunset	No	No	Yes
Sample payloads with real ids	No	No	Yes, redacted

If a desired sentence cannot be placed in one column, split the sentence. "ProviderTimeout is raised from billing" can be compiled. "Clients may retry ProviderTimeout with backoff" cannot be compiled, and it cannot be drafted into existence either.

Limitations

The AST pass will miss exceptions constructed through factories, string imports, or raise on values that are not names. Raise-site counts ignore dynamic dispatch, so they are inventory hints, not coverage. Message templates that are built at runtime cannot be classified for PII by this compiler. The restatement linter is token-based, so a model can still smuggle policy in novel phrasing that tests do not yet ban. Humans still read the published table.

This workflow also assumes a single package root and stable class names. A monorepo that re-exports errors under several public aliases needs an extra alias map, which this article does not provide. Do not treat the sidecar YAML as a substitute for an API review when you change status codes.

Who should not use this approach

Do not use it if your public errors are untyped strings, or if support language must change hourly with incidents. Do not use it to generate legal terms, uptime promises, or security advisories. Do not use a restatement model on catalogs that already embed customer identifiers in class docstrings. Teams that need a narrative architecture guide should write that guide; this pipeline only publishes a signed failure atlas.

The useful close is operational, not promotional: keep compile output, draft restatements, and signed policy in three files, and make the docs job fail when those files disagree. If the restatement host is a free server already in use for other bounded jobs, that does not change who owns retryability. The catalog is documentation only after the signature lane is full.