Walk Raise Sites Into an Error Ledger; Humans Own Severity, Recovery, and Customer Copy

Generated exception inventories stay useful when they are compiled from raise sites rather than from tutorial memory. Human reviewers still own severity, customer-facing wording, and recovery steps because those claims are not present in the AST. A ledger mixing extracted types with unsigned operational copy fails merge until a named reviewer signs the human lanes. This article describes a proposed Python workflow, a JSON ledger schema, and a CI check that keep those lanes separate.

Why error pages drift even when the code is current

Error documentation usually rots at the boundary between extractable symbols and operational claims that never appear in source. A class name, module path, and mapped status code can be compiled from raise sites with a deterministic AST walk. Phrases about customer impact, paging policy, and rollback order cannot be compiled, yet they often share one README paragraph. When a writer fills those phrases from memory, the page looks complete while the recovery advice remains unverified.

Reviewers then spend time negotiating tone instead of noticing that no raise site encoded a severity class. The proposed ledger treats that mismatch as a schema problem rather than a style problem. Extracted fields may update automatically after each commit; owned fields stay empty, flagged, and blocking until a named reviewer writes them. Tutorial prose is generated last, and only from rows that already satisfy the ownership matrix below.

Ownership matrix for one error row

Use this table as the contract for every exception that production code can raise through a documented path. Machine columns come from the AST; reviewer columns stay unsigned until a human writes a value that the CI job can hash. Model output, if present at all, lives in a draft column that the merge gate always strips. Rows that cannot prove a field from a raise site keep that field on the reviewer side even when a draft sounds complete.

Field	Derived from	Owner	Empty on merge
exc_qualname	Class def or raise name	extractor	forbidden
module / lineno	AST location	extractor	forbidden
raise_count	Count of raise nodes	extractor	forbidden
message_const	String literal or null	extractor	allowed if null
http_status_literal	Nearby integer assign	extractor	allowed if null
severity	Incident policy	reviewer	forbidden
customer_copy	Support language	reviewer	forbidden
recovery_steps	Runbook	reviewer	forbidden
retry_class	Idempotency policy	reviewer	forbidden
draft_recovery	Optional assistant text	never	must be absent

The matrix is the working artifact for this workflow, not a decorative summary of later tutorial prose. Teams may add columns, but they should not collapse extracted symbols and owned operational claims into one free-text cell. Fluent assistant text is not a source, and it must not be hashed as if a reviewer had accepted it. CI should hash the owned fields separately so a regenerated inventory cannot silently rewrite signed recovery copy.

Numbered workflow

Follow these five steps in repository order so extracted rows exist before anyone writes recovery prose. Skipping the extractor and drafting the runbook first is the usual path by which invented recovery text enters the tree. Each step writes or validates a concrete file, which keeps the review conversation attached to keys rather than to vibes. If a step cannot run in CI, treat the ledger as a draft local note and do not publish customer pages from it.

1. Freeze the inventory path and schema version

Choose a single JSON document, for example docs/error-ledger.json, and refuse alternate copies in wiki pages or chat exports. Record a schema version at the top of the file so later extractors do not silently rename keys under signed rows. Keep the file in the same repository as the raise sites, because cross-repo docs lose the line numbers that make review cheap. Label this layout as a proposal if your docs already live in a separate publishing repository.

2. Walk raise sites instead of grepping README headings

Run a small AST walker over the packages you actually ship, not over tests that raise exceptions to simulate failure. Collect qualified names, locations, constant messages, and integer status literals when they appear beside the raise. Do not infer severity from class name prefixes such as Fatal or Transient, because those prefixes are conventions, not policy. Write every extracted row with extracted_at set to the commit SHA so reviewers can see whether the inventory matches HEAD.

3. Reconcile the ledger without deleting owned fields

Join new extracted keys onto existing rows by exception qualname plus module, not by customer copy, which will change during review. When a raise site disappears, mark the row removed_in rather than deleting it, so signed recovery text remains available for one release. When a new type appears, insert empty owned fields and fail CI until a reviewer fills them. Never copy owned fields from a similarly named exception, because similar names are not a policy decision.

4. Require named signatures on owned fields only

Store reviewer, reviewed_at, and owned_hash beside severity, customer copy, recovery steps, and retry class. Hash only those fields so an extractor rerun that updates lineno does not invalidate a still-correct runbook paragraph. Reject any row whose owned hash does not match the current text of the four owned fields. Reject rows that still contain a draft_recovery key, even if every owned field is already populated.

5. Render docs from the ledger, not from chat transcripts

A renderer may emit a type and status table without waiting for owned fields, and must label that table extracted. Customer-facing error pages, pager runbooks, and status-banner sentences must read only from signed owned fields. If a row is extracted but unsigned, the renderer prints a placeholder such as recovery: unsigned rather than asking a model to improvise. Tutorial chapters that mention errors should deep-link to ledger keys instead of restating recovery text.

Proposed extractor

The following script is a proposed, unexecuted example that uses only the CPython 3.11+ standard library. It walks raise nodes, records constant messages when present, and prints JSON rows for a later join against the ledger. It does not classify severity, does not invent HTTP codes, and does not call a network model during extraction. Operators should run it against a checkout they maintain and inspect the join before any renderer reads the file.

"""Proposed, unexecuted extractor: compile raise sites into ledger-shaped JSON."""
from __future__ import annotations

import ast
import json
import sys
from pathlib import Path
from typing import Any, Iterator, Optional

SCHEMA = "error-ledger.v1"
STATUS_HINT_NAMES = {"status", "status_code", "http_status", "code"}


def iter_py_files(root: Path) -> Iterator[Path]:
    for path in root.rglob("*.py"):
        if any(part.startswith(".") for part in path.parts):
            continue
        if "tests" in path.parts or "test" in path.name:
            continue
        yield path


def dotted_name(node: ast.AST) -> Optional[str]:
    if isinstance(node, ast.Name):
        return node.id
    if isinstance(node, ast.Attribute):
        left = dotted_name(node.value)
        if left is None:
            return None
        return f"{left}.{node.attr}"
    return None


def raise_qualname(node: ast.Raise) -> Optional[str]:
    if node.exc is None:
        return None
    target = node.exc.func if isinstance(node.exc, ast.Call) else node.exc
    return dotted_name(target)


def raise_message(node: ast.Raise) -> Optional[str]:
    if not isinstance(node.exc, ast.Call) or not node.exc.args:
        return None
    arg0 = node.exc.args[0]
    if isinstance(arg0, ast.Constant) and isinstance(arg0.value, str):
        return arg0.value
    return None


def function_status_hint(fn: ast.AST, raise_lineno: int) -> Optional[int]:
    best: Optional[int] = None
    best_line = -1
    for child in ast.walk(fn):
        if not isinstance(child, ast.Assign) or not hasattr(child, "lineno"):
            continue
        if child.lineno > raise_lineno:
            continue
        if not isinstance(child.value, ast.Constant) or not isinstance(
            child.value.value, int
        ):
            continue
        value = child.value.value
        if value < 400 or value > 599:
            continue
        names = [
            t.id.lower()
            for t in child.targets
            if isinstance(t, ast.Name)
        ]
        if not any(n in STATUS_HINT_NAMES or n.endswith("_status") for n in names):
            continue
        if child.lineno >= best_line:
            best = value
            best_line = child.lineno
    return best


def enclosing_function(tree: ast.AST, lineno: int) -> Optional[ast.AST]:
    found: Optional[ast.AST] = None
    for node in ast.walk(tree):
        if isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef)):
            end = getattr(node, "end_lineno", node.lineno)
            if node.lineno <= lineno <= (end or lineno):
                found = node
    return found


def extract_file(path: Path, root: Path) -> list[dict[str, Any]]:
    tree = ast.parse(path.read_text(encoding="utf-8"), filename=str(path))
    module = path.relative_to(root).as_posix()
    rows: dict[tuple[str, str], dict[str, Any]] = {}
    for node in ast.walk(tree):
        if not isinstance(node, ast.Raise):
            continue
        name = raise_qualname(node)
        if not name:
            continue
        key = (name, module)
        fn = enclosing_function(tree, node.lineno)
        status = function_status_hint(fn, node.lineno) if fn is not None else None
        if key not in rows:
            rows[key] = {
                "exc_qualname": name,
                "module": module,
                "lineno": node.lineno,
                "raise_count": 0,
                "message_const": raise_message(node),
                "http_status_literal": status,
            }
        row = rows[key]
        row["raise_count"] += 1
        row["lineno"] = min(row["lineno"], node.lineno)
        if row["message_const"] is None:
            row["message_const"] = raise_message(node)
        if row["http_status_literal"] is None:
            row["http_status_literal"] = status
    return list(rows.values())


def main() -> None:
    if len(sys.argv) != 2:
        sys.stderr.write("usage: extract_raise_sites.py <src-root>\n")
        sys.exit(2)
    root = Path(sys.argv[1])
    extracted: list[dict[str, Any]] = []
    for path in sorted(iter_py_files(root)):
        extracted.extend(extract_file(path, root))
    json.dump({"schema": SCHEMA, "rows": extracted}, sys.stdout, indent=2, sort_keys=True)
    sys.stdout.write("\n")


if __name__ == "__main__":
    main()

Dynamic raise calls that build classes at runtime will not appear, which is an intended gap rather than a parser bug. String messages constructed with formatting or translation catalogs are stored as null, because the constant is not the production string. Nearby integer literals named status or code are recorded as hints; computed status maps remain null on purpose. Do not treat extractor stdout as a finished runbook, even when the JSON is pretty-printed and looks documentation-ready.

Proposed local commands, still unexecuted, look like this:

python tools/extract_raise_sites.py src > /tmp/extracted.json
python tools/check_error_ledger.py docs/error-ledger.json /tmp/extracted.json

A ledger row after extraction, before review, should look like the following unlabeled example. Owned strings are empty on purpose so CI can fail closed.

{
  "schema": "error-ledger.v1",
  "rows": [
    {
      "exc_qualname": "ChargeError",
      "module": "payments/charges.py",
      "lineno": 142,
      "raise_count": 2,
      "message_const": "charge failed",
      "http_status_literal": 402,
      "severity": "",
      "customer_copy": "",
      "recovery_steps": "",
      "retry_class": "",
      "reviewer": "",
      "reviewed_at": "",
      "owned_hash": "",
      "removed_in": null
    }
  ]
}

Proposed CI gate

The gate must fail closed on missing owned fields, stale hashes, leftover draft keys, and vanished raise sites without removed_in. Warning-only modes recreate the original problem because unsigned recovery text still reaches readers through the renderer. Teams that publish docs from a pipeline should run this gate on the docs job as well as on the unit-test job. The script below is a proposed checker; wire it as a required status rather than as an optional linter comment.

"""Proposed, unexecuted CI gate for docs/error-ledger.json."""
from __future__ import annotations

import hashlib
import json
import sys
from pathlib import Path

OWNED = ("severity", "customer_copy", "recovery_steps", "retry_class")
FORBIDDEN_DRAFT = "draft_recovery"


def owned_hash(row: dict) -> str:
    payload = {k: row.get(k, "") for k in OWNED}
    blob = json.dumps(payload, sort_keys=True, ensure_ascii=True).encode("utf-8")
    return hashlib.sha256(blob).hexdigest()


def row_key(row: dict) -> str:
    return f"{row.get('module', '')}::{row.get('exc_qualname', '')}"


def main() -> None:
    if len(sys.argv) != 3:
        sys.stderr.write("usage: check_error_ledger.py ledger.json extracted.json\n")
        sys.exit(2)
    ledger = json.loads(Path(sys.argv[1]).read_text(encoding="utf-8"))
    extracted = json.loads(Path(sys.argv[2]).read_text(encoding="utf-8"))
    errors: list[str] = []
    live = {row_key(r): r for r in extracted.get("rows", [])}
    seen: set[str] = set()
    for row in ledger.get("rows", []):
        key = row_key(row)
        seen.add(key)
        if row.get(FORBIDDEN_DRAFT):
            errors.append(f"{key}: draft_recovery must be stripped before merge")
        removed = row.get("removed_in")
        if key not in live and not removed:
            errors.append(f"{key}: missing from extract and removed_in is empty")
        if key in live and removed:
            errors.append(f"{key}: still raised but removed_in is set")
        for field in OWNED:
            if not str(row.get(field) or "").strip():
                errors.append(f"{key}: owned field {field} is empty")
        if row.get("owned_hash") != owned_hash(row):
            errors.append(f"{key}: owned_hash mismatch")
        if not str(row.get("reviewer") or "").strip():
            errors.append(f"{key}: reviewer is empty")
    for key in live:
        if key not in seen:
            errors.append(f"{key}: extracted but absent from ledger")
    if errors:
        sys.stderr.write("error-ledger check failed:\n")
        for item in errors:
            sys.stderr.write(f"  - {item}\n")
        sys.exit(1)
    sys.stdout.write(f"ok: {len(seen)} ledger rows\n")


if __name__ == "__main__":
    main()

A practical test plan for the gate, still a proposal rather than a reported result, is four fixtures. Fixture A extracts two raise sites and an empty owned block, and must fail. Fixture B fills owned fields but leaves draft_recovery populated, and must fail. Fixture C matches hashes and reviewers with no draft key, and must pass. Fixture D drops a raise site without removed_in, and must fail.

Unsigned drafts beside the ledger, not inside it

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

MonkeyCode's free model access and free server option can host a drafting session that reads extracted columns and proposes recovery sentences into draft_recovery only. Those sentences are not evidence, not policy, and not mergeable output, regardless of how fluent they look in preview. Keep any drafting workspace beside the ledger rather than on the renderer path that publishes customer pages.

Limitations the extractor will not paper over

The walker cannot see exceptions raised from C extensions, from eval, or from dynamically assembled type objects. Integer status codes that are computed, looked up in a dict, or read from configuration appear as null rather than as guessed numbers. Owned fields still need incident policy that lives outside the repository, such as paging rosters and vendor escalation paths. Message constants are hints for reviewers, not contracts with clients who may receive localized or formatted text.

Severity labels are not portable across organizations, so the schema should reference a local enum rather than a universal scale. Customer copy may be legally reviewed elsewhere; the ledger then stores a ticket identifier rather than a shadow of that text. This workflow also assumes Python AST access in CI, which is a poor fit for polyglot services that raise equivalent errors in several languages. For those services, replace the walker with language-specific extractors that still write the same ledger keys.

Who should not use this approach

Do not adopt this ledger if your product has no stable exception types and communicates failure only through unstructured logs. Do not use it to auto-publish status-page incident language, because extracted class names are not customer-safe descriptions of an ongoing outage. Teams without a named reviewer for operational copy will simply fill owned fields with placeholder adjectives to satisfy CI. That failure mode is worse than a missing page, because it presents invented recovery advice as if it had been signed.

Writers who want a single narrative README should keep this inventory as a data file and narrate from it, not invert the process. Security-sensitive errors that must not disclose existence of a resource may need a redaction column the extractor cannot infer. If your compliance process forbids generated text anywhere in the docs tree, skip the draft column entirely and write owned fields by hand. Local stubs that exist only to satisfy imports should be excluded from the walk, or they will mint error rows nobody can recover.

What this does not replace

An error ledger is not an SLA, not a postmortem, and not a substitute for tracing in production. It will not tell you whether a raise site is reachable, only that the site exists in the current tree. Reachability remains a test and observability problem, which should be linked from recovery_steps when evidence exists. Keep those links in owned fields so a regenerated inventory cannot invent dashboards that nobody actually runs.

The useful part of the system is the split between extracted raise sites and signed operational claims. Renderers, CI hashes, and optional drafts only exist to protect that split under ordinary merge pressure. If a paragraph cannot be traced to a raise site or to a named reviewer, it does not belong on a customer error page. Optional assistants do not change the merge rule, because fluency is not a substitute for a reviewer hash.