Compile Error Catalogs From Source Constants; Hand-Write Impact, Recovery, and Escalation

Troubleshooting copy is unsafe when a model invents recovery steps that never existed in product behavior. Compile identity columns from source constants, then refuse to publish until a human signs impact, recovery, and escalation. Models may draft codes, symbol names, source paths, and frozen message templates already present in the tree. Humans must own severity, customer impact, data-loss language, retry advice, and the paging policy for operators.

Why identity columns and recovery claims must stay separate

Most public error catalogs start as a spreadsheet that later drifts away from the shipping codebase. A constant is renamed in source while the public troubleshooting page still lists the old code. The opposite failure is worse: generated prose claims a retry is safe when the handler already committed a partial write. Keeping extracted identity apart from operator-owned claims makes both failures visible in continuous integration.

What a model may draft, and what a human must own

Treat every catalog row as two records that share a single error code as the join key. The extract record is mechanical and can be rebuilt on every commit without editorial judgment. The claim record is a signed statement about user harm, and it must not be filled from chat output. The table below is the publish contract this workflow enforces before any troubleshooting page is released.

Field	Source of truth	Model may draft	Merge rule
code	source constant	yes, from extract	must equal source
symbol	identifier	yes	must equal source
source_path	repository path	yes	file must exist
message_template	string literal	yes	byte-stable with source
http_status	annotation in source	only if present	omit when absent
severity	ops policy	no	human signature required
customer_impact	product language	no	human signature required
data_loss	storage semantics	no	human signature required
recovery_steps	runbook owner	no	human signature required
retry_safe	handler behavior	no	human signature required
escalation	on-call policy	no	human signature required

Identity fields are invalid as soon as they cannot be reproduced from the source extract. Claim fields are invalid when they are empty, model-authored, or copied from a neighboring error without a new signature. That rule is stricter than a documentation style guide, and the extra strictness is intentional. Operators can extend the field list, but they should not move recovery text into the extract file.

A two-file catalog instead of a generated page

Store extracts and claims in files the compiler can hash, not in a chat transcript. The extract file is disposable and should be rewritten whenever the error constants change in source. The claims file is durable, reviewed, and owned by the person who answers the pager. Publishing concatenates both files only after identity rows and signed claims satisfy the merge rules.

Proposed extract fixture, not live telemetry:

# catalogs/errors.extract.yaml  (generated; do not edit)
version: 1
errors:
  - code: INV-1404
    symbol: INVOICE_NOT_FOUND
    source_path: src/billing/errors.py
    message_template: Invoice {invoice_id} was not found.
    http_status: 404
  - code: INV-1409
    symbol: INVOICE_ALREADY_FINAL
    source_path: src/billing/errors.py
    message_template: Invoice {invoice_id} is already finalized.
    http_status: 409

Human-owned claims for the same codes:

# catalogs/errors.claims.yaml  (human-owned; models must not write)
version: 1
claims:
  INV-1404:
    signed_by: billing-oncall
    signed_at: '2026-09-16'
    severity: low
    customer_impact: The customer cannot download a missing invoice PDF.
    data_loss: none
    retry_safe: true
    recovery_steps:
      - Confirm the invoice id in the billing dashboard.
      - If the invoice was deleted, recreate it from the order record.
    escalation: No page. File a billing ticket if the order exists without an invoice.
  INV-1409:
    signed_by: billing-oncall
    signed_at: '2026-09-16'
    severity: medium
    customer_impact: The customer cannot edit line items after finalization.
    data_loss: none if the client stops retrying POST
    retry_safe: false
    recovery_steps:
      - Do not retry the finalize call.
      - Issue a credit note if the finalized totals are wrong.
    escalation: Page billing-oncall when finalization loops exceed five conflicts in ten minutes.

Numbered workflow

Step 1: Extract identity from source constants

Keep error identity in one module so the extractor never needs to scrape production log lines. String-scraping from logs mixes runtime interpolation with the catalog and will create unstable codes. A single constants file is enough for this tutorial, and it keeps the compiler fully deterministic. The module below is a proposed fixture, not an extract from a live billing system.

# src/billing/errors.py
from dataclasses import dataclass

SOURCE_PATH = 'src/billing/errors.py'

@dataclass(frozen=True)
class AppError:
    code: str
    symbol: str
    message_template: str
    http_status: int
    source_path: str = SOURCE_PATH

INVOICE_NOT_FOUND = AppError(
    code='INV-1404',
    symbol='INVOICE_NOT_FOUND',
    message_template='Invoice {invoice_id} was not found.',
    http_status=404,
)

INVOICE_ALREADY_FINAL = AppError(
    code='INV-1409',
    symbol='INVOICE_ALREADY_FINAL',
    message_template='Invoice {invoice_id} is already finalized.',
    http_status=409,
)

CATALOG = (INVOICE_NOT_FOUND, INVOICE_ALREADY_FINAL)

The src and src/billing directories need empty __init__.py files so the import in the compiler resolves. PyYAML is the only extra dependency in this fixture, and tests call the gate through a subprocess. Commands assume the repository root is the working directory for every compiler and gate command.

Step 2: Compile extract YAML without claim fields

The compiler may only emit identity columns, and it should ignore any prose sitting in the claims file. If a language model assists at all, it is limited to formatting the extract from the CATALOG tuple. It must not invent HTTP statuses, paths, or message templates that are missing from AppError.

# tools/compile_error_extract.py
# Proposed local compiler. Run against the constants module, not against chat text.
from __future__ import annotations

import importlib
import pathlib
import sys

import yaml

ROOT = pathlib.Path(__file__).resolve().parents[1]
sys.path.insert(0, str(ROOT))


def compile_extract(module_path: str) -> dict:
    mod = importlib.import_module(module_path)
    rows = []
    for err in mod.CATALOG:
        rows.append(
            {
                'code': err.code,
                'symbol': err.symbol,
                'source_path': err.source_path,
                'message_template': err.message_template,
                'http_status': err.http_status,
            }
        )
    return {'version': 1, 'errors': rows}


def main() -> None:
    extract = compile_extract('src.billing.errors')
    out = ROOT / 'catalogs' / 'errors.extract.yaml'
    out.parent.mkdir(parents=True, exist_ok=True)
    out.write_text(yaml.safe_dump(extract, sort_keys=False), encoding='utf-8')
    print(f"wrote {out} ({len(extract['errors'])} identity rows)")


if __name__ == '__main__':
    main()

Step 3: Reject unsigned or model-touched claims

Claim fields need an explicit signer and a calendar date before the catalog can merge. Empty strings, placeholder prose, and keys missing from the claims file all fail the gate. A simple marker file records that a model last wrote a claim, which the test treats as a merge blocker. Revert that marker only after a human rewrites the claims file in an ordinary editor.

# tools/gate_error_catalog.py
# Fail the build when identity drifts or recovery claims are unsigned.
from __future__ import annotations

import pathlib
import sys

import yaml

ROOT = pathlib.Path(__file__).resolve().parents[1]
CLAIM_KEYS = {
    'signed_by',
    'signed_at',
    'severity',
    'customer_impact',
    'data_loss',
    'retry_safe',
    'recovery_steps',
    'escalation',
}
PLACEHOLDERS = {'todo', 'tbd', 'ask the model', 'generated'}


def load(path: pathlib.Path) -> dict:
    return yaml.safe_load(path.read_text(encoding='utf-8'))


def gate() -> list[str]:
    extract = load(ROOT / 'catalogs' / 'errors.extract.yaml')
    claims = load(ROOT / 'catalogs' / 'errors.claims.yaml')['claims']
    model_touch = ROOT / 'catalogs' / '.model-touched-claims'
    failures: list[str] = []

    if model_touch.exists():
        failures.append('claims file was last written in a model lane; revert and resign')

    codes = [row['code'] for row in extract['errors']]
    if len(codes) != len(set(codes)):
        failures.append('extract contains duplicate error codes')

    for row in extract['errors']:
        code = row['code']
        if code not in claims:
            failures.append(f'{code}: missing human claim record')
            continue
        rec = claims[code]
        missing = CLAIM_KEYS - set(rec)
        if missing:
            failures.append(f'{code}: unsigned fields {sorted(missing)}')
        for key in CLAIM_KEYS:
            val = rec.get(key)
            text = ' '.join(val) if isinstance(val, list) else str(val or '')
            if text.strip().lower() in PLACEHOLDERS or not str(val).strip() and val is not False:
                failures.append(f'{code}: {key} is empty or placeholder')
        recovery = str(rec.get('recovery_steps')).lower()
        if rec.get('retry_safe') is True and 'do not retry' in recovery:
            failures.append(f'{code}: retry_safe=true conflicts with recovery text')

    extra = set(claims) - set(codes)
    for code in sorted(extra):
        failures.append(f'{code}: claim has no matching source constant')
    return failures


def main() -> None:
    failures = gate()
    if failures:
        print('catalog gate failed:')
        for item in failures:
            print(f' - {item}')
        sys.exit(1)
    print('catalog gate passed')


if __name__ == '__main__':
    main()

Step 4: Add a reproducible test and a local command

The tests below are fixture checks for local files, and they are not a production incident study. They lock the two failure classes this workflow cares about: drifted identity and unsigned recovery claims. Run the compiler first so the extract file matches source before the gate reads it.

# tests/test_error_catalog_gate.py
# Proposed fixture tests. They exercise files in catalogs/, not live incidents.
from pathlib import Path
import subprocess
import sys

ROOT = Path(__file__).resolve().parents[1]


def run_gate() -> subprocess.CompletedProcess:
    return subprocess.run(
        [sys.executable, str(ROOT / 'tools' / 'gate_error_catalog.py')],
        cwd=ROOT,
        capture_output=True,
        text=True,
    )


def test_gate_passes_on_signed_fixture():
    result = run_gate()
    assert result.returncode == 0, result.stdout + result.stderr
    assert 'catalog gate passed' in result.stdout

Labeled unexecuted check: drop INV-1404 from errors.claims.yaml and rerun the gate. Expected stdout includes INV-1404: missing human claim record, and the process should exit with status 1.

.PHONY: catalog-gate
catalog-gate:
    python tools/compile_error_extract.py
    python tools/gate_error_catalog.py
    python -m pytest tests/test_error_catalog_gate.py -q

python tools/compile_error_extract.py
python tools/gate_error_catalog.py
python -m pytest tests/test_error_catalog_gate.py -q

Completing the published page

A renderer may print identity columns as a table and recovery claims as unmodified prose. It must not paraphrase claim text, because paraphrase is how retry advice quietly changes meaning. If a new constant appears in source, the page stays unpublished until a human adds a claim row. If a constant disappears, the claim becomes extra and the gate fails until the owner archives it.

# tools/render_error_page.py
# Proposed renderer. Prints Markdown; does not rewrite claim prose.
from __future__ import annotations

import pathlib
import yaml

ROOT = pathlib.Path(__file__).resolve().parents[1]


def render() -> str:
    extract = yaml.safe_load((ROOT / 'catalogs' / 'errors.extract.yaml').read_text())
    claims = yaml.safe_load((ROOT / 'catalogs' / 'errors.claims.yaml').read_text())['claims']
    lines = ['# Billing errors', '']
    for row in extract['errors']:
        rec = claims[row['code']]
        lines.append(f"## {row['code']} ({row['symbol']})")
        lines.append(f"Status: {row['http_status']}")
        lines.append(f"Message: `{row['message_template']}`")
        lines.append(f"Impact: {rec['customer_impact']}")
        lines.append(f"Data loss: {rec['data_loss']}")
        lines.append('Recovery:')
        for step in rec['recovery_steps']:
            lines.append(f'- {step}')
        lines.append(f"Escalation: {rec['escalation']}")
        lines.append('')
    return '\n'.join(lines)


if __name__ == '__main__':
    print(render())

Where a free model session can participate

Mechanical extract formatting is the only step that benefits from a model sitting in the loop. Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode's free model access and free server option can host that extract-and-review session while the gate still runs as local commands. Point that session at compile_error_extract.py, and never at errors.claims.yaml, so severity and recovery stay human-owned.

Limitations

This workflow does not prove that recovery steps work in production, because it never executes the handler. Simple extractors also miss errors constructed by string concatenation, dynamic factories, or third-party library wrappers. The duplicate-code check will not catch two symbols that share a user-facing message with different codes.

Signed claims can still be factually wrong, because the gate only proves a human accepted ownership. Teams that need legal review for data-loss language should add a second signer field, which this fixture does not implement. Timestamp fields are calendar dates in this example, not cryptographic signatures, and they can be forged in a local commit. The .model-touched-claims marker is only a convention, and it is not an editor-proof audit log.

Who should not use this approach

Do not adopt this compiler when error codes exist only as unstructured lines in production logs. Do not use it to generate customer-facing legal notices, security advisories, or medical/safety recovery instructions. Do not let a model fill claim fields for a first draft and then rubber-stamp the YAML. Small libraries with five errors and one maintainer may prefer a handwritten page with a link to the constants file.