Generated error pages should list only codes, HTTP statuses, and source locations the parser can prove. Recovery text, customer tone, and any availability promise must stay human-owned, dated, and signed before merge. Mixing those two classes of sentences is how runbooks silently contradict the service that actually ships. The workflow below extracts a machine catalog, overlays a signed recovery file, and fails CI when cells remain unsigned.
Why published error pages go stale
Exception classes change when handlers start returning 409 instead of 400, but published copy often lags by several releases. Authors then paste model-written recovery paragraphs that recommend retries the HTTP client does not perform. Reviewers see fluent English and miss the missing Idempotency-Key requirement that only exists in code. A compiler cannot invent that operational judgment, and it should not be asked to.
Help-center articles also collapse several audiences into one paragraph, which hides who may retry and who must open a ticket. Platform engineers want the exception class, support writers want a stable public code, and legal reviewers want SLA verbs out of chat output. Splitting the page into a generated index and a signed overlay makes those audiences review different files. The pull request then shows hash drift instead of a six-hundred-word narrative diff.
Decision table: generated cells versus signed cells
Treat every error-doc field as either parser-visible or human-owned before anyone opens an editor. The table below is the review contract this repository would enforce on every documentation pull request.
| Field | Source of truth | Model role | Merge rule |
|---|---|---|---|
error_code |
class attribute | extract only | must match AST |
http_status |
class attribute | extract only | must match AST |
source |
file and line | extract only | must match AST |
stable_since |
optional version attribute | extract if present | warn when missing |
user_facing_summary |
human overlay | draft off-tree only | signer and date required |
recovery_steps |
human overlay | draft off-tree only | signer and date required |
retry_safe |
human overlay, checked against the client | never infer |
true / false / unknown plus signer |
sla_language |
legal or ops overlay | never draft for merge | named owner required |
public |
human overlay | never infer | default false until signed |
Parser-visible fields are regenerated on every CI run so the markdown table cannot drift from HEAD. Human-owned fields live in a separate JSON overlay whose keys are error codes, not prose filenames. If a new exception appears without an overlay row, the gate fails closed instead of publishing an empty recovery story.
1. Annotate exceptions with stable machine codes
Do not scrape HTTP status integers out of unstructured raise HTTPException calls if the project can afford a base class. A tiny base type gives the compiler a single attribute layout and keeps codes stable when class names refactor. The following example is labeled as a local pattern, not as a framework or language recommendation.
# errors/base.py — worked example, run locally against this tree
class AppError(Exception):
error_code: str = ""
http_status: int = 500
stable_since: str = ""
public: bool = False # catalog default; overlay may promote later
def __init__(self, detail: str = ""):
super().__init__(detail)
self.detail = detail
class ConflictError(AppError):
error_code = "order.conflict"
http_status = 409
stable_since = "2026.09"
class RateLimitedError(AppError):
error_code = "order.rate_limited"
http_status = 429
stable_since = "2026.09"
Each public error must carry a stable error_code string that survives class renames during ordinary refactors. Integer status codes remain on the class so handlers cannot return a different status than the catalog prints. retry_safe is intentionally absent here because that claim is operational, not a compiler fact.
2. Compile the catalog from the AST
Walk only files under a declared errors package so random Exception subclasses do not enter the public index. The compiler records the source path and line number for each class, then hashes the parser-visible tuple. That hash later proves the overlay still describes the same status and code that HEAD actually raises.
# tools/compile_error_catalog.py — worked example
from __future__ import annotations
import ast
import hashlib
import json
from pathlib import Path
ROOT = Path("errors")
OUT = Path("docs/generated/error_catalog.json")
def class_str(node: ast.ClassDef, name: str) -> str | None:
for item in node.body:
if isinstance(item, ast.Assign):
for target in item.targets:
if isinstance(target, ast.Name) and target.id == name:
if isinstance(item.value, ast.Constant) and isinstance(
item.value.value, str
):
return item.value.value
if isinstance(item, ast.AnnAssign) and isinstance(item.target, ast.Name):
if item.target.id == name and isinstance(item.value, ast.Constant):
if isinstance(item.value.value, str):
return item.value.value
return None
def class_int(node: ast.ClassDef, name: str) -> int | None:
for item in node.body:
if isinstance(item, ast.Assign):
for target in item.targets:
if isinstance(target, ast.Name) and target.id == name:
if isinstance(item.value, ast.Constant) and isinstance(
item.value.value, int
):
return item.value.value
return None
def is_app_error(node: ast.ClassDef) -> bool:
return any(
isinstance(base, ast.Name) and base.id == "AppError" for base in node.bases
)
def compile_catalog() -> list[dict]:
rows: list[dict] = []
for path in sorted(ROOT.rglob("*.py")):
tree = ast.parse(path.read_text(encoding="utf-8"), filename=str(path))
for node in tree.body:
if not isinstance(node, ast.ClassDef) or not is_app_error(node):
continue
code = class_str(node, "error_code")
status = class_int(node, "http_status")
if not code or status is None:
raise SystemExit(f"unsigned machine fields in {path}:{node.lineno}")
visible = f"{code}|{status}|{path.as_posix()}|{node.lineno}"
digest = hashlib.sha256(visible.encode()).hexdigest()[:16]
rows.append(
{
"error_code": code,
"http_status": status,
"class_name": node.name,
"source": f"{path.as_posix()}:{node.lineno}",
"stable_since": class_str(node, "stable_since") or "",
"source_hash": digest,
}
)
rows.sort(key=lambda row: row["error_code"])
return rows
def main() -> None:
rows = compile_catalog()
OUT.parent.mkdir(parents=True, exist_ok=True)
OUT.write_text(json.dumps(rows, indent=2) + "\n", encoding="utf-8")
print(f"wrote {len(rows)} rows to {OUT}")
if __name__ == "__main__":
main()
Run the compiler from a clean checkout so the output path stays deterministic across agents.
python3 tools/compile_error_catalog.py
python3 -m json.tool docs/generated/error_catalog.json | head
Commit docs/generated/error_catalog.json only after CI rewrites it, never after a chat window pastes a table. Reviewers then diff codes and hashes instead of arguing about heading synonyms. Duplicate error_code values should fail the compiler; colliding public codes are a product defect, not a documentation style issue.
3. Keep recovery copy in a signed overlay
The overlay is JSON so a linter can require keys without parsing Markdown personality. Models may propose user_facing_summary and recovery_steps into a scratch file, but CI reads only the signed overlay. Each row needs signer, signed_at, and source_hash equal to the current catalog hash for that code.
{
"order.conflict": {
"public": true,
"retry_safe": false,
"user_facing_summary": "This order already reflects a competing update.",
"recovery_steps": [
"GET the order by id and compare updated_at before retrying a write.",
"Do not retry the original PUT without a new If-Match value."
],
"sla_language": "",
"signer": "docs-oncall",
"signed_at": "2026-09-23",
"source_hash": "replace-with-catalog-hash"
}
}
Leave sla_language empty unless a named owner pastes an already approved sentence. Empty is a valid signed state; invented uptime verbs are not. retry_safe must stay false for order.conflict because a second PUT without a new precondition can overwrite a later write.
Render the public page from both files in a third command that refuses to interpolate unsigned rows. Keep customer Markdown as a projection, not as the store of record, so copy edits cannot hide a status-code change. If your help center requires HTML, generate it from the same projection instead of maintaining a parallel article.
4. Fail merge when cells are unsigned or stale
The gate is a short checker, not a documentation style linter. It loads the compiled catalog and the overlay, then applies four closed-world rules. Failures should print error codes, not paragraph numbers, so the reviewer opens the overlay directly.
- Every catalog
error_codemust exist in the overlay, even whenpublicis stillfalse. - Overlay
source_hashmust equal the catalog hash, or the recovery copy is stale against HEAD. -
public: truerows must include a non-emptyuser_facing_summary, a non-emptyrecovery_stepslist, and asigner. -
sla_languagemay be empty, but a non-empty value requires a signer that matches an owners file.
# tools/check_error_overlay.py — worked example
from __future__ import annotations
import json
from pathlib import Path
CATALOG = Path("docs/generated/error_catalog.json")
OVERLAY = Path("docs/signed/error_overlay.json")
OWNERS = {"docs-oncall", "sre-oncall", "legal-docs"}
def main() -> None:
catalog = json.loads(CATALOG.read_text(encoding="utf-8"))
overlay = json.loads(OVERLAY.read_text(encoding="utf-8"))
failures: list[str] = []
catalog_codes = [row["error_code"] for row in catalog]
if len(catalog_codes) != len(set(catalog_codes)):
failures.append("duplicate error_code in catalog")
for row in catalog:
code = row["error_code"]
entry = overlay.get(code)
if entry is None:
failures.append(f"{code}: missing overlay row")
continue
if entry.get("source_hash") != row["source_hash"]:
failures.append(f"{code}: stale source_hash")
if entry.get("retry_safe") not in {True, False, "unknown"}:
failures.append(f"{code}: retry_safe must be true, false, or unknown")
if entry.get("public"):
if not entry.get("user_facing_summary"):
failures.append(f"{code}: public row missing user_facing_summary")
if not entry.get("recovery_steps"):
failures.append(f"{code}: public row missing recovery_steps")
if not entry.get("signer"):
failures.append(f"{code}: public row missing signer")
sla = entry.get("sla_language") or ""
if sla and entry.get("signer") not in OWNERS:
failures.append(f"{code}: sla_language signer is not in OWNERS")
extra = set(overlay) - set(catalog_codes)
for code in sorted(extra):
failures.append(f"{code}: overlay row has no catalog source")
if failures:
raise SystemExit("\n".join(failures))
print(f"overlay ok for {len(catalog)} codes")
if __name__ == "__main__":
main()
Wire both commands in the documentation job so a green Markdown preview cannot hide a red overlay.
python3 tools/compile_error_catalog.py
python3 tools/check_error_overlay.py
Add a unit fixture that mutates http_status on ConflictError and asserts the checker fails. That test documents the policy better than a style guide paragraph about honesty. If the fixture cannot fail the gate, the workflow is theater and should not ship.
Where a drafting workspace fits, and where it does not
Disclosure: This article was prepared as part of MonkeyCode's product outreach. A model may draft candidate recovery bullets against the compiled catalog, especially when the exception docstring already states a precondition. Those drafts belong in a disposable branch or scratch directory, never in docs/signed/error_overlay.json without a human signer. MonkeyCode's free model access and free server option can host that scratch experiment when a team wants the compiler and a drafting agent on the same throwaway workspace.
Do not ask the model to fill sla_language, retry_safe, or public. Those fields encode legal exposure, client behavior, and product policy that the AST cannot see. If a draft recovery step mentions refunds, credits, or minutes of downtime, delete the sentence before review rather than tightening the adjectives. The useful model output is a checklist of questions for the signer, not a paragraph that sounds finished.
Limitations
This compiler only understands class attributes on AppError subclasses, so dynamic HTTPException factories remain invisible. Teams that raise anonymous status codes in middleware will under-count public errors and should extend the extractor or stop raising anonymous codes. Hashing file paths means a file move looks like a stale overlay even when status and code are unchanged, which is noisy but safer than silent moves.
The overlay does not prove that recovery steps match the client SDK. retry_safe: false can still be wrong if a mobile wrapper retries 409 in production. Pair this gate with a contract test that reads the same overlay and asserts the official client refuses those retries. Without that second test, the catalog is a documentation index, not an operational control.
Generated Markdown can still be copied into a CMS that strips the source_hash column. Once the hash leaves the page, support writers cannot see staleness and will edit prose in the CMS instead of the overlay. Publish the hash beside the public code, or keep the help center on the generated projection.
Who should not use this approach
Skip the pipeline if the API has fewer than a handful of stable error codes and a single owner already writes every status by hand. The overlay ceremony costs more than a one-page runbook in that shape. Also skip it when exception types are generated from a protobuf package you do not control, unless you compile from that schema instead of Python classes.
Do not use this workflow to launder SLA sentences through a coding model and then collect a signer after the fact. The signer is the author of the operational claim, not a witness to fluent output. Security-sensitive errors that reveal rate-limit internals or fraud checks should stay public: false even when the compiler can see them.
The core rule stays narrow after the tooling is in place. Compilers may print codes, statuses, and source coordinates that HEAD can prove. Humans still own recovery copy, retry policy, publication scope, and every sentence that sounds like a promise.
Top comments (0)