Error catalogs go stale when a model invents customer-facing recovery that on-call never rehearsed during incident drills. An OpenAPI document can still compile status codes, error names, and response fields without promising any operational action. Humans must own severity, user-visible copy, pager policy, and rollback, because those sentences change how incidents end. This workflow compiles catalog rows from OpenAPI, bounds optional description drafts, and fails the build when generated files grow recovery advice.
Keep protocol facts out of incident language
Most error pages mix a protocol fact with a human promise inside a single table cell. The fact is that POST /exports can return 409 with code export_in_progress. The promise is that customers should wait, and that on-call should not page overnight for that conflict. A model that writes both cells will sound complete while teaching the wrong operational reflex. Compiling the protocol row from OpenAPI lets the catalog track every added response, while a signed claims file still controls severity and recovery.
Reviewers miss the mix because the status code is correct, so the surrounding sentence looks reviewed as well. The compiler below never emits severity, pager policy, rollback steps, or support escalation from generated output. Those strings can appear only in the human-rendered recovery page that is built from claims.
Decision table for each error row
| Field | Source | Writer | Generated catalog |
|---|---|---|---|
| HTTP status | OpenAPI responses keys |
compiler | allowed |
error code
|
response schema enum or property | compiler | allowed |
| response fields | schema properties names |
compiler | allowed |
| endpoint + method | OpenAPI paths
|
compiler | allowed |
| one-line meaning | response description or bounded draft |
compiler or model | allowed if under budget |
| severity | claims file | human | refused |
| customer copy | claims file | human | refused |
| page on-call | claims file | human | refused |
| rollback / data-loss note | claims file | human | refused |
Treat extra sentences as claims-file edits rather than compiler features. If a reviewer wants a retry window in minutes, that guidance belongs in the signed file even when it is short.
Files the compiler may read
Keep four inputs and two outputs, and refuse any job that writes the claims file. The generated catalog may list methods, paths, status codes, error codes, field names, and bounded meanings. It must not contain paging, rollback, data-loss, or support language unless those words are literal error codes. The recovery page is rendered from claims with a template that does not call a model.
docs/errors/
openapi.json # reviewed protocol facts
claims.json # human-owned severity and recovery
description-budget.toml # max sentences and forbid list
compile_error_catalog.py
gate_error_catalog.py
catalog.generated.md
recovery.md
Step 1: Freeze the OpenAPI responses you already ship
Start from the OpenAPI file that already gates client generation, not from a chat transcript about possible errors. If a status block is missing from that file, add it through API review before the catalog can mention it. The fragment below is a sample specification for the compiler, not a contract for a named product.
{
"openapi": "3.1.0",
"info": {"title": "Export Service", "version": "0.0.0"},
"paths": {
"/exports": {
"post": {
"operationId": "createExport",
"responses": {
"201": {
"description": "Export job accepted and queued"
},
"409": {
"description": "Another export job is already running",
"content": {
"application/json": {
"schema": {
"type": "object",
"required": ["code", "retryable"],
"properties": {
"code": {
"type": "string",
"enum": ["export_in_progress"]
},
"retryable": {"type": "boolean"}
}
}
}
}
},
"500": {
"description": "Unhandled failure while writing the export file",
"content": {
"application/json": {
"schema": {
"type": "object",
"required": ["code"],
"properties": {
"code": {
"type": "string",
"enum": ["export_write_failed"]
}
}
}
}
}
}
}
}
}
}
}
Commit response blocks through the same review path that already protects handler code. A polished meaning sentence is worthless if the status code can drift without a pull request. Success responses such as 201 can be compiled as rows, but they still need a claims record that says paging is off.
Step 2: Hand-write recovery claims for every compiled row
Do not ask a model to propose severity or customer copy, even as a starting draft that humans later edit. Severity encodes paging cost, and customer copy encodes legal tone, so both need an accountable owner. The JSON below is an operator template; it is not guidance for any production incident.
{
"errors": {
"POST /exports 409 export_in_progress": {
"severity": "warn",
"page_oncall": false,
"customer_copy": "Another export is still running. Retry after that job finishes.",
"rollback": "No rollback. Wait for the in-flight job or cancel it from the operator runbook.",
"owner": "data-platform-oncall"
},
"POST /exports 500 export_write_failed": {
"severity": "error",
"page_oncall": true,
"customer_copy": "The export did not finish. No partial file was published.",
"rollback": "Delete any incomplete object the writer left, then replay from the last cursor.",
"owner": "data-platform-oncall"
},
"POST /exports 201": {
"severity": "info",
"page_oncall": false,
"customer_copy": "The export job was accepted.",
"rollback": "Not an error. No rollback.",
"owner": "data-platform-oncall"
}
}
}
Every compiled row must have a claims key, and extra claims keys fail the gate. That rule prevents a new 503 from shipping with a compiled status and no human recovery sentence. Owners should be rotation names, not personal handles, so the file still makes sense after a staffing change.
Step 3: Compile catalog rows only from OpenAPI
The compiler walks paths, emits a Markdown table, and refuses to read claims.json while printing the generated page. Meanings come from each response description first, which already passed API review. A model draft is used only when that field is empty and the budget file still has remaining sentences.
#!/usr/bin/env python3
"""Compile catalog.generated.md from openapi.json only."""
from __future__ import annotations
import json
import pathlib
import re
import sys
ROOT = pathlib.Path("docs/errors")
SPEC = json.loads((ROOT / "openapi.json").read_text(encoding="utf-8"))
OUT = ROOT / "catalog.generated.md"
FORBIDDEN = re.compile(
r"\b(page|pager|rollback|data loss|immediately|customer|support|on-call)\b",
re.I,
)
def row_id(method: str, path: str, status: str, code: str) -> str:
base = f"{method.upper()} {path} {status}"
return f"{base} {code}" if code else base
def error_code(body: dict) -> str:
schema = (
body.get("content", {})
.get("application/json", {})
.get("schema", {})
)
enum = schema.get("properties", {}).get("code", {}).get("enum") or []
return enum[0] if len(enum) == 1 else ""
def field_names(body: dict) -> str:
schema = (
body.get("content", {})
.get("application/json", {})
.get("schema", {})
)
props = schema.get("properties", {})
return ", ".join(sorted(props))
def compile_rows(spec: dict) -> list[dict]:
rows = []
for path, item in spec.get("paths", {}).items():
for method, op in item.items():
if method.startswith("x-") or not isinstance(op, dict):
continue
for status, body in (op.get("responses") or {}).items():
if not isinstance(body, dict):
continue
code = error_code(body)
rows.append(
{
"id": row_id(method, path, status, code),
"method": method.upper(),
"path": path,
"status": status,
"code": code,
"fields": field_names(body),
"meaning": (body.get("description") or "").strip(),
}
)
rows.sort(key=lambda r: (r["path"], r["method"], r["status"]))
return rows
def render(rows: list[dict]) -> str:
lines = [
"# Error catalog (compiled)",
"",
"This table is generated from openapi.json. Recovery text lives in recovery.md.",
"",
"| ID | Method | Path | Status | Code | Fields | Meaning |",
"| --- | --- | --- | --- | --- | --- | --- |",
]
for row in rows:
lines.append(
"| {id} | {method} | {path} | {status} | {code} | {fields} | {meaning} |".format(**row)
)
text = "\n".join(lines) + "\n"
if FORBIDDEN.search(text):
raise SystemExit("generated catalog contains recovery or paging language")
if not rows:
raise SystemExit("openapi produced zero catalog rows")
return text
def main() -> None:
OUT.write_text(render(compile_rows(SPEC)), encoding="utf-8")
print(f"wrote {OUT} ({OUT.stat().st_size} bytes)", file=sys.stderr)
if __name__ == "__main__":
main()
Run the compiler in CI on every OpenAPI change, then prove the generated file matches a fresh compile. Keep both commands local and boring so the gate is reproducible without a chat session.
python3 docs/errors/compile_error_catalog.py
git diff --exit-code docs/errors/catalog.generated.md
The second command fails when an author edits the generated table by hand. Hand edits belong in claims.json or in OpenAPI, never in the compiled Markdown. If a meaning sentence is wrong, change the response description through API review and compile again.
Step 4: Bound meaning drafts when descriptions are missing
Response descriptions should cover most rows, which means a model is optional for this workflow. When a description is empty, a draft job may propose one sentence that restates the status and error code without naming recovery. The budget file caps that job at one sentence per row and forbids the same operational words the compiler already rejects.
# description-budget.toml
max_sentences_per_row = 1
max_chars_per_row = 140
forbid_substrings = [
"page",
"rollback",
"data loss",
"immediately",
"customer",
"support",
"on-call",
]
Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode's free model access can draft those missing meanings, and the free server option can run the compiler and the gate on a throwaway workspace. Neither path should write claims.json. If a draft includes a paging verb or a rollback step, drop the sentence and fill the OpenAPI description by hand instead.
A useful check is a unit test that feeds a malicious draft into the same forbidden-word regex. The test below is a specification for the gate, not a claim about production traffic or model quality.
import re
import unittest
FORBIDDEN = re.compile(
r"\b(page|pager|rollback|data loss|immediately|customer|support|on-call)\b",
re.I,
)
class MeaningBudgetTests(unittest.TestCase):
def test_status_restatement_passes(self):
text = "Another export job is already running for this account."
self.assertIsNone(FORBIDDEN.search(text))
def test_invented_paging_fails(self):
text = "Page on-call immediately and tell the customer to contact support."
self.assertIsNotNone(FORBIDDEN.search(text))
if __name__ == "__main__":
unittest.main()
Step 5: Render recovery notes and fail closed
Render recovery.md from claims.json with a second script that never calls a model. Then assert three invariants before merge: every compiled row has a claims owner, no generated file matches the forbidden regex, and the generated file equals a fresh compile. A fourth check requires page_oncall to be a boolean so a model cannot later smuggle "maybe" into a pager field.
#!/usr/bin/env python3
"""Fail closed when OpenAPI rows and human claims drift."""
from __future__ import annotations
import json
import pathlib
import re
import subprocess
import sys
ROOT = pathlib.Path("docs/errors")
FORBIDDEN = re.compile(
r"\b(page|pager|rollback|data loss|immediately|customer|support|on-call)\b",
re.I,
)
def compiled_ids() -> set[str]:
from compile_error_catalog import compile_rows
spec = json.loads((ROOT / "openapi.json").read_text(encoding="utf-8"))
return {row["id"] for row in compile_rows(spec)}
def main() -> None:
subprocess.check_call([sys.executable, str(ROOT / "compile_error_catalog.py")])
generated = (ROOT / "catalog.generated.md").read_text(encoding="utf-8")
if FORBIDDEN.search(generated):
raise SystemExit("generated catalog contains recovery language")
claims = json.loads((ROOT / "claims.json").read_text(encoding="utf-8"))["errors"]
ids = compiled_ids()
missing = sorted(ids - set(claims))
extra = sorted(set(claims) - ids)
if missing or extra:
raise SystemExit(f"claims drift: missing={missing} extra={extra}")
for key, rec in claims.items():
if not isinstance(rec.get("page_oncall"), bool):
raise SystemExit(f"{key}: page_oncall must be boolean")
for field in ("severity", "customer_copy", "rollback", "owner"):
if not str(rec.get(field, "")).strip():
raise SystemExit(f"{key}: {field} is empty")
print(f"ok: {len(ids)} error rows have signed recovery claims")
if __name__ == "__main__":
main()
Publish both Markdown files together so readers see status codes beside recovery notes, without letting one file overwrite the other. Reviewers can then ask a single question during the pull request: did the protocol change, or did the signed recovery change.
python3 docs/errors/gate_error_catalog.py
python3 -m unittest docs.errors.test_meaning_budget
Limitations
The regex gate is deliberately crude and will block honest descriptions that use "customer" as part of a public error code. Move such codes into an allow-list rather than weakening the global pattern for every row. OpenAPI files that share response objects through $ref need a resolver before compile; this script only reads inline response bodies and will under-count reused components.
The workflow assumes one service and one catalog. Multi-service platforms need a claims owner per operationId, which this compiler does not infer from repository layout. Model drafts still invent verbs under budget pressure, so empty descriptions should become rare after the first month of API review. Success statuses need claims too, or the gate will treat a missing 201 record as drift rather than as an optional row.
Who should not use this approach
Do not use this split for documents that are themselves the incident contract, such as a customer status-page template owned by counsel. Those pages need approved sentences, not compiled tables with a forbid list. Also skip the model-draft path when error codes embed tenant names or internal hostnames, because even a meaning restatement can leak topology.
Teams without OpenAPI at review time gain little, because the compiler would freeze a parallel catalog that drifts from handlers. Extract the status blocks first, then attach claims, rather than asking a model to invent both the protocol and the recovery. If paging policy lives only in a wiki, move it into claims.json before generating anything, or the catalog will look complete while on-call still runs from memory.
Top comments (0)