DEV Community

Avery Lin
Avery Lin

Posted on

Compile an Error Catalog From OpenAPI Status Blocks; Hand-Write Severity and Recovery

Error catalogs go stale when a model invents customer-facing recovery that on-call never rehearsed during incident drills. An OpenAPI document can still compile status codes, error names, and response fields without promising any operational action. Humans must own severity, user-visible copy, pager policy, and rollback, because those sentences change how incidents end. This workflow compiles catalog rows from OpenAPI, bounds optional description drafts, and fails the build when generated files grow recovery advice.

Keep protocol facts out of incident language

Most error pages mix a protocol fact with a human promise inside a single table cell. The fact is that POST /exports can return 409 with code export_in_progress. The promise is that customers should wait, and that on-call should not page overnight for that conflict. A model that writes both cells will sound complete while teaching the wrong operational reflex. Compiling the protocol row from OpenAPI lets the catalog track every added response, while a signed claims file still controls severity and recovery.

Reviewers miss the mix because the status code is correct, so the surrounding sentence looks reviewed as well. The compiler below never emits severity, pager policy, rollback steps, or support escalation from generated output. Those strings can appear only in the human-rendered recovery page that is built from claims.

Decision table for each error row

Field Source Writer Generated catalog
HTTP status OpenAPI responses keys compiler allowed
error code response schema enum or property compiler allowed
response fields schema properties names compiler allowed
endpoint + method OpenAPI paths compiler allowed
one-line meaning response description or bounded draft compiler or model allowed if under budget
severity claims file human refused
customer copy claims file human refused
page on-call claims file human refused
rollback / data-loss note claims file human refused

Treat extra sentences as claims-file edits rather than compiler features. If a reviewer wants a retry window in minutes, that guidance belongs in the signed file even when it is short.

Files the compiler may read

Keep four inputs and two outputs, and refuse any job that writes the claims file. The generated catalog may list methods, paths, status codes, error codes, field names, and bounded meanings. It must not contain paging, rollback, data-loss, or support language unless those words are literal error codes. The recovery page is rendered from claims with a template that does not call a model.

docs/errors/
  openapi.json              # reviewed protocol facts
  claims.json               # human-owned severity and recovery
  description-budget.toml   # max sentences and forbid list
  compile_error_catalog.py
  gate_error_catalog.py
  catalog.generated.md
  recovery.md
Enter fullscreen mode Exit fullscreen mode

Step 1: Freeze the OpenAPI responses you already ship

Start from the OpenAPI file that already gates client generation, not from a chat transcript about possible errors. If a status block is missing from that file, add it through API review before the catalog can mention it. The fragment below is a sample specification for the compiler, not a contract for a named product.

{
  "openapi": "3.1.0",
  "info": {"title": "Export Service", "version": "0.0.0"},
  "paths": {
    "/exports": {
      "post": {
        "operationId": "createExport",
        "responses": {
          "201": {
            "description": "Export job accepted and queued"
          },
          "409": {
            "description": "Another export job is already running",
            "content": {
              "application/json": {
                "schema": {
                  "type": "object",
                  "required": ["code", "retryable"],
                  "properties": {
                    "code": {
                      "type": "string",
                      "enum": ["export_in_progress"]
                    },
                    "retryable": {"type": "boolean"}
                  }
                }
              }
            }
          },
          "500": {
            "description": "Unhandled failure while writing the export file",
            "content": {
              "application/json": {
                "schema": {
                  "type": "object",
                  "required": ["code"],
                  "properties": {
                    "code": {
                      "type": "string",
                      "enum": ["export_write_failed"]
                    }
                  }
                }
              }
            }
          }
        }
      }
    }
  }
}
Enter fullscreen mode Exit fullscreen mode

Commit response blocks through the same review path that already protects handler code. A polished meaning sentence is worthless if the status code can drift without a pull request. Success responses such as 201 can be compiled as rows, but they still need a claims record that says paging is off.

Step 2: Hand-write recovery claims for every compiled row

Do not ask a model to propose severity or customer copy, even as a starting draft that humans later edit. Severity encodes paging cost, and customer copy encodes legal tone, so both need an accountable owner. The JSON below is an operator template; it is not guidance for any production incident.

{
  "errors": {
    "POST /exports 409 export_in_progress": {
      "severity": "warn",
      "page_oncall": false,
      "customer_copy": "Another export is still running. Retry after that job finishes.",
      "rollback": "No rollback. Wait for the in-flight job or cancel it from the operator runbook.",
      "owner": "data-platform-oncall"
    },
    "POST /exports 500 export_write_failed": {
      "severity": "error",
      "page_oncall": true,
      "customer_copy": "The export did not finish. No partial file was published.",
      "rollback": "Delete any incomplete object the writer left, then replay from the last cursor.",
      "owner": "data-platform-oncall"
    },
    "POST /exports 201": {
      "severity": "info",
      "page_oncall": false,
      "customer_copy": "The export job was accepted.",
      "rollback": "Not an error. No rollback.",
      "owner": "data-platform-oncall"
    }
  }
}
Enter fullscreen mode Exit fullscreen mode

Every compiled row must have a claims key, and extra claims keys fail the gate. That rule prevents a new 503 from shipping with a compiled status and no human recovery sentence. Owners should be rotation names, not personal handles, so the file still makes sense after a staffing change.

Step 3: Compile catalog rows only from OpenAPI

The compiler walks paths, emits a Markdown table, and refuses to read claims.json while printing the generated page. Meanings come from each response description first, which already passed API review. A model draft is used only when that field is empty and the budget file still has remaining sentences.

#!/usr/bin/env python3
"""Compile catalog.generated.md from openapi.json only."""
from __future__ import annotations

import json
import pathlib
import re
import sys

ROOT = pathlib.Path("docs/errors")
SPEC = json.loads((ROOT / "openapi.json").read_text(encoding="utf-8"))
OUT = ROOT / "catalog.generated.md"
FORBIDDEN = re.compile(
    r"\b(page|pager|rollback|data loss|immediately|customer|support|on-call)\b",
    re.I,
)


def row_id(method: str, path: str, status: str, code: str) -> str:
    base = f"{method.upper()} {path} {status}"
    return f"{base} {code}" if code else base


def error_code(body: dict) -> str:
    schema = (
        body.get("content", {})
        .get("application/json", {})
        .get("schema", {})
    )
    enum = schema.get("properties", {}).get("code", {}).get("enum") or []
    return enum[0] if len(enum) == 1 else ""


def field_names(body: dict) -> str:
    schema = (
        body.get("content", {})
        .get("application/json", {})
        .get("schema", {})
    )
    props = schema.get("properties", {})
    return ", ".join(sorted(props))


def compile_rows(spec: dict) -> list[dict]:
    rows = []
    for path, item in spec.get("paths", {}).items():
        for method, op in item.items():
            if method.startswith("x-") or not isinstance(op, dict):
                continue
            for status, body in (op.get("responses") or {}).items():
                if not isinstance(body, dict):
                    continue
                code = error_code(body)
                rows.append(
                    {
                        "id": row_id(method, path, status, code),
                        "method": method.upper(),
                        "path": path,
                        "status": status,
                        "code": code,
                        "fields": field_names(body),
                        "meaning": (body.get("description") or "").strip(),
                    }
                )
    rows.sort(key=lambda r: (r["path"], r["method"], r["status"]))
    return rows


def render(rows: list[dict]) -> str:
    lines = [
        "# Error catalog (compiled)",
        "",
        "This table is generated from openapi.json. Recovery text lives in recovery.md.",
        "",
        "| ID | Method | Path | Status | Code | Fields | Meaning |",
        "| --- | --- | --- | --- | --- | --- | --- |",
    ]
    for row in rows:
        lines.append(
            "| {id} | {method} | {path} | {status} | {code} | {fields} | {meaning} |".format(**row)
        )
    text = "\n".join(lines) + "\n"
    if FORBIDDEN.search(text):
        raise SystemExit("generated catalog contains recovery or paging language")
    if not rows:
        raise SystemExit("openapi produced zero catalog rows")
    return text


def main() -> None:
    OUT.write_text(render(compile_rows(SPEC)), encoding="utf-8")
    print(f"wrote {OUT} ({OUT.stat().st_size} bytes)", file=sys.stderr)


if __name__ == "__main__":
    main()
Enter fullscreen mode Exit fullscreen mode

Run the compiler in CI on every OpenAPI change, then prove the generated file matches a fresh compile. Keep both commands local and boring so the gate is reproducible without a chat session.

python3 docs/errors/compile_error_catalog.py
git diff --exit-code docs/errors/catalog.generated.md
Enter fullscreen mode Exit fullscreen mode

The second command fails when an author edits the generated table by hand. Hand edits belong in claims.json or in OpenAPI, never in the compiled Markdown. If a meaning sentence is wrong, change the response description through API review and compile again.

Step 4: Bound meaning drafts when descriptions are missing

Response descriptions should cover most rows, which means a model is optional for this workflow. When a description is empty, a draft job may propose one sentence that restates the status and error code without naming recovery. The budget file caps that job at one sentence per row and forbids the same operational words the compiler already rejects.

# description-budget.toml
max_sentences_per_row = 1
max_chars_per_row = 140
forbid_substrings = [
  "page",
  "rollback",
  "data loss",
  "immediately",
  "customer",
  "support",
  "on-call",
]
Enter fullscreen mode Exit fullscreen mode

Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode's free model access can draft those missing meanings, and the free server option can run the compiler and the gate on a throwaway workspace. Neither path should write claims.json. If a draft includes a paging verb or a rollback step, drop the sentence and fill the OpenAPI description by hand instead.

A useful check is a unit test that feeds a malicious draft into the same forbidden-word regex. The test below is a specification for the gate, not a claim about production traffic or model quality.

import re
import unittest

FORBIDDEN = re.compile(
    r"\b(page|pager|rollback|data loss|immediately|customer|support|on-call)\b",
    re.I,
)

class MeaningBudgetTests(unittest.TestCase):
    def test_status_restatement_passes(self):
        text = "Another export job is already running for this account."
        self.assertIsNone(FORBIDDEN.search(text))

    def test_invented_paging_fails(self):
        text = "Page on-call immediately and tell the customer to contact support."
        self.assertIsNotNone(FORBIDDEN.search(text))

if __name__ == "__main__":
    unittest.main()
Enter fullscreen mode Exit fullscreen mode

Step 5: Render recovery notes and fail closed

Render recovery.md from claims.json with a second script that never calls a model. Then assert three invariants before merge: every compiled row has a claims owner, no generated file matches the forbidden regex, and the generated file equals a fresh compile. A fourth check requires page_oncall to be a boolean so a model cannot later smuggle "maybe" into a pager field.

#!/usr/bin/env python3
"""Fail closed when OpenAPI rows and human claims drift."""
from __future__ import annotations

import json
import pathlib
import re
import subprocess
import sys

ROOT = pathlib.Path("docs/errors")
FORBIDDEN = re.compile(
    r"\b(page|pager|rollback|data loss|immediately|customer|support|on-call)\b",
    re.I,
)

def compiled_ids() -> set[str]:
    from compile_error_catalog import compile_rows

    spec = json.loads((ROOT / "openapi.json").read_text(encoding="utf-8"))
    return {row["id"] for row in compile_rows(spec)}


def main() -> None:
    subprocess.check_call([sys.executable, str(ROOT / "compile_error_catalog.py")])
    generated = (ROOT / "catalog.generated.md").read_text(encoding="utf-8")
    if FORBIDDEN.search(generated):
        raise SystemExit("generated catalog contains recovery language")
    claims = json.loads((ROOT / "claims.json").read_text(encoding="utf-8"))["errors"]
    ids = compiled_ids()
    missing = sorted(ids - set(claims))
    extra = sorted(set(claims) - ids)
    if missing or extra:
        raise SystemExit(f"claims drift: missing={missing} extra={extra}")
    for key, rec in claims.items():
        if not isinstance(rec.get("page_oncall"), bool):
            raise SystemExit(f"{key}: page_oncall must be boolean")
        for field in ("severity", "customer_copy", "rollback", "owner"):
            if not str(rec.get(field, "")).strip():
                raise SystemExit(f"{key}: {field} is empty")
    print(f"ok: {len(ids)} error rows have signed recovery claims")


if __name__ == "__main__":
    main()
Enter fullscreen mode Exit fullscreen mode

Publish both Markdown files together so readers see status codes beside recovery notes, without letting one file overwrite the other. Reviewers can then ask a single question during the pull request: did the protocol change, or did the signed recovery change.

python3 docs/errors/gate_error_catalog.py
python3 -m unittest docs.errors.test_meaning_budget
Enter fullscreen mode Exit fullscreen mode

Limitations

The regex gate is deliberately crude and will block honest descriptions that use "customer" as part of a public error code. Move such codes into an allow-list rather than weakening the global pattern for every row. OpenAPI files that share response objects through $ref need a resolver before compile; this script only reads inline response bodies and will under-count reused components.

The workflow assumes one service and one catalog. Multi-service platforms need a claims owner per operationId, which this compiler does not infer from repository layout. Model drafts still invent verbs under budget pressure, so empty descriptions should become rare after the first month of API review. Success statuses need claims too, or the gate will treat a missing 201 record as drift rather than as an optional row.

Who should not use this approach

Do not use this split for documents that are themselves the incident contract, such as a customer status-page template owned by counsel. Those pages need approved sentences, not compiled tables with a forbid list. Also skip the model-draft path when error codes embed tenant names or internal hostnames, because even a meaning restatement can leak topology.

Teams without OpenAPI at review time gain little, because the compiler would freeze a parallel catalog that drifts from handlers. Extract the status blocks first, then attach claims, rather than asking a model to invent both the protocol and the recovery. If paging policy lives only in a wiki, move it into claims.json before generating anything, or the catalog will look complete while on-call still runs from memory.

Top comments (0)