DEV Community

Avery Lin
Avery Lin

Posted on

Generate Field Tables From OpenAPI; Bind Every Error to a Human Runbook

Generated API documentation remains honest when models only restate schema-backed field tables, and humans own every recovery path. Failure advice encodes product judgment, retry policy, and support commitments that no OpenAPI file can actually prove. A small CI contract can keep those two document classes in separate files, then reject mixed drafts automatically. The rest of this article specifies that split, a generator, and a linter you can run before merge.

Why recovery prose fails differently from field tables

Reference generators often emit a status-code table, then invent a paragraph that tells clients when to retry. That second paragraph is not a restatement of the schema; it is an operational promise with legal and on-call consequences. Reviewers miss the invention because the surrounding table looks correct, and the prose reads like typical vendor documentation. Treating recovery language as a separate artifact, with a required binding from each error schema, makes the invention detectable in CI.

Field tables fail in a bounded way when a property is renamed or a type is widened in the spec. Recovery paragraphs fail in an unbounded way because they introduce duties the spec never stated, such as backoff schedules or support contacts. Mixing those classes in one Markdown file forces reviewers to grade two different risk models in a single diff. Splitting them lets a model draft the bounded class while CODEOWNERS and a verb linter protect the unbounded class.

Decision table: what may be drafted

Claim class Example tokens Allowed drafter Required evidence
Structural field name, type, required, enum Model OpenAPI properties
Wire error shape code, message, request_id Model Error schema in the spec
Status listing 409, 429, 503 Model responses entries
Recovery action retry, backoff, contact support Human Runbook file with owners
Temporal promise duration, remaining quota, SLA Human Runbook plus operator sign-off
Blame assignment user error versus platform incident Human Runbook classification field

A model may draft field names, types, required flags, and enum members when those tokens appear verbatim in the OpenAPI document. A model may also draft a compact Markdown table that mirrors properties under a named schema, including nested objects one level deep. A human must own retry intervals, idempotency expectations, support contacts, quota numbers, and any sentence that tells a client what to do after a failure. A human must also own the decision that a given error code is user-fixable, operator-fixable, or a platform incident.

Directory contract the pipeline can enforce

Keep generated tables and human runbooks in sibling trees so Git history, CODEOWNERS, and linters can treat them as different write surfaces.

docs/
  generated/tables/
    PaymentIntent.md
    ErrorBody.md
  runbooks/errors/
    409-payment-conflict.md
    429-rate-limited.md
openapi.yaml
tools/
  generate_tables.py
  lint_docs_ownership.py
tests/
  test_docs_ownership.py
.github/CODEOWNERS
Enter fullscreen mode Exit fullscreen mode

Four rules make the tree enforceable without a documentation committee meeting on every pull request. First, docs/generated/ is machine-writable and must not contain recovery verbs or second-person troubleshooting. Second, docs/runbooks/ is human-writable and is the only tree where recovery verbs are legal. Third, every 4xx and 5xx response that declares an application/json body must include an x-runbook string. Fourth, CODEOWNERS must require a human team on docs/runbooks/ so a model-authored patch cannot land there unattended.

Step 1 — Annotate error responses with runbook paths

Add a vendor extension on each error response, not on the path item, so two operations that share an error shape can still point at different operator procedures. The path is relative to the repository root and must exist as a file before the linter exits zero. Leave numeric quotas, wait windows, and support routes inside the runbook, never inside description strings the generator might copy.

# openapi.yaml (fragment)
openapi: 3.1.0
info:
  title: Payments
  version: 0.0.0
paths:
  /payment_intents:
    post:
      responses:
        "409":
          description: Conflict against an idempotency key.
          x-runbook: docs/runbooks/errors/409-payment-conflict.md
          content:
            application/json:
              schema:
                $ref: "#/components/schemas/ErrorBody"
        "429":
          description: Rate limited.
          x-runbook: docs/runbooks/errors/429-rate-limited.md
          content:
            application/json:
              schema:
                $ref: "#/components/schemas/ErrorBody"
components:
  schemas:
    ErrorBody:
      type: object
      required: [code, message, request_id]
      properties:
        code:
          type: string
          enum: [idempotency_conflict, rate_limited]
        message:
          type: string
        request_id:
          type: string
    PaymentIntent:
      type: object
      required: [id, amount, currency]
      properties:
        id:
          type: string
        amount:
          type: integer
          minimum: 1
        currency:
          type: string
          enum: [usd, eur]
Enter fullscreen mode Exit fullscreen mode

The description fields above restate the HTTP meaning only. They do not tell the client to retry, wait, or open a ticket, which keeps the generator from copying procedural advice into docs/generated/.

Step 2 — Generate Markdown tables and nothing else

The generator walks components.schemas, emits a heading plus a property table, and refuses to write narrative paragraphs. Nested objects are flattened one level with dot notation so reviewers can still diff a single table. Unknown types are rendered as unsupported rather than guessed, which keeps the model from inventing a wire format.

# tools/generate_tables.py
from __future__ import annotations

import pathlib
import sys

try:
    import yaml
except ImportError:
    print("pip install pyyaml", file=sys.stderr)
    raise

ROOT = pathlib.Path(__file__).resolve().parents[1]
SPEC = ROOT / "openapi.yaml"
OUT = ROOT / "docs" / "generated" / "tables"


def type_label(node: dict) -> str:
    if "$ref" in node:
        return node["$ref"].rsplit("/", 1)[-1]
    if "type" in node:
        base = node["type"]
        if "enum" in node:
            return f"{base} enum {node['enum']}"
        return str(base)
    return "unsupported"


def rows_for(schema: dict) -> list[tuple[str, str, str]]:
    required = set(schema.get("required") or [])
    props = schema.get("properties") or {}
    rows = []
    for name, node in props.items():
        rows.append((name, type_label(node), "yes" if name in required else "no"))
        if node.get("type") == "object" and "properties" in node:
            nested_required = set(node.get("required") or [])
            for child, child_node in node["properties"].items():
                flag = "yes" if child in nested_required else "no"
                rows.append((f"{name}.{child}", type_label(child_node), flag))
    return rows


def render(name: str, schema: dict) -> str:
    lines = [
        f"# {name}",
        "",
        "| Field | Type | Required |",
        "| --- | --- | --- |",
    ]
    for field, label, req in rows_for(schema):
        lines.append(f"| `{field}` | `{label}` | {req} |")
    lines.append("")
    lines.append("_Generated from openapi.yaml. Recovery steps live in docs/runbooks._")
    lines.append("")
    return "\n".join(lines)


def main() -> int:
    spec = yaml.safe_load(SPEC.read_text(encoding="utf-8"))
    schemas = (spec.get("components") or {}).get("schemas") or {}
    OUT.mkdir(parents=True, exist_ok=True)
    for name, schema in schemas.items():
        path = OUT / f"{name}.md"
        path.write_text(render(name, schema), encoding="utf-8")
        print(f"wrote {path.relative_to(ROOT)}")
    return 0


if __name__ == "__main__":
    raise SystemExit(main())
Enter fullscreen mode Exit fullscreen mode

Run it from the repository root after the spec changes, then commit the tables as build output rather than as freehand edits.

python tools/generate_tables.py
Enter fullscreen mode Exit fullscreen mode

Step 3 — Lint generated files for recovery language

The linter is the actual gate. It loads the spec, requires x-runbook on error responses, checks that each runbook file exists, and scans docs/generated for a fixed verb list. The verb list is intentionally dull and mechanical so authors cannot hide advice behind friendlier synonyms without updating tests.

# tools/lint_docs_ownership.py
from __future__ import annotations

import pathlib
import re
import sys

try:
    import yaml
except ImportError:
    print("pip install pyyaml", file=sys.stderr)
    raise

ROOT = pathlib.Path(__file__).resolve().parents[1]
SPEC = ROOT / "openapi.yaml"
GENERATED = ROOT / "docs" / "generated"
ERROR_STATUSES = re.compile(r"^[45]\d\d$")
RECOVERY = re.compile(
    r"\b(retry|retries|backoff|back off|contact support|sla|"
    r"guaranteed|escalate|on-call|wait\\s+\\d+|exponential)\b",
    re.I,
)


def iter_operations(spec: dict):
    for path, item in (spec.get("paths") or {}).items():
        if not isinstance(item, dict):
            continue
        for method, op in item.items():
            if isinstance(op, dict) and "responses" in op:
                yield path, method, op


def lint_bindings(spec: dict) -> list[str]:
    errors = []
    for path, method, op in iter_operations(spec):
        for status, resp in (op.get("responses") or {}).items():
            if not ERROR_STATUSES.match(str(status)):
                continue
            if not isinstance(resp, dict):
                continue
            runbook = resp.get("x-runbook")
            if not runbook:
                errors.append(f"{method.upper()} {path} {status} missing x-runbook")
                continue
            target = ROOT / runbook
            if not target.is_file():
                errors.append(f"{method.upper()} {path} {status} missing file {runbook}")
    return errors


def lint_generated() -> list[str]:
    errors = []
    if not GENERATED.exists():
        return ["docs/generated is missing"]
    for path in GENERATED.rglob("*.md"):
        text = path.read_text(encoding="utf-8")
        if RECOVERY.search(text):
            errors.append(f"recovery language in {path.relative_to(ROOT)}")
        if re.search(r"(?m)^you should\b", text, re.I):
            errors.append(f"imperative advice in {path.relative_to(ROOT)}")
    return errors


def main() -> int:
    spec = yaml.safe_load(SPEC.read_text(encoding="utf-8"))
    errors = lint_bindings(spec) + lint_generated()
    if errors:
        print("ownership lint failed:")
        for item in errors:
            print(f"  - {item}")
        return 1
    print("ownership lint passed")
    return 0


if __name__ == "__main__":
    raise SystemExit(main())
Enter fullscreen mode Exit fullscreen mode
python tools/lint_docs_ownership.py
Enter fullscreen mode Exit fullscreen mode

Step 4 — Lock the gate with a planted-failure test

A linter without a test that plants a forbidden sentence will rot as soon as somebody relaxes the regex. The test writes a temporary generated file, expects a non-zero process, and also asserts that a missing x-runbook fails. Keep the planted sentence ugly and explicit so future editors cannot claim it was accidental documentation tone.

# tests/test_docs_ownership.py
from __future__ import annotations

import pathlib
import subprocess
import sys

ROOT = pathlib.Path(__file__).resolve().parents[1]
LINT = ROOT / "tools" / "lint_docs_ownership.py"
GENERATED = ROOT / "docs" / "generated" / "tables"


def run_lint() -> subprocess.CompletedProcess[str]:
    return subprocess.run(
        [sys.executable, str(LINT)],
        cwd=ROOT,
        capture_output=True,
        text=True,
    )


def test_clean_tree_passes():
    result = run_lint()
    assert result.returncode == 0, result.stdout + result.stderr


def test_planted_retry_sentence_fails(tmp_path, monkeypatch):
    planted = GENERATED / "_planted.md"
    planted.write_text("Clients should retry with exponential backoff.\n", encoding="utf-8")
    try:
        result = run_lint()
        assert result.returncode == 1
        assert "recovery language" in result.stdout
    finally:
        planted.unlink(missing_ok=True)
Enter fullscreen mode Exit fullscreen mode
pip install pyyaml pytest
python tools/generate_tables.py
pytest tests/test_docs_ownership.py -q
Enter fullscreen mode Exit fullscreen mode

Step 5 — Keep runbooks off the model write path

CODEOWNERS is the second lock after the verb linter. Generated tables can be owned by a docs-pipeline bot; runbooks cannot. A model session that is asked to "complete the 429 page" should receive only the schema table as editable context, plus a stub runbook path it is forbidden to fill.

# .github/CODEOWNERS
/docs/runbooks/    @payments-oncall
/openapi.yaml      @api-stewards
/tools/            @api-stewards
Enter fullscreen mode Exit fullscreen mode

A human runbook can be short and still be more trustworthy than generated advice. Own classification, client action, and operator action as explicit fields rather than as a narrative that hides a quota inside a metaphor.

# 429 rate limited

- owner: payments-oncall
- classification: user-fixable with later retry
- client_action: honor Retry-After if present; otherwise wait 60s once
- operator_action: check edge gateway saturation before paging compute
- quota_source: edge config, not this repository
Enter fullscreen mode Exit fullscreen mode

Those bullets are product commitments. They belong in review with the on-call rotation, not in a draft that was asked to sound helpful beside a field table.

Where a hosted draft runner fits

Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode's free model access can draft the schema tables from openapi.yaml after the generator prompt is constrained to Markdown tables only. The free server option can run generate_tables.py and lint_docs_ownership.py as an unattended check so local laptops are not the only place the gate exists. Do not send docs/runbooks/ into that draft loop; the useful product behavior here is cheap iteration on the restatable tables, not autonomous ownership of failure advice.

Prompt the table draft with the spec fragment and the render format, then discard any output that is not a table. If the model adds a closing paragraph about retries, the linter is doing the job the prompt failed to do. That failure is expected and cheap; merging the paragraph is the expensive mistake.

Limitations

This workflow does not repair a wrong OpenAPI document. If amount is documented as integer while the service returns a decimal string, the generated table will faithfully restate the lie. The verb list is not a legal parser of English, so a determined author can smuggle advice through novel phrasing until the tests are updated. Nested allOf and oneOf compositions are flattened poorly, which means polymorphic error envelopes still need a human-owned diagram or a dedicated schema per variant. Webhook delivery guarantees, streaming status events, and billing credits are outside the table generator and should start life in runbooks even when no HTTP error status exists.

The x-runbook extension is not a substitute for OpenAPI callbacks, SLA contracts, or status-page policy. It is a repository-local pointer that CI can resolve. External URL runbooks break the file-existence check unless you add an allowlist and a fetch step, which this article does not specify because link availability is a different failure class.

Who should not use this split

Skip the split if the product has no machine-readable schema and the reference is already a human book. Skip it if the only errors are untyped 500 pages with no client-visible body, because there is no table worth generating. Skip it for marketing comparison pages, architecture decision records, and security threat models, which are judgment documents from the first sentence. Teams that let any contributor merge to main without CODEOWNERS will not get the second lock, and the verb linter alone will not stop a determined paste into the runbook tree.

Start with one error schema, one generated table, and one runbook file. Expand only after the planted-failure test fails on a retry sentence and passes on a clean table render.

Top comments (0)