Compile API Docs From Extractable Facts, Not From a Blank Prompt

Generated documentation fails when a model invents commitments that no schema, test, or flag can prove. A more reliable workflow treats docs as a compile step over extractable facts, then reserves human authorship for promises. The model may rewrite proven fields into prose; a named owner must still write deprecation, support, and caution pages. The rest of this article is a proposed pipeline, labeled as unexecuted example code, that a team can run against one OpenAPI file.

The failure mode is not bad grammar. It is a page that sounds finished while the underlying contract is still a guess. Status codes, required fields, and flag names are facts a parser can extract; “supported,” “safe to retry,” and “will not break callers” are promises a human must own. If those two classes share one prompt, review becomes archaeology instead of certification. The compiler metaphor keeps them apart before any markdown is written.

What counts as a source, and what does not

Treat the docs tree as a graph of nodes. Each generated node must point at a machine-readable source identifier, or the compile step must refuse to emit the file. OpenAPI operations, JSON Schema properties, protobuf fields, CLI flags, and environment variables parsed from code are acceptable sources. Architecture rationale, migration timelines, support hours, and “when not to use this” guidance are not sources, because no parser can prove them from the repository alone.

The table below is a working rule, not a personality test for writers. Apply it per file, not per repository, because mixed pages are where invented SLAs hide.

Node type	Example inputs	Model may draft?	Human must own
Endpoint reference	operationId, path, verb, required fields	Yes, from extract only	Final merge, examples that hit real tenants
Error catalog	Documented status codes and schema names	Yes, listed codes only	Whether a code is user-facing
Flag / env reference	argparse names, os.environ keys	Yes, names and types	Default policy in production
Breaking-change note	Diff of two schema versions	Draft a candidate list	Classify break versus additive
Support / SLA / security	None that a parser can prove	No	Entire page
Deprecation timeline	None unless a dated annotation exists	No, unless the date is in source	Calendar, audience, and fallback

A useful side effect of this split is smaller drafts. The model stops writing tutorials that smuggle policy into a parameter table, and the reviewer stops grepping for adjectives that no test covers.

Inventory first, prose second

Do not start from a “write the docs” prompt. Start from an inventory file that a compiler can fail closed. The JSON below is a proposed schema for that inventory, not a measured production format.

{
  "schema_version": 1,
  "sources": [
    {
      "id": "openapi:users.list",
      "kind": "openapi.operation",
      "path": "docs/sources/openapi.yaml",
      "pointer": "#/paths/~1users/get",
      "owner": null
    },
    {
      "id": "human:deprecation.users.list",
      "kind": "human.promise",
      "path": "docs/human/deprecation-users-list.md",
      "pointer": null,
      "owner": "api-platform"
    }
  ]
}

Every later markdown file must declare source_ids in front matter. Human-owned nodes must also declare owner. Missing either field is a compile error, not a style comment. That rule is the entire gate; tone and length are out of scope for the compiler.

A five-step compile workflow

The steps are sequential on purpose. Skipping extraction and asking a model to “fill gaps” recreates the blank-prompt problem with nicer tooling.

1. Extract facts into a stub directory. Parse OpenAPI, JSON Schema, or --help output into JSON records that contain only names, types, required flags, and enumerated values. Do not extract descriptions if those descriptions were themselves model-written in a previous loop.
2. Classify each record as draftable or human_only. Draftable records have a parser-backed identifier. Human-only records are promises, timelines, or audience advice, even when they sit next to an endpoint.
3. Render draftable records into constrained markdown. The model may reorder sentences and expand field names into short definitions. It may not add status codes, defaults, or guarantees that the stub lacks.
4. Lint the draft for promise tokens. Reject files that introduce words the extract never authorized, such as guarantee, SLA, always, never break, or production-safe.
5. Merge only after a named owner certifies human pages. Reference drafts can merge with schema review. Promise pages cannot merge with an empty owner field.

The command sketch below is an unexecuted local flow. Adjust paths to the repository you actually maintain.

python tools/extract_openapi.py \
  --in docs/sources/openapi.yaml \
  --out .doc-build/facts.json

python tools/classify_facts.py \
  --facts .doc-build/facts.json \
  --inventory docs/inventory.json \
  --out .doc-build/plan.json

python tools/render_drafts.py \
  --plan .doc-build/plan.json \
  --out docs/generated/

python tools/lint_promises.py \
  --drafts docs/generated/ \
  --forbidden-tokens guarantee,SLA,always,never-break,production-safe

pytest tools/test_doc_gate.py -q

Disclosure: This article was prepared as part of MonkeyCode's product outreach. The rendering step is the only place a free model and a free server option belong in this workflow: they format already extracted facts into markdown on a machine that does not hold production secrets. They do not replace the inventory, the promise linter, or the human owner field, and this article does not claim specific model names, quotas, or hardware.

Extraction that refuses to invent fields

The extractor should copy, not complete. If an operation omits 4xx responses, the stub must omit them rather than “helpfully” adding a generic error section. The fragment below is proposed Python, not a harvested production parser.

# tools/extract_openapi.py — proposed, unexecuted example
from __future__ import annotations

import json
import sys
from pathlib import Path

try:
    import yaml
except ImportError as exc:
    raise SystemExit("install pyyaml before running this proposed extractor") from exc


def operations(spec: dict) -> list[dict]:
    rows = []
    for path, item in (spec.get("paths") or {}).items():
        if not isinstance(item, dict):
            continue
        for method, op in item.items():
            if method.startswith("x-") or not isinstance(op, dict):
                continue
            responses = sorted((op.get("responses") or {}).keys())
            params = op.get("parameters") or []
            required = [
                p.get("name") for p in params
                if isinstance(p, dict) and p.get("required") is True
            ]
            rows.append({
                "id": f"openapi:{op.get('operationId') or method + ':' + path}",
                "method": method.upper(),
                "path": path,
                "required_params": required,
                "status_codes": responses,
                "summary": op.get("summary") or "",
            })
    return rows


def main(argv: list[str]) -> int:
    src = Path(argv[argv.index("--in") + 1])
    dst = Path(argv[argv.index("--out") + 1])
    spec = yaml.safe_load(src.read_text())
    dst.write_text(json.dumps({"facts": operations(spec)}, indent=2))
    return 0


if __name__ == "__main__":
    raise SystemExit(main(sys.argv))

Notice the extractor never writes deprecated_on, retry_safe, or audience. Those keys are absent because they are not facts in the OpenAPI document. A later model prompt that is allowed to “complete missing sections” will put them back; the gate exists to stop that completion.

Front matter the gate can parse

Generated pages and human pages should share a tiny schema so CI does not special-case filenames. Keep the fields boring and enumerable.

---
page_kind: generated.reference
source_ids:
  - openapi:users.list
owner: null
review_state: draft
---

---
page_kind: human.promise
source_ids:
  - human:deprecation.users.list
owner: api-platform
review_state: certified
---

The test file below encodes three failures that reviewers usually catch too late. It is a proposed pytest module and should be treated as such until you run it against your tree.

# tools/test_doc_gate.py — proposed, unexecuted example
from pathlib import Path
import re
import yaml

ROOT = Path("docs")
PROMISE_TOKENS = re.compile(
    r"\b(guarantee|sla|always|never break|production-safe)\b",
    re.I,
)
REQUIRED = {"page_kind", "source_ids", "owner", "review_state"}


def pages():
    for path in list(ROOT.rglob("*.md")):
        text = path.read_text()
        if not text.startswith("---"):
            raise AssertionError(f"{path} missing front matter")
        _, fm, body = text.split("---", 2)
        meta = yaml.safe_load(fm) or {}
        yield path, meta, body


def test_front_matter_complete():
    for path, meta, _ in pages():
        missing = REQUIRED - set(meta)
        assert not missing, f"{path} missing {missing}"
        assert meta["source_ids"], f"{path} has empty source_ids"


def test_human_pages_have_owners():
    for path, meta, _ in pages():
        if str(meta.get("page_kind", "")).startswith("human."):
            assert meta.get("owner"), f"{path} is human-owned without owner"


def test_generated_pages_avoid_promise_tokens():
    for path, meta, body in pages():
        if str(meta.get("page_kind", "")).startswith("generated."):
            hit = PROMISE_TOKENS.search(body)
            assert hit is None, f"{path} introduced {hit.group(0)!r}"

Run the tests the same way you run unit tests, not as a docs linter that people can skip. A red gate on source_ids is cheaper than a postmortem on a published “never break” sentence.

pytest tools/test_doc_gate.py -q
rg -n "page_kind: generated" docs | wc -l
rg -n "page_kind: human" docs | wc -l

If the generated count rises while the human count stays at zero, the inventory is lying. Reference surface area grew, but nobody accepted the promises that usually travel with new endpoints.

Constraining the render prompt

The render step should receive stubs, not the whole repository. Passing the git history, the handbook, and last quarter’s incident review invites the model to paper over missing status codes with narrative. A proposed prompt contract looks like the following block, which you should version next to the extractor rather than leaving it in a chat window.

You receive JSON facts. Write markdown for one operation only.
Use only keys present in the JSON. Do not add status codes, defaults,
timelines, or recommendations. Do not use: guarantee, SLA, always,
never break, production-safe. If a field is missing, omit the section.

That contract is intentionally hostile to completeness. Completeness is the human page’s job, and completeness without an owner is how support pages acquire dates nobody can defend. When a stub is empty, the correct compile output is no file, not a plausible tutorial.

Limitations, and who should not use this

This workflow does not make documentation true. It only prevents a class of falsehoods that come from formatting guesses as reference. It will not detect a schema that is itself wrong, a flag that is parsed but unused, or an OpenAPI file that lags the running service by two releases. Those are product defects, and a docs compiler cannot certify the runtime.

Do not use this approach for security advisories, legal terms, pricing, medical or safety copy, or customer-specific runbooks. Do not use it when the API surface is not machine-readable, because the inventory would then be hand-written fiction. Do not use it to auto-publish GA support pages; generated reference can wait in a draft directory until a human certifies the adjacent promise pages. Teams that need narrative design docs should keep those documents entirely in the human class, even if an endpoint table appears in an appendix.

The promise-token linter is also crude. It will flag a quoted error string that contains “always,” and it will miss a polite paragraph that implies an SLA without using the letters. Keep the linter as a tripwire, then require a named owner for anything that tells a customer what the company will do next.

If you already publish docs from a repository, run the extractor on a single OpenAPI file and count how many generated sentences still lack a source identifier. That count is the actual backlog; the model is optional until the inventory is honest.