Keep Generated API Reference Prose Inside an Extracted Facts Envelope

Keep generated API reference prose inside an extracted facts envelope, and refuse any token the envelope does not list. A deterministic extractor should read the OpenAPI document first, before a model is allowed to draft a slot. Humans still own every commitment that cannot be recovered from paths, fields, and declared status codes. The workflow below implements that split with a slot map, a fill step, and a CI scanner.

The failure mode this envelope is built to catch

Reference pages go stale when a model finishes a sentence with a status code the service never returns. The same leak appears when required field names, enum members, or path prefixes come from model memory. Reviewers often miss those tokens because the surrounding prose still matches the tone of the rest of the manual. An extracted facts file makes the allowed vocabulary explicit, so CI can fail a draft that introduces unknown identifiers.

Invented tokens are not a style problem; they are a contract problem that support teams inherit later. A path that does not exist will be copied into client snippets and internal runbooks without a second schema check. A status code that exists only in prose will be asserted in tests that never hit the server. The envelope does not prove runtime behavior, but it does stop documentation from growing a second, unofficial API.

What a model may draft, and what a human must own

Treat each documentation slot as either recoverable from machine-readable sources or as a product commitment. Recoverable slots include parameter tables, enum lists, response field names, and request skeletons without live values. Commitment slots include deprecation dates, rate limits, support windows, migration promises, and advice about when to use an endpoint. The table below is a proposed ownership map for an HTTP reference set, not a measured production study.

Slot	Source of truth	Owner after extract	Model may draft?
Parameter table	OpenAPI path item	extractor, then restatement	Yes, from facts only
Status code list	Declared responses	extractor, then restatement	Yes, from facts only
Enum members	Schema enum arrays	extractor, then restatement	Yes, from facts only
Example field set	Required properties	extractor; freeze live values elsewhere	Yes, without new keys
Auth product policy	Security schemes are incomplete	human fragment	No
Rate limit numbers	Usually absent from OpenAPI	human fragment	No
Sunset or deprecation	Calendar is a promise	human fragment	No
Use-X-versus-Y guidance	Product judgment	human fragment	No
Error recovery advice	Runbooks, not schemas	human fragment	No

If a slot needs a date, a numeric limit, or a guarantee, it is a commitment and stays human-owned. Models may restate identifiers that already appear in the facts file, including paths, methods, field names, and declared codes. They may not introduce a sibling path, an extra required field, or a status code that the extractor did not see. The renderer then concatenates human fragments and scanned model slots into one markdown page.

Step 1 — Define a facts schema the scanner can consume

Keep the intermediate record boring, typed, and small enough to diff inside ordinary pull requests. Each endpoint record should list path, method, required fields, optional fields, enum values, and declared status codes. Later prose may use an identifier only when that identifier already appears in this record. The JSON below is an example facts envelope for a tiny export API, labeled as an unexecuted illustration.

{
  "api_id": "exports-v1",
  "source": "openapi.json",
  "endpoints": [
    {
      "operation_id": "createExport",
      "method": "POST",
      "path": "/v1/exports",
      "required_fields": ["dataset", "format"],
      "optional_fields": ["notify_email"],
      "enums": {
        "format": ["csv", "parquet"]
      },
      "status_codes": [202, 400, 401, 409]
    }
  ],
  "allow_tokens": [
    "createExport",
    "POST",
    "/v1/exports",
    "dataset",
    "format",
    "notify_email",
    "csv",
    "parquet",
    "202",
    "400",
    "401",
    "409"
  ]
}

Commit facts.json only as a generated artifact of the extractor, never as a file that authors edit by hand. Hand-edited facts files become a third source of truth and then drift from the spec in the same way prose already drifts. Review the extractor output when the spec changes, and reject a facts diff that adds tokens the OpenAPI file does not contain. That review is still cheaper than reviewing unconstrained model prose for stray identifiers.

Step 2 — Extract facts from OpenAPI without calling a model

Do not ask a model to summarize the spec before the facts file exists, because summaries reintroduce invented tokens. Parse the spec with ordinary JSON code so path strings, methods, and schema keys stay mechanically consistent. Run the extractor in CI on every spec change so documentation vocabulary cannot drift from the merged contract. The script below walks paths and writes facts.json; it is a simplified example, not a full OpenAPI implementation.

# Example (unexecuted): extract_facts.py
from __future__ import annotations

import json
import sys
from pathlib import Path

HTTP_METHODS = {"get", "post", "put", "patch", "delete", "head", "options"}


def field_names(schema: dict, acc: set[str]) -> None:
    if not isinstance(schema, dict):
        return
    props = schema.get("properties") or {}
    for name, child in props.items():
        acc.add(str(name))
        field_names(child, acc)
    if "items" in schema:
        field_names(schema["items"], acc)


def extract(spec: dict) -> dict:
    endpoints = []
    tokens: set[str] = set()
    for path, item in (spec.get("paths") or {}).items():
        tokens.add(path)
        for method, op in item.items():
            if method.lower() not in HTTP_METHODS or not isinstance(op, dict):
                continue
            required: list[str] = []
            optional: list[str] = []
            enums: dict[str, list[str]] = {}
            fields: set[str] = set()
            body = ((op.get("requestBody") or {}).get("content") or {})
            for media in body.values():
                schema = media.get("schema") or {}
                field_names(schema, fields)
                required.extend(schema.get("required") or [])
                for name, child in (schema.get("properties") or {}).items():
                    if isinstance(child, dict) and "enum" in child:
                        enums[name] = [str(v) for v in child["enum"]]
            required = [f for f in required if f in fields]
            optional = sorted(fields.difference(required))
            codes = sorted(int(c) for c in (op.get("responses") or {}) if str(c).isdigit())
            op_id = str(op.get("operationId") or f"{method}_{path}")
            record = {
                "operation_id": op_id,
                "method": method.upper(),
                "path": path,
                "required_fields": sorted(set(required)),
                "optional_fields": optional,
                "enums": enums,
                "status_codes": codes,
            }
            endpoints.append(record)
            tokens.update(
                [op_id, method.upper(), path, *record["required_fields"], *optional]
            )
            for values in enums.values():
                tokens.update(values)
            tokens.update(str(code) for code in codes)
    return {
        "api_id": spec.get("info", {}).get("title", "api"),
        "source": "openapi.json",
        "endpoints": endpoints,
        "allow_tokens": sorted(tokens),
    }


def main(argv: list[str]) -> int:
    spec_path = Path(argv[1] if len(argv) > 1 else "openapi.json")
    out_path = Path(argv[2] if len(argv) > 2 else "facts.json")
    spec = json.loads(spec_path.read_text())
    facts = extract(spec)
    out_path.write_text(json.dumps(facts, indent=2) + "\n")
    print(f"wrote {out_path} endpoints={len(facts['endpoints'])}")
    return 0


if __name__ == "__main__":
    raise SystemExit(main(sys.argv))

python extract_facts.py openapi.json facts.json

Count endpoints in the extractor output and compare that count with the number of path-method pairs in the spec. A mismatch means the walker skipped a method, a $ref body, or a non-JSON spec feature that this example does not resolve. Expand the extractor before you widen model permissions, because missing facts look identical to forbidden commitments later. The fill step should not start when the endpoint count is zero or when allow_tokens is empty.

Step 3 — Attach ownership to every documentation slot

After facts exist, list the markdown files you intend to generate, slot by slot, in a committed map. A slot file is cheaper to review than a free-form prompt, because ownership is visible before any prose appears. Human-owned slots should be finished markdown fragments that the renderer copies verbatim into the page. Model-fill slots should point at one facts record and declare a maximum sentence budget.

# Example (unexecuted): slots.yaml
page: reference/exports.md
slots:
  - id: createExport.params
    owner: model_fill
    operation_id: createExport
    max_sentences: 4
    allowed_kinds: [fields, enums, status_codes]
  - id: createExport.statuses
    owner: model_fill
    operation_id: createExport
    max_sentences: 3
    allowed_kinds: [status_codes]
  - id: createExport.rate_limits
    owner: human
    source: fragments/exports-rate-limits.md
  - id: createExport.when_to_use
    owner: human
    source: fragments/exports-when-to-use.md

Reject a slots map that names an operation_id absent from facts.json, because that is how unofficial endpoints enter the manual. Reject a model_fill slot that lists allowed_kinds the facts record cannot satisfy, such as status codes when the spec omitted responses. Keep human fragments in git history so product promises are reviewable without reading model output. The fill job should receive only the facts slice for that operation_id, not the rest of the API.

Step 4 — Fill model slots, then reject stray identifiers

Prompt the model with the facts slice, the slot id, and a hard rule against tokens outside allow_tokens. After the model returns, compare identifiers in the draft against that allowlist, plus a short list of ordinary English words. Reject the build when a path, status code, field name, or enum member outside the allowlist appears in model output. Skip files marked owner: human, because those fragments are allowed to contain dates, limits, and other commitments.

# Example (unexecuted): scan_docs.py
from __future__ import annotations

import json
import re
from pathlib import Path

PATH_RE = re.compile(r"/[A-Za-z0-9._~:/?#\[\]@!$&'()*+,;=%-]+")
STATUS_RE = re.compile(r"\b[1-5]\d{2}\b")
IDENT_RE = re.compile(r"\b[A-Za-z_][A-Za-z0-9_]{2,}\b")
ENGLISH = {
    "the", "and", "for", "with", "from", "this", "that", "when",
    "request", "response", "field", "fields", "required", "optional",
    "returns", "status", "code", "codes", "export", "exports",
    "body", "header", "error", "errors", "must", "may",
}


def stray_tokens(text: str, allow: set[str]) -> list[str]:
    found: list[str] = []
    for match in PATH_RE.findall(text):
        if match not in allow:
            found.append(match)
    for match in STATUS_RE.findall(text):
        if match not in allow:
            found.append(match)
    for match in IDENT_RE.findall(text):
        if match.lower() in ENGLISH or match in allow:
            continue
        if match[0].isupper() and match.lower() not in {t.lower() for t in allow}:
            # Headings and ordinary capitalized English are not API tokens.
            continue
        if match not in allow:
            found.append(match)
    return sorted(set(found))


def scan_model_slot(draft_path: Path, facts: dict) -> list[str]:
    allow = set(facts.get("allow_tokens") or [])
    text = draft_path.read_text()
    return stray_tokens(text, allow)

# Example (unexecuted): test_scan_docs.py
from scan_docs import stray_tokens

ALLOW = {
    "createExport",
    "POST",
    "/v1/exports",
    "dataset",
    "format",
    "csv",
    "parquet",
    "202",
    "400",
}


def test_accepts_restated_facts():
    prose = "POST /v1/exports requires dataset and format csv or parquet and returns 202."
    assert stray_tokens(prose, ALLOW) == []


def test_rejects_invented_status_and_path():
    prose = "GET /v1/exports/download returns 200 when the file is ready."
    stray = stray_tokens(prose, ALLOW)
    assert "/v1/exports/download" in stray
    assert "200" in stray

python -m pytest test_scan_docs.py -q

Keep the scanner boring: it should fail closed on unknown paths and status codes, even when the sentence reads fluently. False positives will appear around version numbers in headings, so put version prose in human-owned fragments or add those exact strings to facts during extract. False negatives remain possible for paraphrased field names, which is why the slot prompt should require backticks around every identifier. A scan pass is a vocabulary gate, not a substitute for contract tests against a running server.

Step 5 — Keep the fill job off the control plane

The extractor and scanner are local Python, and they remain the source of pass or fail. The model is only a restatement engine for slots that already have facts and an explicit fill permit. Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode's free model access and free server option can host that fill step while CI still runs the scanner on the facts file.

If a facts envelope already gates the docs pipeline, the fill job can stay on a free server option without relaxing ownership rules. Do not record model names, hardware, quotas, or duration in the documentation repository, because those details are not facts about the API. Durable inputs are openapi.json, slots.yaml, human fragments, and the extractor output. Durable outputs are scanned slot files and the rendered page, plus the scanner's token diff.

Run the pipeline in this order so a failed extract never reaches a model:

1. Validate that openapi.json parses and contains at least one path item.
2. Generate facts.json and fail if the endpoint count does not match the spec walk.
3. Load slots.yaml and fail if any operation_id is missing from facts.
4. Copy human fragments into the work tree without sending them to a model.
5. Fill only model_fill slots, then run scan_docs.py before render.
6. Assemble the page only when the scanner reports an empty stray-token list.

Limitations

This workflow does not prove that the running service behaves as the OpenAPI file claims. Incomplete specs produce incomplete facts, and the model will then under-describe polymorphic bodies or vendor extensions. The scanner will false-flag ordinary numerals in headings unless those strings are extracted or moved to human fragments. Examples copied from OpenAPI can already be wrong, so the extractor should omit example values unless another freeze file owns them. Teams still need contract tests if they want runtime evidence rather than documentation consistency alone.

Unresolved $ref graphs, callbacks, and webhooks will not appear in this simplified walker, and the envelope will then forbid legitimate tokens. Over-broad English allowlists will hide invented field names that look like common words, especially short keys such as id or type. The sentence budget is a review aid, not a quality metric, and it should not be treated as a measured readability score. None of these checks replace legal review for terms that bind customers.

Who should not adopt this approach

Skip this pipeline if you do not have a machine-readable API description that reviewers already trust. Skip it for narrative tutorials, architecture decision records, and legal pages, because those genres are almost entirely human-owned. Skip it if the document is a changelog of product promises, where nearly every sentence is a commitment rather than a restatement. A facts envelope helps reference docs; it does not replace editorial review for anything that binds the company.

The practical rule is simple once the files exist: models restate facts, and humans write commitments. If a sentence needs a date, a limit, or a guarantee, it does not belong in a model-fill slot. Put that sentence in a human-owned fragment, and then let the renderer assemble the finished page. The scanner only has to answer one question after that assembly: did the model speak a token the spec never gave it?