Generated API documentation fails most often when restatement and commitment occupy the same paragraph without a gate. A model can safely restate a field type that an OpenAPI schema already encodes as a string. It cannot safely invent a deprecation window, a compatibility promise, or a production rate-limit guarantee. This article presents a role-gated workflow that classifies every intended sentence before any model drafts it.
Why mixed speech acts break generated docs
Most doc pipelines treat a section as one blob of prose that a model should complete from a prompt. That blob usually mixes four different speech acts that do not share the same evidence bar. Restatements are recoverable from schema, tests, or error enumerations that already exist in the repository. Commitments describe future behavior that no file currently proves, including support windows and public compatibility.
When those acts share a paragraph, reviewers cannot tell which sentences remain mechanically true. A later schema change updates the type, while the neighboring promise about "no breaking changes this year" stays untouched. Readers treat both sentences as equally official, which is exactly how silent contract drift enters a public reference.
Four roles, one permit file
Assign every planned sentence exactly one role before generation starts, and refuse mixed roles inside one sentence. Keep the taxonomy small enough that a script can enforce it in continuous integration without a trained classifier. Larger taxonomies look precise in design reviews and then collapse during merge, because authors cannot agree on the boundary between guidance and a promise.
-
RESTATE— a sentence rebuilt from extractable facts such as path, method, type, required, enum, or status code. -
EXAMPLE— a request or response snippet that must match a checked-in fixture file, not a model-invented payload. -
ADVICE— operational guidance such as retry shape or timeout ranges that is not encoded in the schema. -
COMMIT— a promise about compatibility, deprecation timing, availability, or behavior the product will still honor.
Only RESTATE and fixture-bound EXAMPLE may be drafted by a model. ADVICE and COMMIT remain stub headings that a named human must write and sign.
A minimal permit the compiler can read
Store the permit next to the OpenAPI file so the gate is versioned with the contract. The following YAML is a worked example for a tiny pets endpoint, not a production policy dump. Copy it as structure, then replace owners, paths, and verbs with the contract you actually ship.
# docs/permits/pets.yaml
source: openapi.yaml
human_owner: api-docs
allowed_model_roles:
- RESTATE
- EXAMPLE
forbidden_commitment_verbs:
- guarantee
- always
- never
- forever
- promise
- compatible
- deprecate
- sla
sections:
- id: pets-list
path: /pets
method: get
roles:
- RESTATE
- EXAMPLE
- id: pets-list-stability
path: /pets
method: get
roles:
- COMMIT
stub: "Human-owned: pagination stability and deprecation window."
The permit does not try to score writing quality, tone, or completeness of the generated section. It only answers a narrower question: may a model emit a sentence in this section at all. If the answer is no, the file must stay a stub until a reviewer adds a Signed-off-by trailer.
Workflow: six numbered gates
Run the six gates in listed order and skip none of them, because later gates assume earlier extracts exist on disk. A skipped extract step is how unverifiable prose re-enters the branch under a green paraphrase job.
- Freeze the contract input by hashing
openapi.yaml, and refuse generation when the working tree hash mismatches the permit digest. - Extract facts by pulling path, method, parameters, types, required flags, enums, and status codes into
facts.jsonwith sorted keys. - Expand
RESTATEtemplates by rendering sentences from facts with string templates, and do not ask a model to invent structure here. - Optionally rewrite
RESTATEprose so a model may paraphrase template sentences only when every entity still appears infacts.json. - Attach
EXAMPLEblocks from fixtures, and reject any example whose JSON is not byte-equal to a file underfixtures/. - Fail the build if
ADVICEorCOMMITtext is non-stub, or if forbidden verbs appear in model-authored files.
Artifact: extract, template, and classify
The scripts below are a self-contained example that you should treat as unexecuted until the commands in the last section run. They implement extraction, template rendering, and a verb-level role gate for a single endpoint. Copy them into tools/ and tests/ before running the shell block that follows the listings.
# tools/extract_facts.py
from __future__ import annotations
import json
import sys
from pathlib import Path
try:
import yaml
except ImportError:
print("pip install pyyaml", file=sys.stderr)
raise
def extract(spec: dict) -> list[dict]:
facts = []
for path, methods in spec.get("paths", {}).items():
for method, op in methods.items():
if method.startswith("x-") or not isinstance(op, dict):
continue
params = []
for p in op.get("parameters", []):
schema = p.get("schema", {})
params.append(
{
"name": p.get("name"),
"in": p.get("in"),
"required": bool(p.get("required", False)),
"type": schema.get("type"),
"enum": schema.get("enum"),
}
)
codes = sorted(str(c) for c in (op.get("responses") or {}).keys())
facts.append(
{
"path": path,
"method": method.lower(),
"operation_id": op.get("operationId"),
"parameters": params,
"status_codes": codes,
}
)
facts.sort(key=lambda f: (f["path"], f["method"]))
return facts
def main() -> None:
spec_path = Path(sys.argv[1])
out_path = Path(sys.argv[2])
spec = yaml.safe_load(spec_path.read_text())
out_path.write_text(json.dumps(extract(spec), indent=2, sort_keys=True) + "\n")
if __name__ == "__main__":
main()
Render restatements from facts.json, not from a blank prompt. The template is intentionally dull, because dull sentences are easier to check for extra entities.
# tools/render_restate.py
from __future__ import annotations
import json
import sys
from pathlib import Path
def sentence_for(fact: dict) -> list[str]:
lines = [
f"`{fact['method'].upper()} {fact['path']}` returns one of: {', '.join(fact['status_codes'])}."
]
for p in fact["parameters"]:
req = "required" if p["required"] else "optional"
enum = f" enumerated as {p['enum']}" if p.get("enum") else ""
lines.append(
f"The {req} {p['in']} parameter `{p['name']}` is typed as {p['type']}{enum}."
)
return lines
def main() -> None:
facts = json.loads(Path(sys.argv[1]).read_text())
out = Path(sys.argv[2])
blocks = []
for fact in facts:
heading = f"### {fact['method'].upper()} {fact['path']}\n"
body = "\n".join(sentence_for(fact))
blocks.append(heading + "\n" + body + "\n")
out.write_text("\n".join(blocks))
if __name__ == "__main__":
main()
The classifier is the merge gate. It scans model output and the human stub file as two different classes of text, not as one markdown blob.
# tools/classify_docs.py
from __future__ import annotations
import re
import sys
from pathlib import Path
import yaml
def main() -> None:
permit = yaml.safe_load(Path(sys.argv[1]).read_text())
model_md = Path(sys.argv[2]).read_text()
human_md = Path(sys.argv[3]).read_text()
verbs = permit.get("forbidden_commitment_verbs", [])
pattern = re.compile(r"\b(" + "|".join(map(re.escape, verbs)) + r")\b", re.I)
errors = []
if pattern.search(model_md):
errors.append("model draft contains forbidden commitment verbs")
if "TODO(HUMAN)" not in human_md and "Signed-off-by:" not in human_md:
errors.append("COMMIT file lacks TODO(HUMAN) or Signed-off-by")
if errors:
print("\n".join(errors), file=sys.stderr)
sys.exit(1)
print("role gate passed")
if __name__ == "__main__":
main()
A tiny OpenAPI fixture keeps the example reproducible across machines. Keep this file small until the gate is green, then grow the surface area one path at a time.
# openapi.yaml
openapi: 3.0.3
info:
title: Pets
version: 0.1.0
paths:
/pets:
get:
operationId: listPets
parameters:
- name: limit
in: query
required: false
schema:
type: integer
responses:
"200":
description: ok
"400":
description: bad request
Human-owned text stays in a second file so generated restatements can be deleted and rebuilt. The stub below is the starting state, not a finished stability policy.
# docs/human/pets.commit.md
## GET /pets stability
TODO(HUMAN): state whether page size and sort order are part of the public contract.
Owner: api-docs
Commands that make the gate visible
Run extraction and classification as separate commands so a failing classifier cannot be mistaken for a failing parser. The optional paraphrase file is just another input to the same gate, not a privileged overlay.
python3 -m pip install pyyaml
python3 tools/extract_facts.py openapi.yaml facts.json
python3 tools/render_restate.py facts.json docs/generated/pets.restate.md
cp docs/generated/pets.restate.md docs/generated/pets.model.md
python3 tools/classify_docs.py docs/permits/pets.yaml \
docs/generated/pets.model.md \
docs/human/pets.commit.md
A pytest file should treat a planted compatibility guarantee as a red build rather than a warning. If this test is skipped, the rest of the workflow is documentation theater.
# tests/test_role_gate.py
from pathlib import Path
import subprocess
import sys
def test_forbidden_verb_fails(tmp_path: Path) -> None:
permit = Path("docs/permits/pets.yaml")
model = tmp_path / "model.md"
human = tmp_path / "human.md"
model.write_text("GET /pets is stable and we guarantee compatibility.\n")
human.write_text("TODO(HUMAN): pagination stability\n")
result = subprocess.run(
[sys.executable, "tools/classify_docs.py", str(permit), str(model), str(human)],
capture_output=True,
)
assert result.returncode != 0
Three leaks this gate is designed to catch
Commitment smuggling is the first leak, and it is also the most common in paraphrased reference pages. A rewrite adds "always returns 200 for valid keys" when the spec only lists 200 and 400 as documented codes. The extra adverb is not style; it is a new contract that no test file currently proves.
Example drift is the second leak, and byte equality is the cheapest defense against it. A model invents a pet object with extra fields that the fixture and schema do not share, and readers copy that object into production clients. If the example is not byte-equal to fixtures/, the section is not an example in this workflow.
Timeline leakage is the third leak, and it often hides inside a true restatement. A field marked deprecated: true becomes "will be removed next quarter" without a human COMMIT owner or date. The boolean is extractable; the quarter is not, and the classifier must keep those sentences in different files.
Decision table for reviewers
Reviewers can use the table below as a merge checklist instead of rereading every paragraph for tone. If a sentence cannot be placed in one row, split the sentence before generation. Do not weaken the table to make the model more fluent, because fluency is not evidence.
| Role | Evidence required | Model may draft? | Human must sign? |
|---|---|---|---|
| RESTATE | field present in facts.json
|
yes, after templates | no, review diffs only |
| EXAMPLE | byte-equal fixture | layout only | no, if fixture matches |
| ADVICE | runbook or incident note | no | yes |
| COMMIT | explicit owner plus date | no | yes |
Where a drafting model still helps
Template English is accurate, and it also becomes monotonous across a large public surface area. A model is useful when it paraphrases RESTATE sentences without adding entities, units, or timelines that facts.json does not contain. That rewrite remains a constrained pass over templates, not an unconstrained authoring session against a blank prompt.
Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode offers free model access and a free server option that can host this extract-and-gate loop while you iterate on permits. The workflow above does not depend on that hosting; the same scripts run with local Python and a checked-in OpenAPI file. Keep the model inside the RESTATE lane even when the host is convenient, because convenience does not convert a commitment into an extractable fact.
Limitations and who should skip this
This gate does not prove that restatements are kind, complete, or correctly ordered for newcomers. It only proves they are reconstructable and that commitment verbs did not leak into model output. Forbidden-verb lists are brittle against euphemism; "unchanged for the foreseeable future" evades the regex while remaining a COMMIT in substance.
Do not use this approach if your public docs are primarily narrative tutorials with no schema. Do not use it to auto-publish legal terms, pricing, or security promises. Do not treat a green classifier as a substitute for an API review when you are changing pagination, auth, or error semantics. Teams that already compile reference tables from OpenAPI tooling may still want the COMMIT stub file, because generated tables rarely include the sentences customers migrate on.
What to version, and what not to generate
Version the permit, the fact extractor, the fixtures, and the human COMMIT file. Do not version unconstrained model chat as if it were a source of record. If you keep an optional paraphrase, store it as a build artifact and require the classifier to pass on that artifact before merge.
The core rule stays narrow enough to enforce in CI without a second review culture. Models may draft sentences whose entities already live in a frozen contract extract. Humans own every sentence that tells a customer what the product will still do next quarter. If you try the role gate on one endpoint this week, start with the permit and the failing pytest, not with a longer prompt.
Top comments (0)