The typical regen failure is quiet. A /list-invoices page comes back from the model with a parameter table. The table looks complete. Then a client sends limit=0 because the table said 0 means “no cap.”
The OpenAPI file never said that. limit was optional. The server default was 20. Sending 0 returned an empty page. Sending a JSON null in a query string was a 400. Three states, one collapsed sentence.
This is not a tone problem. It is a contract problem. Models are good at filling tables. They are bad at preserving the difference between a missing field, an explicit null, and a numeric zero.
The collapse, in one response
Consider a list endpoint with this query shape:
parameters:
- name: limit
in: query
required: false
schema:
type: integer
minimum: 0
maximum: 100
default: 20
- name: cursor
in: query
required: false
schema:
type: string
nullable: true
- name: include_deleted
in: query
required: false
schema:
type: boolean
A docs model often emits something like this:
limit(integer, default0) — Number of invoices to return. Use0for all.
cursor— Opaque pagination token. Passnullto start from the beginning.
include_deleted— Optional. Defaults tofalseif omitted ornull.
Every line is fluent. Every line is wrong in a different way.
-
limitdefaulted to20, not0. Zero is a legal value with a different meaning. -
cursoromitted means “first page.”cursor=nullis not a documented start token. -
include_deletedhas no default in the schema. Treating omitted andnullasfalseis an invented policy.
If you only grep for hallucinated URLs, this page ships. The damage shows up in client SDKs and in support tickets, not in the Markdown linter.
What the model may draft
Keep the model inside the schema. If a fact is in OpenAPI, the model may restate it. If a fact is not in OpenAPI, the model may not invent a substitute.
Draftable (schema-backed):
- Parameter name, location (
query/path/header/body), and JSON type -
requiredvs optional, when the flag is explicit -
minimum,maximum,enum,pattern,maxLength - Example payloads copied from a checked-in fixture, not from the prompt
- Cross-links to other operations that already exist in the same spec
Not draftable (human-owned):
- Default values that are missing from the schema
- Equivalence claims: “omitted means X”, “null means Y”, “zero means Z”
- Server-side fallback timing (client default vs server default vs proxy default)
- Pagination opacity, cursor expiry, and what happens after a filter change
- Idempotency, retry, and partial-success behavior
- Units, timezones, rounding, and currency minor units
The split is mechanical. Name and type are structural. Meaning of absence is behavioral. Regenerating the first should never rewrite the second.
A decision table you can check in CI
| Schema facts | Draft lane | Human lane | Fail the regen if |
|---|---|---|---|
type + required
|
Parameter table row | — | Model adds a default not in schema |
default present |
Restate the default, quoted from schema | When the default is applied (client vs server) | Model changes the number or says “ignore if zero” |
nullable: true |
Mark the field nullable | What null does vs omit | Model equates null with omit or with zero |
no default, not required |
“Omitted is allowed” | What the server does on omit | Model invents “defaults to …” |
minimum: 0 on integer |
Show the bound | Whether 0 is empty, unlimited, or invalid |
Model says 0 means unlimited |
enum |
List members | Why a member exists | Model adds a member |
The table is the artifact, not the prose around it. If a row cannot be filled from the spec, the cell stays empty until a human writes it in a signed file.
Fixture: three states, one field family
Save this as fixtures/list-invoices.openapi.yaml. It is small on purpose. The gate should fail on meaning, not on file size.
openapi: 3.0.3
info:
title: Invoices
version: 0.0.0
paths:
/v1/invoices:
get:
operationId: listInvoices
parameters:
- name: limit
in: query
required: false
schema:
type: integer
minimum: 0
maximum: 100
default: 20
- name: cursor
in: query
required: false
schema:
type: string
nullable: false
- name: include_deleted
in: query
required: false
schema:
type: boolean
responses:
"200":
description: A page of invoices
Human-owned notes live beside the spec, not inside the generated page. Example: docs/owned/listInvoices.states.md.
# listInvoices — three-state notes
# owner: api-docs
# signed: true
## limit
- omit: server applies 20
- 0: empty page, not “unlimited”
- null: not a legal query value (400)
## cursor
- omit: first page
- empty string: 400
- null: 400
- opaque token: server-defined; do not document internals
## include_deleted
- omit: unspecified; do not claim a default
- true / false: filter as named
- null: 400
The generated Markdown may cite these notes. It may not paraphrase them into a friendlier default.
The gate: classify, then scan
The following checker is a self-contained example. Run it against the fixture. It is a contract linter, not a production docs platform.
#!/usr/bin/env python3
"""Fail docs regen when a model collapses omit / null / zero."""
from __future__ import annotations
import json
import re
import sys
from pathlib import Path
try:
import yaml
except ImportError:
yaml = None
INVENTED_DEFAULT = re.compile(
r"defaults?\s+to\s+[`'"]?(all|unlimited|none|null|0|false|true)",
re.I,
)
EQUATES_NULL_OMIT = re.compile(r"null\s+(means|is)\s+(omitted|absent|missing)", re.I)
ZERO_MEANS_ALL = re.compile(r"\b0\b.{0,40}(unlimited|all records|no cap)", re.I)
def load_spec(path: Path) -> dict:
text = path.read_text(encoding="utf-8")
if path.suffix in {".yaml", ".yml"}:
if yaml is None:
raise SystemExit("pip install pyyaml")
return yaml.safe_load(text)
return json.loads(text)
def iter_params(spec: dict):
for path, item in (spec.get("paths") or {}).items():
for method, op in item.items():
if not isinstance(op, dict):
continue
op_id = op.get("operationId") or f"{method.upper()} {path}"
for param in op.get("parameters") or []:
schema = param.get("schema") or {}
yield {
"op": op_id,
"name": param.get("name"),
"required": bool(param.get("required")),
"type": schema.get("type"),
"default": schema.get("default", _Missing),
"nullable": schema.get("nullable", False),
"minimum": schema.get("minimum", _Missing),
}
class _Missing:
pass
def classify(param: dict) -> dict:
flags = []
if param["default"] is _Missing and not param["required"]:
flags.append("no_default_do_not_invent")
if param["default"] is not _Missing:
flags.append("default_is_schema_owned")
if param["nullable"]:
flags.append("null_is_not_omit")
else:
flags.append("null_is_invalid_unless_spec_says")
if param["type"] == "integer" and param["minimum"] == 0:
flags.append("zero_is_a_value")
return {**param, "flags": flags}
def scan_markdown(md: str, classified: list[dict]) -> list[str]:
errors = []
if INVENTED_DEFAULT.search(md):
errors.append("invented_default_phrase")
if EQUATES_NULL_OMIT.search(md):
errors.append("null_equated_to_omit")
if ZERO_MEANS_ALL.search(md):
errors.append("zero_collapsed_to_unlimited")
for row in classified:
if "no_default_do_not_invent" in row["flags"]:
pat = re.compile(
rf"{re.escape(row['name'])}.{{0,80}}defaults?\s+to",
re.I | re.S,
)
if pat.search(md):
errors.append(f"invented_default:{row['op']}:{row['name']}")
if "zero_is_a_value" in row["flags"] and row["default"] != 0:
pat = re.compile(
rf"{re.escape(row['name'])}.{{0,80}}default\s+[`']?0",
re.I | re.S,
)
if pat.search(md):
errors.append(f"wrong_default_zero:{row['op']}:{row['name']}")
return errors
def main(argv: list[str]) -> int:
if len(argv) != 3:
print("usage: three_state_gate.py <openapi> <generated.md>", file=sys.stderr)
return 2
spec = load_spec(Path(argv[1]))
md = Path(argv[2]).read_text(encoding="utf-8")
classified = [classify(p) for p in iter_params(spec)]
errors = scan_markdown(md, classified)
print(json.dumps({"params": classified, "errors": errors}, indent=2, default=str))
return 1 if errors else 0
if __name__ == "__main__":
raise SystemExit(main(sys.argv))
A failing page is useful. Save this as fixtures/list-invoices.bad.md:
## Query parameters
- `limit` (integer, default `0`) — use `0` for unlimited.
- `cursor` — pass `null` to start. Null means omitted.
- `include_deleted` defaults to false.
Run:
python3 three_state_gate.py \
fixtures/list-invoices.openapi.yaml \
fixtures/list-invoices.bad.md
Expected: non-zero exit. The JSON report should include wrong_default_zero, zero_collapsed_to_unlimited, null_equated_to_omit, and invented_default:listInvoices:include_deleted.
A clean page restates schema facts and points at the signed notes:
## Query parameters
- `limit` (integer, optional, schema default `20`, minimum `0`). Zero is a legal value; it is not unlimited. See owned notes.
- `cursor` (string, optional, not nullable). Omit for the first page. Do not send null.
- `include_deleted` (boolean, optional). No schema default. Do not equate omit with false.
The gate does not try to understand invoices. It only refuses collapsed states.
Regen workflow
Keep three inputs, one output, one reject path.
-
Extract. Read OpenAPI. Classify each parameter with the table above. Write
classified.json. -
Draft. Prompt the model with the classified rows and a ban list: no default unless
defaultis present; no “null means omitted”; no “0 means all.” - Scan. Run the gate on the draft. Fail the job on any error code from the scanner.
- Join. If the scan passes, concatenate the draft with the signed three-state notes. Do not let the model rewrite the notes file.
- Publish. Only the joined document is canonical. The draft file is disposable.
Prompt text should be boring. A usable system prompt is a list of refusals, not a style guide.
You draft parameter tables from classified.json only.
If default is missing, write “no schema default.”
Never equate null with omit.
Never equate 0 with unlimited.
Never invent units, timezones, or retry rules.
If a human note file is provided, quote it; do not paraphrase defaults.
The model is a formatter sitting on a classifier. It is not the source of behavior.
Where a free model is enough
This job is classification plus constrained drafting. It does not need a long-running agent and it does not need the model to invent retry policy.
Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode’s free model access and free server option are enough to run the extract-draft-scan loop on a spec like the fixture: the model fills tables, the gate rejects collapsed states, and the signed notes stay off the regen path. That is the only product role here. The ownership split still works if you swap the model.
If you already treat OpenAPI as canonical, the useful experiment is the gate, not a new prose style.
Limitations
The scanner is phrase-based. A model can still smuggle a wrong default in a sentence the regex does not catch. Tighten it with AST checks on generated MDX, or require every default to appear as a fenced schema.default value.
OpenAPI 3.0 nullable is not the same as a 3.1 union type. The classifier above does not expand oneOf / anyOf. If your spec uses unions for “string or null,” extend classify() before you trust the gate.
Query parameters are the easy case. Body fields with additionalProperties, vendor extensions, and polymorphic discriminator objects need a different owned-notes layout. Do not reuse this script as a general API linter.
The gate does not prove the server matches the spec. If production applies a default of 50 while OpenAPI says 20, both the model and the human notes will be consistently wrong. Spec drift is a different pipeline.
Who should not use this
Skip the split if your docs are hand-written and rarely regenerated. A gate that assumes disposable drafts will only add ceremony.
Skip it if legal or safety text lives in the same Markdown file as parameter tables. Those files should not be model-writable at all.
Skip it if you cannot name an owner for the three-state notes. Unsigned “behavior” comments next to the spec will be overwritten, and the gate cannot tell a reviewed sentence from a leftover prompt.
The practical rule is narrow. Let the model draft the table. Keep omit, null, and zero in a signed file. Fail the regen when those three states collapse into one adjective.
Top comments (0)