Gate Model-Drafted API Docs on Evidence Hashes and Modal Verbs

Model-drafted API documentation stays useful only when tables stay bound to schemas and promises stay human-owned. A merge gate can enforce that split by hashing the OpenAPI source and by rejecting unowned modal verbs. The rest of this article specifies a proposed workflow, a scanner, and a human overlay file. Teams can adopt the pieces without changing their generator, provided generated files never mix reference rows with compatibility language.

Separate reference files from compatibility overlays

Generated Markdown often fails review because one file contains both extracted fields and invented guarantees. Parameter names, types, and example payloads can be restated from OpenAPI together with a content hash. Support windows, deprecation dates, retry policy, and breaking-change language cannot be restated from that schema. Mixing those classes in one pull request hides the judgment line from reviewers who scan tables quickly.

The proposed layout keeps two artifacts beside each operation so reviewers can diff them independently. A generated file named {operationId}.ref.md may contain tables, example JSON, and a footer hash. A human file named {operationId}.compat.yaml may contain dates, named owners, and modal commitments only. Continuous integration should fail when the reference file uses commitment vocabulary or when the overlay cites a missing operation.

What the model may draft

The draft step should read a pinned OpenAPI document and emit only restatable units for each operation. Allowed units include path strings, HTTP methods, parameter names, JSON Schema types, enumerations, and spec-provided examples. Allowed units also include error code lists that already exist as named response objects in the document. The generator must print a footer with the SHA-256 of the spec file and the operationId used.

The draft step must not write prose that implies duration, vendor support, or fitness for production traffic. It must not invent rate limits, availability minutes, or guidance that a failed request is always safe to retry. It must not relabel a field as deprecated unless the spec already sets deprecated: true on that field. Even then, the calendar date and the replacement endpoint belong in the overlay, not in generated prose.

What a human must own

Humans own every sentence that would survive a schema rewrite without remaining operationally true for customers. Compatibility calendars, partner promises, default retry budgets, and production-support flags all sit in that class. Incident-derived caveats, regional availability, and authentication exceptions are also human-owned, because they rarely appear as typed fields. The overlay file should record an owner identifier and a ticket URL for each remaining commitment.

If a reviewer cannot point to a schema node or a test name, the sentence does not belong in {operationId}.ref.md. Move that sentence into {operationId}.compat.yaml or delete it before the documentation merge completes. The scanner below encodes the rule as a vocabulary check plus a hash check, which is weaker than semantic proof. That cheaper check is still worth running on every pull request because it catches the usual leak of support language.

Numbered workflow

The proposed sequence is five steps and assumes the OpenAPI document is already reviewed as a contract.

1. Pin the specification. Store contracts/openapi.pinned.yaml in git and refuse generators that fetch a floating URL at draft time. Floating sources make the footer hash meaningless, because two runs can describe different contracts under one filename. Record the SHA-256 in the job log so a reviewer can compare it with the Markdown footer.

2. Extract fields without a model. Run a deterministic parser that writes fields.json with operationId, method, path, and parameter metadata. Sorting keys keeps the file stable across machines and makes diffs reviewable. This extraction is the evidence the formatter is allowed to see.

3. Draft reference tables only. A formatter, including a hosted model job, may turn fields.json into {operationId}.ref.md tables and example blocks. The prompt, checked into the repository, should list permitted units and forbid modal verbs explicitly. If the formatter emits a paragraph that is not a table or a fenced example, fail the job.

4. Scan before review. Execute scan_docs_gate.py against docs/ref and fail on modal verbs, calendar tokens, and missing or mismatched evidence footers. Extend the word list when legal or support teams use domain-specific commitment phrases. Do not accept a warning mode for public docs, because warnings are ignored under release pressure.

5. Render with a labeled overlay. Concatenate the reference Markdown with a Human commitments section sourced from YAML, and never send that YAML back through a model rewrite. Empty overlays are acceptable when an operation carries no customer promise. Non-empty overlays require owner and ticket on every commitment row.

Artifact: deterministic extraction

The extractor should stay boring and local. The following script is proposed example code, not a claim about a production corpus.

#!/usr/bin/env python3
"""Deterministic OpenAPI field extraction. Proposed example, unexecuted here."""
from __future__ import annotations

import argparse
import json
from pathlib import Path

try:
    import yaml
except ImportError as exc:  # pragma: no cover
    raise SystemExit("install pyyaml to run this proposed extractor") from exc


def extract(spec: dict) -> list[dict]:
    rows: list[dict] = []
    for path, item in (spec.get("paths") or {}).items():
        if not isinstance(item, dict):
            continue
        for method, op in item.items():
            if not isinstance(op, dict) or "operationId" not in op:
                continue
            params = [
                {
                    "deprecated": bool(p.get("deprecated")),
                    "in": p.get("in"),
                    "name": p.get("name"),
                    "required": bool(p.get("required")),
                }
                for p in op.get("parameters") or []
                if isinstance(p, dict)
            ]
            rows.append(
                {
                    "method": method.upper(),
                    "operationId": op["operationId"],
                    "parameters": params,
                    "path": path,
                }
            )
    rows.sort(key=lambda r: r["operationId"])
    return rows


def main() -> None:
    parser = argparse.ArgumentParser()
    parser.add_argument("--spec", required=True)
    parser.add_argument("--out", required=True)
    args = parser.parse_args()
    spec = yaml.safe_load(Path(args.spec).read_text(encoding="utf-8"))
    Path(args.out).write_text(
        json.dumps(extract(spec), indent=2, sort_keys=True) + "\n",
        encoding="utf-8",
    )


if __name__ == "__main__":
    main()

Artifact: commitment-language gate

The gate reads generated Markdown only, while overlay YAML is validated separately for required keys.

#!/usr/bin/env python3
"""Proposed docs gate: evidence footer plus modal-verb rejection."""
from __future__ import annotations

import hashlib
import re
import sys
from pathlib import Path

MODAL = re.compile(
    r"\b(must|shall|should|always|never|guarantee[ds]?|"
    r"supported|compatible|will not break|sla|uptime|"
    r"deprecated on|production-ready)\b",
    re.I,
)
DATE = re.compile(r"\b(20\d{2}-\d{2}-\d{2}|Q[1-4]\s*20\d{2})\b")
FOOTER = re.compile(
    r"<!-- evidence: sha256=(?P<sha>[0-9a-f]{64}) "
    r"operation=(?P<op>[A-Za-z0-9_]+) -->"
)


def sha256(path: Path) -> str:
    return hashlib.sha256(path.read_bytes()).hexdigest()


def scan_ref(path: Path, spec_hash: str) -> list[str]:
    text = path.read_text(encoding="utf-8")
    errors: list[str] = []
    footer = FOOTER.search(text)
    if not footer:
        errors.append(f"{path}: missing evidence footer")
        return errors
    if footer.group("sha") != spec_hash:
        errors.append(f"{path}: footer hash does not match pinned OpenAPI")
    body = text[: footer.start()]
    for i, line in enumerate(body.splitlines(), 1):
        if line.strip().startswith("|") and "deprecated" in line.lower():
            continue  # boolean column copied from spec is allowed
        if MODAL.search(line):
            errors.append(f"{path}:{i}: unowned modal or support language")
        if DATE.search(line):
            errors.append(f"{path}:{i}: calendar token belongs in overlay")
    return errors


def main(argv: list[str]) -> int:
    spec = Path(argv[1])
    refs = Path(argv[2])
    spec_hash = sha256(spec)
    errors: list[str] = []
    for path in sorted(refs.glob("*.ref.md")):
        errors.extend(scan_ref(path, spec_hash))
    for err in errors:
        print(err, file=sys.stderr)
    return 1 if errors else 0


if __name__ == "__main__":
    raise SystemExit(main(sys.argv))

The overlay remains a small YAML file with explicit owners, tickets, and commitment strings for reviewers.

# billingGetInvoice.compat.yaml — human-owned, never generated
operationId: billingGetInvoice
owner: api-steward
ticket: "https://tracker.example/DOC-1842"
supported_until: "2027-03-31"
commitments:
  - text: Idempotent retries are safe for HTTP 409 on this operation.
    owner: api-steward
  - text: Field taxRegion remains optional through the supported_until date.
    owner: api-steward

A generated reference file should end with a machine footer rather than a slogan. Example shape for reviewers and for the scanner:

| name | in | required | deprecated |
| --- | --- | --- | --- |
| invoiceId | path | true | false |

json
{
"invoiceId": "inv_123"
}

<!-- evidence: sha256=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa operation=billingGetInvoice -->

shell

Commands for a local check

The following commands assume a vendored spec and a docs/ref output directory. They are a proposed sequence, not a measured benchmark of draft quality.

sha256sum contracts/openapi.pinned.yaml
python3 tools/extract_fields.py \
  --spec contracts/openapi.pinned.yaml \
  --out /tmp/fields.json
python3 tools/draft_ref_tables.py \
  --fields /tmp/fields.json \
  --out docs/ref
python3 tools/scan_docs_gate.py \
  contracts/openapi.pinned.yaml \
  docs/ref
python3 tools/check_overlay.py \
  --spec contracts/openapi.pinned.yaml \
  --overlays docs/compat

extract_fields.py should remain deterministic and should not call a model under any branch. draft_ref_tables.py may call a model only to format tables from fields.json. check_overlay.py should confirm each operationId still exists and that every commitment row has owner plus ticket.

Decision table for reviewers

Unit in the draft	Allowed in .ref.md	Required overlay field	Scanner action
Path, method, operationId	Yes	None	Pass
Parameter name and type	Yes	None	Pass
Example JSON copied from spec	Yes	None	Pass
Deprecated boolean already in spec	Yes, as a column	Replacement and date	Pass column, fail date prose
Supported-until or SLA language	No	supported_until, owner	Fail merge
Retry or idempotency advice	No	commitments[]	Fail merge
Regional authentication exception	No	commitments[]	Fail merge

Reviewers can apply the table without reading model logs or prompt transcripts. If a cell forbids the unit in .ref.md, moving the sentence into YAML is cheaper than negotiating with the formatter. If the scanner flags a table heading that contains "must", rename the heading rather than shrinking the vocabulary list.

Where a hosted draft step fits

Disclosure: This article was prepared as part of MonkeyCode's product outreach. A free model access path and a free server option can run the table formatter when a team lacks a dedicated docs runner. The scanner, extractor, and overlay remain ordinary repository objects and do not depend on that host. Treat hosting as interchangeable infrastructure, and keep the pinned spec plus the gate script in git. If a free model endpoint already formats your tables, attach this gate after the formatter rather than expanding the prompt.

Limitations

The vocabulary list cannot prove that a sentence is a promise, and it cannot prove that a table is complete. Hash equality shows the spec file did not change under the footer, not that every field was copied. Models can still omit a required parameter while producing a clean scan, which is why extraction should be deterministic. Teams that need completeness should render rows from fields.json with a template engine and skip the model formatter for shipping pages.

The gate also fails closed on dates, which will annoy changelog writers who want an added-on stamp inside reference files. Put changelog dates in a human-owned CHANGELOG.md, or extend the scanner to allow dates only inside a fenced changelog block. Do not loosen the global date regular expression without that fence, because support calendars will leak back into generated tables. Legal teams may also need additional tokens such as warranty or indemnify in the modal list.

Who should not use this approach

Do not adopt this split if public docs are entirely human-written and already cite tests for every paragraph. Do not adopt it if counsel requires inline legal language inside every operation page without a second file. Do not treat the scanner as a substitute for contract tests, because customers experience runtime behavior rather than Markdown tables. Skip the model formatter entirely when the field list is small enough to render with a template engine.

The core conclusion does not change under those exceptions for generated reference material. Reference tables can be generated from a pinned spec; compatibility language cannot be generated safely. Hash the source, reject unowned modal verbs, and keep a named human on every remaining promise.