DEV Community: Morgan Sun

Generated Pages Need Owners: A Lane Manifest for API Docs

Morgan Sun — Thu, 24 Sep 2026 19:52:59 +0000

A release candidate goes out with regenerated endpoint pages. The new prose is easier to scan than the old notes. One added sentence tells clients they can cache a successful response for a day. Another says a 404 means the order was deleted, not that it was missing.

Treat that scene as a composite. It is not a measured incident from this account. The OpenAPI document in the scene says neither sentence. The page is clear anyway. The owner of the two promises is not.

A recent public thread on DEV separated clean code from clear code, and treated comments as a clarity tool rather than a violation (topic signal). The documentation version of that split is stricter. A generated walkthrough can be clear and still unauthorized. This piece is a workflow for that boundary, with a proposed checker. It is not a report of production rates.

The job split

This workflow asks one question before anyone prompts a model. Which sentences may be drafted, and which sentences must be owned by a named person?

Three lanes are enough for an endpoint page. More lanes tend to become a second specification. Keep the split mechanical so a script can enforce it.

Narrative

The narrative lane explains fields and call order using only what the spec already states. A model may draft it. A human reviews it for invented steps. Regeneration is allowed when the spec changes.

Specimen

The specimen lane holds request and response examples. This article does not reopen how fixtures relate to SDK snippets. The boundary is enough. Examples do not live in the narrative file, and a prose pass does not invent them.

Promise

The promise lane is compatibility language. Caching, retry rules, deletion meaning, deprecation windows, and any sentence with "always", "never", "guaranteed", or a time bound belong here. A model must not draft this lane. A named human writes it, dates it, and locks it against regeneration.

A decision matrix

Use the matrix in review. Do not paste it into a prompt and call that a control.

Field restatement, such as id is a string. Lane: narrative. Keep it only when the spec agrees.
Sequence help, such as "create the order, then fetch it." Lane: narrative. Reject it when the second call is not in the source.
Status gloss absent from the spec, such as "404 means deleted." Lane: promise. Drop it, or have a human write a sourced sentence.
Time bound, such as "cache for 24 hours." Lane: promise. Drop it unless a human owner accepts the number and names the source.
Sample JSON. Lane: specimen. Do not let a narrative pass invent bodies.

The matrix is a review aid. It is not a measured failure rate. No frequency was collected for this draft.

The ownership manifest

The paths below are a proposed layout. They are not files from a published repository. Dates in the sample are review markers, not evidence that a team signed them.

version: 1
page: endpoints/orders.get
lanes:
  narrative:
    path: docs/orders.get.narrative.md
    regenerate: true
    inputs:
      - openapi/orders.yaml
  specimen:
    path: docs/orders.get.examples.md
    fixture: fixtures/orders/get-200.json
  promise:
    path: docs/orders.get.promise.md
    owner: docs-review
    signed_on: 2026-09-25
    forbid_regen: true
    source: openapi/orders.yaml#/paths/~1orders~1{id}/get

The promise file stays short. Length is not ownership. Policy changes happen by editing this file and moving the date, not by regenerating the page.

<!-- regen:forbidden -->
Owner: docs-review
Signed-on: 2026-09-25
Source: openapi/orders.yaml#/paths/~1orders~1{id}/get

This operation publishes no cache lifetime.
A 404 means no matching order was found. It does not mean the order was deleted.

If product later decides that a 404 does mean deletion, a human edits those two sentences and changes Signed-on. A regeneration job does not get a vote.

The checker

The script is a proposed example. It was not executed for this draft. It needs PyYAML. It does not call a model. It fails when the promise lane is unlocked, or when narrative prose contains a small set of promise-like phrases.

#!/usr/bin/env python3
"""Lane checker for generated API pages. Proposed example; not run for this draft."""

import argparse
import pathlib
import re
import sys

try:
    import yaml
except ImportError:
    sys.exit("PyYAML is required: pip install pyyaml")

PROMISE_HINTS = (
    r"\bsafe to cache\b",
    r"\bguaranteed\b",
    r"\balways returns\b",
    r"\bwill not change\b",
    r"\bbackward[- ]compatible\b",
    r"\bretry until\b",
    r"\bmeans the resource was deleted\b",
    r"\bfor \d+ (?:hours|minutes|days)\b",
)
PROMISE_HEADERS = ("owner:", "signed-on:", "source:")


def main() -> int:
    parser = argparse.ArgumentParser()
    parser.add_argument("--manifest", required=True)
    args = parser.parse_args()
    manifest_path = pathlib.Path(args.manifest)
    data = yaml.safe_load(manifest_path.read_text(encoding="utf-8"))
    lanes = data.get("lanes") if isinstance(data, dict) else None
    if not isinstance(lanes, dict):
        print(f"{manifest_path}: missing lanes")
        return 1

    errors: list[str] = []
    narrative = lanes.get("narrative") or {}
    promise = lanes.get("promise") or {}

    if not promise.get("forbid_regen"):
        errors.append("promise.forbid_regen must be true")
    if not promise.get("owner") or not promise.get("signed_on") or not promise.get("source"):
        errors.append("promise owner, signed_on, and source are required")

    narrative_path = pathlib.Path(narrative.get("path", ""))
    promise_path = pathlib.Path(promise.get("path", ""))
    if not narrative_path.is_file():
        errors.append(f"missing narrative file: {narrative_path}")
    else:
        text = narrative_path.read_text(encoding="utf-8")
        for pattern in PROMISE_HINTS:
            for match in re.finditer(pattern, text, flags=re.IGNORECASE):
                errors.append(
                    f"{narrative_path}: promise-like phrase in narrative: {match.group(0)!r}"
                )

    if not promise_path.is_file():
        errors.append(f"missing promise file: {promise_path}")
    else:
        body = promise_path.read_text(encoding="utf-8")
        lowered = body.lower()
        if "<!-- regen:forbidden -->" not in lowered:
            errors.append(f"{promise_path}: missing regen lock comment")
        for header in PROMISE_HEADERS:
            if header not in lowered:
                errors.append(f"{promise_path}: missing header {header}")

    if errors:
        print("\n".join(errors))
        return 1
    print("lane check passed")
    return 0


if __name__ == "__main__":
    raise SystemExit(main())

Run it from the directory that contains docs/. These commands are part of the proposal. They are not a transcript.

python3 -m pip install pyyaml
python3 check_doc_lanes.py --manifest docs/orders.get.lanes.yaml

A useful negative case is a narrative line that says clients may cache the response for 24 hours. The script should exit non-zero and name that phrase. That is the intended behavior of the example, not a captured log. Run it locally before you wire it to CI.

What the model may see

Give the draft pass only the narrative inputs listed in the manifest. Do not attach the promise file as a style sample. Nearby policy sentences get copied.

Keep the prompt boundary dull and specific.

Paste the operation from the spec, not the whole repository.
Name the output as the narrative path only.
Forbid cache lifetimes, retry policy, deletion meaning, and compatibility claims.
Ask for no sample JSON in that file.

Then run the checker. A green run means the hint list did not match. It does not mean the prose is free of policy. Read the diff anyway. The read is the control the script cannot be.

Where a free draft host fits

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The operator states that MonkeyCode offers free model access and a free server option. This draft does not name models, token quotas, hardware, or a time window. Those details were not verified here. Read them from current project documentation before depending on them.

Use that access after the manifest exists, not before. A free model pass may draft the narrative file from the spec excerpt. A free server, if you use the hosted option, is only a place to run the checker so review is not tied to one laptop. Your repository CI remains the merge gate. The host does not own the promise file.

If a hosted draft lane is useful once that gate is in the repo, confirm the free option in the project docs and then try it. Do not add a host in order to invent cache rules or status meanings.

Limitations

Phrase misses

The hint list is short. "Clients can keep the body until the next deploy" will not match safe to cache. Paraphrase is the usual miss. Extend the list from your own incident notes, and still read the narrative file.

Sign-off is not behavior

A signed promise can be false. The date records who accepted the sentence. It does not prove the server behaves that way. Contract tests remain the behavioral check. This script only guards against policy smuggled into a regenerated walkthrough.

Who should skip it

Skip the approach in these cases.

No named person will accept the promise file. A shared label with no rotation is theater.
The page is a security, privacy, or legal commitment. That text needs its own review path.
You want the model to choose product policy. That repeats the opening failure.
The pipeline assumes a fixed free quota or a permanent runner. Availability can change. Keep the checker in the repo so local CI still runs.

Review order

Follow the same order on each change. Skipping to generation is how promise sentences reappear.

Update the spec, or record the human decision the spec does not yet encode.
Edit the promise file only if policy changed, and change Signed-on.
Regenerate narrative from spec inputs, not from the previous narrative.
Run the lane checker.
Read every sentence the hints did not catch. Budget that read. Do not replace it with a green script.

Clear generated prose is a drafting result. A compatibility sentence is a decision. The split holds only when those live in different files, with different rights to regenerate.

Null, Zero, Absent: A Three-State Gate for AI-Drafted API Docs

Morgan Sun — Wed, 23 Sep 2026 16:29:53 +0000

The typical regen failure is quiet. A /list-invoices page comes back from the model with a parameter table. The table looks complete. Then a client sends limit=0 because the table said 0 means “no cap.”

The OpenAPI file never said that. limit was optional. The server default was 20. Sending 0 returned an empty page. Sending a JSON null in a query string was a 400. Three states, one collapsed sentence.

This is not a tone problem. It is a contract problem. Models are good at filling tables. They are bad at preserving the difference between a missing field, an explicit null, and a numeric zero.

The collapse, in one response

Consider a list endpoint with this query shape:

parameters:
  - name: limit
    in: query
    required: false
    schema:
      type: integer
      minimum: 0
      maximum: 100
      default: 20
  - name: cursor
    in: query
    required: false
    schema:
      type: string
      nullable: true
  - name: include_deleted
    in: query
    required: false
    schema:
      type: boolean

A docs model often emits something like this:

limit (integer, default 0) — Number of invoices to return. Use 0 for all.
cursor — Opaque pagination token. Pass null to start from the beginning.
include_deleted — Optional. Defaults to false if omitted or null.

Every line is fluent. Every line is wrong in a different way.

limit defaulted to 20, not 0. Zero is a legal value with a different meaning.
cursor omitted means “first page.” cursor=null is not a documented start token.
include_deleted has no default in the schema. Treating omitted and null as false is an invented policy.

If you only grep for hallucinated URLs, this page ships. The damage shows up in client SDKs and in support tickets, not in the Markdown linter.

What the model may draft

Keep the model inside the schema. If a fact is in OpenAPI, the model may restate it. If a fact is not in OpenAPI, the model may not invent a substitute.

Draftable (schema-backed):

Parameter name, location (query / path / header / body), and JSON type
required vs optional, when the flag is explicit
minimum, maximum, enum, pattern, maxLength
Example payloads copied from a checked-in fixture, not from the prompt
Cross-links to other operations that already exist in the same spec

Not draftable (human-owned):

Default values that are missing from the schema
Equivalence claims: “omitted means X”, “null means Y”, “zero means Z”
Server-side fallback timing (client default vs server default vs proxy default)
Pagination opacity, cursor expiry, and what happens after a filter change
Idempotency, retry, and partial-success behavior
Units, timezones, rounding, and currency minor units

The split is mechanical. Name and type are structural. Meaning of absence is behavioral. Regenerating the first should never rewrite the second.

A decision table you can check in CI

Schema facts	Draft lane	Human lane	Fail the regen if
`type` + `required`	Parameter table row	—	Model adds a default not in schema
`default` present	Restate the default, quoted from schema	When the default is applied (client vs server)	Model changes the number or says “ignore if zero”
`nullable: true`	Mark the field nullable	What null does vs omit	Model equates null with omit or with zero
no `default`, not required	“Omitted is allowed”	What the server does on omit	Model invents “defaults to …”
`minimum: 0` on integer	Show the bound	Whether `0` is empty, unlimited, or invalid	Model says `0` means unlimited
`enum`	List members	Why a member exists	Model adds a member

The table is the artifact, not the prose around it. If a row cannot be filled from the spec, the cell stays empty until a human writes it in a signed file.

Fixture: three states, one field family

Save this as fixtures/list-invoices.openapi.yaml. It is small on purpose. The gate should fail on meaning, not on file size.

openapi: 3.0.3
info:
  title: Invoices
  version: 0.0.0
paths:
  /v1/invoices:
    get:
      operationId: listInvoices
      parameters:
        - name: limit
          in: query
          required: false
          schema:
            type: integer
            minimum: 0
            maximum: 100
            default: 20
        - name: cursor
          in: query
          required: false
          schema:
            type: string
            nullable: false
        - name: include_deleted
          in: query
          required: false
          schema:
            type: boolean
      responses:
        "200":
          description: A page of invoices

Human-owned notes live beside the spec, not inside the generated page. Example: docs/owned/listInvoices.states.md.

# listInvoices — three-state notes
# owner: api-docs
# signed: true

## limit
- omit: server applies 20
- 0: empty page, not “unlimited”
- null: not a legal query value (400)

## cursor
- omit: first page
- empty string: 400
- null: 400
- opaque token: server-defined; do not document internals

## include_deleted
- omit: unspecified; do not claim a default
- true / false: filter as named
- null: 400

The generated Markdown may cite these notes. It may not paraphrase them into a friendlier default.

The gate: classify, then scan

The following checker is a self-contained example. Run it against the fixture. It is a contract linter, not a production docs platform.

#!/usr/bin/env python3
"""Fail docs regen when a model collapses omit / null / zero."""
from __future__ import annotations

import json
import re
import sys
from pathlib import Path

try:
    import yaml
except ImportError:
    yaml = None

INVENTED_DEFAULT = re.compile(
    r"defaults?\s+to\s+[`'"]?(all|unlimited|none|null|0|false|true)",
    re.I,
)
EQUATES_NULL_OMIT = re.compile(r"null\s+(means|is)\s+(omitted|absent|missing)", re.I)
ZERO_MEANS_ALL = re.compile(r"\b0\b.{0,40}(unlimited|all records|no cap)", re.I)


def load_spec(path: Path) -> dict:
    text = path.read_text(encoding="utf-8")
    if path.suffix in {".yaml", ".yml"}:
        if yaml is None:
            raise SystemExit("pip install pyyaml")
        return yaml.safe_load(text)
    return json.loads(text)


def iter_params(spec: dict):
    for path, item in (spec.get("paths") or {}).items():
        for method, op in item.items():
            if not isinstance(op, dict):
                continue
            op_id = op.get("operationId") or f"{method.upper()} {path}"
            for param in op.get("parameters") or []:
                schema = param.get("schema") or {}
                yield {
                    "op": op_id,
                    "name": param.get("name"),
                    "required": bool(param.get("required")),
                    "type": schema.get("type"),
                    "default": schema.get("default", _Missing),
                    "nullable": schema.get("nullable", False),
                    "minimum": schema.get("minimum", _Missing),
                }


class _Missing:
    pass


def classify(param: dict) -> dict:
    flags = []
    if param["default"] is _Missing and not param["required"]:
        flags.append("no_default_do_not_invent")
    if param["default"] is not _Missing:
        flags.append("default_is_schema_owned")
    if param["nullable"]:
        flags.append("null_is_not_omit")
    else:
        flags.append("null_is_invalid_unless_spec_says")
    if param["type"] == "integer" and param["minimum"] == 0:
        flags.append("zero_is_a_value")
    return {**param, "flags": flags}


def scan_markdown(md: str, classified: list[dict]) -> list[str]:
    errors = []
    if INVENTED_DEFAULT.search(md):
        errors.append("invented_default_phrase")
    if EQUATES_NULL_OMIT.search(md):
        errors.append("null_equated_to_omit")
    if ZERO_MEANS_ALL.search(md):
        errors.append("zero_collapsed_to_unlimited")
    for row in classified:
        if "no_default_do_not_invent" in row["flags"]:
            pat = re.compile(
                rf"{re.escape(row['name'])}.{{0,80}}defaults?\s+to",
                re.I | re.S,
            )
            if pat.search(md):
                errors.append(f"invented_default:{row['op']}:{row['name']}")
        if "zero_is_a_value" in row["flags"] and row["default"] != 0:
            pat = re.compile(
                rf"{re.escape(row['name'])}.{{0,80}}default\s+[`']?0",
                re.I | re.S,
            )
            if pat.search(md):
                errors.append(f"wrong_default_zero:{row['op']}:{row['name']}")
    return errors


def main(argv: list[str]) -> int:
    if len(argv) != 3:
        print("usage: three_state_gate.py <openapi> <generated.md>", file=sys.stderr)
        return 2
    spec = load_spec(Path(argv[1]))
    md = Path(argv[2]).read_text(encoding="utf-8")
    classified = [classify(p) for p in iter_params(spec)]
    errors = scan_markdown(md, classified)
    print(json.dumps({"params": classified, "errors": errors}, indent=2, default=str))
    return 1 if errors else 0


if __name__ == "__main__":
    raise SystemExit(main(sys.argv))

A failing page is useful. Save this as fixtures/list-invoices.bad.md:

## Query parameters

- `limit` (integer, default `0`) — use `0` for unlimited.
- `cursor` — pass `null` to start. Null means omitted.
- `include_deleted` defaults to false.

Run:

python3 three_state_gate.py \
  fixtures/list-invoices.openapi.yaml \
  fixtures/list-invoices.bad.md

Expected: non-zero exit. The JSON report should include wrong_default_zero, zero_collapsed_to_unlimited, null_equated_to_omit, and invented_default:listInvoices:include_deleted.

A clean page restates schema facts and points at the signed notes:

## Query parameters

- `limit` (integer, optional, schema default `20`, minimum `0`). Zero is a legal value; it is not unlimited. See owned notes.
- `cursor` (string, optional, not nullable). Omit for the first page. Do not send null.
- `include_deleted` (boolean, optional). No schema default. Do not equate omit with false.

The gate does not try to understand invoices. It only refuses collapsed states.

Regen workflow

Keep three inputs, one output, one reject path.

Extract. Read OpenAPI. Classify each parameter with the table above. Write classified.json.
Draft. Prompt the model with the classified rows and a ban list: no default unless default is present; no “null means omitted”; no “0 means all.”
Scan. Run the gate on the draft. Fail the job on any error code from the scanner.
Join. If the scan passes, concatenate the draft with the signed three-state notes. Do not let the model rewrite the notes file.
Publish. Only the joined document is canonical. The draft file is disposable.

Prompt text should be boring. A usable system prompt is a list of refusals, not a style guide.

You draft parameter tables from classified.json only.
If default is missing, write “no schema default.”
Never equate null with omit.
Never equate 0 with unlimited.
Never invent units, timezones, or retry rules.
If a human note file is provided, quote it; do not paraphrase defaults.

The model is a formatter sitting on a classifier. It is not the source of behavior.

Where a free model is enough

This job is classification plus constrained drafting. It does not need a long-running agent and it does not need the model to invent retry policy.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode’s free model access and free server option are enough to run the extract-draft-scan loop on a spec like the fixture: the model fills tables, the gate rejects collapsed states, and the signed notes stay off the regen path. That is the only product role here. The ownership split still works if you swap the model.

If you already treat OpenAPI as canonical, the useful experiment is the gate, not a new prose style.

Limitations

The scanner is phrase-based. A model can still smuggle a wrong default in a sentence the regex does not catch. Tighten it with AST checks on generated MDX, or require every default to appear as a fenced schema.default value.

OpenAPI 3.0 nullable is not the same as a 3.1 union type. The classifier above does not expand oneOf / anyOf. If your spec uses unions for “string or null,” extend classify() before you trust the gate.

Query parameters are the easy case. Body fields with additionalProperties, vendor extensions, and polymorphic discriminator objects need a different owned-notes layout. Do not reuse this script as a general API linter.

The gate does not prove the server matches the spec. If production applies a default of 50 while OpenAPI says 20, both the model and the human notes will be consistently wrong. Spec drift is a different pipeline.

Who should not use this

Skip the split if your docs are hand-written and rarely regenerated. A gate that assumes disposable drafts will only add ceremony.

Skip it if legal or safety text lives in the same Markdown file as parameter tables. Those files should not be model-writable at all.

Skip it if you cannot name an owner for the three-state notes. Unsigned “behavior” comments next to the spec will be overwritten, and the gate cannot tell a reviewed sentence from a leftover prompt.

The practical rule is narrow. Let the model draft the table. Keep omit, null, and zero in a signed file. Fail the regen when those three states collapse into one adjective.

The HTTP Fixture Is Canonical. SDK Samples Are Syntax.

Morgan Sun — Tue, 22 Sep 2026 14:55:08 +0000

A review queue once held two snippets from the same docs page. The Python sample put customer_id in the query string. The curl sample put it in the JSON body. Both had been drafted the same afternoon, from the same OpenAPI file, by the same class of model.

This is a reconstructed incident pattern, not a customer report. It is common enough that the failure is structural. Multi-language examples are not “the same request in different syntax” unless something outside the model owns the request.

The model is a reasonable syntax clerk. It is a poor registrar of HTTP identity. Docs that let it own both jobs will drift, and the drift is hard to see in a markdown diff.

What actually diverged

The OpenAPI document still said POST /v1/invoices with customer_id in the body. Neither sample was “unauthorized.” Each was locally plausible. Python HTTP clients often encourage query params. Curl examples often start from a JSON blob the model saw in a tutorial.

Three classes of drift show up in review:

Location drift: a field moves between path, query, header, and body.
Shape drift: a scalar becomes an object, a string becomes an integer, an optional field becomes required in the sample only.
Header drift: Authorization vs X-Api-Key, Content-Type omitted, Idempotency-Key invented or dropped.

None of these are spelling problems. They are contract problems wearing syntax.

Ownership split

Treat every SDK sample as two documents glued together. One of them may be drafted. The other must be signed.

The model may draft

Import lines and client-library boilerplate
Local variable names, comments, and language idioms
How the frozen request is expressed in that HTTP library
Retry/sleep examples that are clearly marked non-normative

A human must own

Method, path, and which parameters live where
Auth scheme and which headers are required
Field names, types, and requiredness for the example payload
Status codes presented as contractual
Idempotency, pagination, and ordering promises
Any numeral that looks like a quota, SLA, timeout, or rate limit

If a sentence would still be true after you deleted every SDK, a human owns it. If it would be false after you swapped Python for Go, the model may draft it.

Freeze one fixture per operation

Do not start from prose. Start from a file that cannot speak in two dialects at once.

# fixtures/create_invoice.yaml
id: create_invoice
method: POST
path: /v1/invoices
headers:
  Authorization: "Bearer $API_KEY"
  Content-Type: application/json
  Idempotency-Key: "$IDEMPOTENCY_KEY"
query: {}
body:
  customer_id: cus_123
  currency: usd
  amount_cents: 1999
ownership:
  human:
    - method
    - path
    - headers.Authorization
    - headers.Idempotency-Key
    - body
  model:
    - language_syntax
    - comments
    - client_library_calls

The fixture is the only object that may define the wire request. Examples may restate it. They may not edit it.

Each sample then carries a machine-readable echo, not as documentation for humans, but as a checksum for CI:

# examples/create_invoice.py
# HTTP-CANONICAL: {"id":"create_invoice","method":"POST","path":"/v1/invoices","headers":{"Authorization":"Bearer $API_KEY","Content-Type":"application/json","Idempotency-Key":"$IDEMPOTENCY_KEY"},"query":{},"body":{"customer_id":"cus_123","currency":"usd","amount_cents":1999}}

import os
import requests

resp = requests.post(
    "https://api.example.com/v1/invoices",
    headers={
        "Authorization": f"Bearer {os.environ['API_KEY']}",
        "Content-Type": "application/json",
        "Idempotency-Key": os.environ["IDEMPOTENCY_KEY"],
    },
    json={
        "customer_id": "cus_123",
        "currency": "usd",
        "amount_cents": 1999,
    },
    timeout=30,
)
print(resp.status_code)

The timeout=30 is a client default in the sample. It is not an SLA. Keep that distinction boring and explicit. If the page needs a server-side time guarantee, a human writes it next to a source, not inside the SDK snippet.

A checker you can run

The script below is a proposal you can save as tools/check_example_fixtures.py. It uses only the standard library. It does not call a network. It fails when the canonical block disagrees with the fixture, or when the surrounding markdown makes a guarantee the fixture does not support.

#!/usr/bin/env python3
"""Fail CI when SDK samples drift from frozen HTTP fixtures."""
from __future__ import annotations

import json
import pathlib
import re
import sys

ROOT = pathlib.Path(__file__).resolve().parents[1]
FIXTURES = ROOT / "fixtures"
EXAMPLES = ROOT / "examples"
CANON_RE = re.compile(r"HTTP-CANONICAL:\s*(\{.*\})")
GUARANTEE_RE = re.compile(
    r"\b(always|never|guarantees?|SLA|p99|rate limit|timeout of)\b",
    re.I,
)

def load_fixtures() -> dict[str, dict]:
    out = {}
    for path in sorted(FIXTURES.glob("*.yaml")):
        # Tiny YAML subset: "key: value" and nested 2-space maps.
        data = parse_simple_yaml(path.read_text())
        out[data["id"]] = data
    return out

def parse_simple_yaml(text: str) -> dict:
    """Enough YAML for this fixture shape. Not a general parser."""
    try:
        import yaml  # optional
        return yaml.safe_load(text)
    except ImportError:
        raise SystemExit("Install pyyaml, or emit fixtures as JSON.")

def canonical_view(fx: dict) -> dict:
    return {
        "id": fx["id"],
        "method": fx["method"],
        "path": fx["path"],
        "headers": fx.get("headers") or {},
        "query": fx.get("query") or {},
        "body": fx.get("body") or {},
    }

def check_example(path: pathlib.Path, fixtures: dict) -> list[str]:
    errors = []
    text = path.read_text()
    match = CANON_RE.search(text)
    if not match:
        return [f"{path}: missing HTTP-CANONICAL block"]
    try:
        echo = json.loads(match.group(1))
    except json.JSONDecodeError as exc:
        return [f"{path}: canonical JSON is invalid ({exc})"]
    fx = fixtures.get(echo.get("id"))
    if fx is None:
        return [f"{path}: unknown fixture id {echo.get('id')!r}"]
    expected = canonical_view(fx)
    if echo != expected:
        errors.append(
            f"{path}: canonical echo drifted from fixtures/{fx['id']}.yaml\n"
            f"  expected: {json.dumps(expected, sort_keys=True)}\n"
            f"  found:    {json.dumps(echo, sort_keys=True)}"
        )
    # Samples may include timeout kwargs. Prose around them may not promise SLA.
    md = path.with_suffix(".md")
    if md.exists():
        prose = md.read_text()
        if GUARANTEE_RE.search(prose):
            errors.append(
                f"{md}: guarantee language is human-owned; "
                f"move it out of the generated page or cite a source"
            )
    return errors

def main() -> int:
    fixtures = load_fixtures()
    errors: list[str] = []
    for path in sorted(EXAMPLES.glob("create_*.*")):
        if path.suffix in {".py", ".js", ".ts", ".go", ".rb", ".java"}:
            errors.extend(check_example(path, fixtures))
    for line in errors:
        print(line, file=sys.stderr)
    print(f"checked {len(list(EXAMPLES.glob('create_*.*')))} examples, "
          f"{len(errors)} error(s)")
    return 1 if errors else 0

if __name__ == "__main__":
    sys.exit(main())

Run it locally:

pip install pyyaml
python tools/check_example_fixtures.py

A green run means the samples still confess the same request. It does not mean the production API matches the fixture. That is a different test, against a staging server or a recorded contract suite.

Where a model is allowed to work

Generation sits after the fixture is signed. The prompt may see the YAML. It may not propose edits to it. Ask only for one language at a time, and require the HTTP-CANONICAL line to be copied, not rewritten.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. If you use MonkeyCode's free model access — and, when you want the job off your laptop, the free server option — send the frozen fixture plus a language name. Do not send a blank “write docs for this endpoint” prompt. The output you keep is syntax. The object you merge first is still the YAML.

A prompt that respects the split looks like this:

You are filling SDK syntax for one frozen HTTP fixture.
Copy the HTTP-CANONICAL JSON exactly. Do not add, drop, or relocate fields.
Do not invent rate limits, SLAs, status codes, or auth schemes.
Language: Python 3, stdlib + requests.
Fixture follows:
<paste fixtures/create_invoice.yaml>

If the model returns a different path, the checker fails. That is the point. Reviewers then stop arguing about taste in variable names and start arguing about whether the fixture itself is wrong.

Decision table

Claim in the docs page	Draftable by a model?	Merge rule
`requests.post(...)` boilerplate	Yes	Must echo fixture
Path `/v1/invoices`	No	Fixture only
`customer_id` in body vs query	No	Fixture only
`Idempotency-Key` required	No	Human signs fixture
“Timeout of 30s is guaranteed”	No	Block; needs a measured source
“This client retries twice”	Yes, if labeled client-side	Must not imply server behavior
Example `cus_123`	Yes	Keep clearly fake
“We never return 409 after commit”	No	Human-owned failure contract

Print the table in the docs repo README. Authors stop treating “the model wrote it” as a review status.

What this does not prove

The checker is a string gate. It will not parse Python AST and reconstruct the real requests.post call unless you add that later. A sample can echo the right canonical block and still execute the wrong code. Pair this with at least one compiled or executed example per language when the client library makes that cheap.

It also will not save a wrong fixture. If a human freezes amount_cents as a string, every language sample will be consistently wrong. Fixture review is still engineering review.

YAML here is a convenience. JSON fixtures are easier to diff in some repos. The ownership rule does not care.

Do not use this approach when:

There is no canonical HTTP contract, only a GUI walkthrough
Examples are intentionally non-normative (blog posts, conference talks)
A single language is hand-maintained and never regenerated
The “sample” is an SDK that hides the wire format so deeply that an HTTP echo would lie

Those cases need a different artifact: a recorded traffic file, a Pact-style contract, or a human-written tutorial with no generated samples at all.

Keep the regen loop small

A practical sequence:

Human edits fixtures/create_invoice.yaml in the same PR as the API change.
Model drafts or refreshes one sample per language, canonical block included.
python tools/check_example_fixtures.py runs in CI.
Reviewers read the fixture diff first, sample diffs second.
Guarantee sentences stay in a human-owned markdown file that regeneration cannot overwrite.

The cost of skipping step 1 is not a messy paragraph. It is two support snippets that disagree about where customer_id lives. Syntax is cheap to regenerate. Identity is not.

No Source, No Number: A Provenance Gate for AI-Drafted API Docs

Morgan Sun — Mon, 21 Sep 2026 13:03:27 +0000

A generated Quick Start page told integrators to wait 30 seconds before retrying a 429. The edge gateway used a five-second token bucket. Integrators followed the page, not the gateway. They collected a longer 429 streak, opened a Sev-2, and pasted the docs URL into the ticket.

The API had not changed that week. The docs had. A model restated the OpenAPI paths correctly, then filled the gaps with plausible intervals. Plausible is not an SLA.

That pattern is a docs bug class, not a model-quality complaint. Schema restatement is cheap to generate. Numeric and temporal claims are not restatements. They are promises.

The sentence that should have failed CI

Here is a compressed version of the bad paragraph. It looks finished. It even uses the right status code.

If the API returns `429 Too Many Requests`, wait 30 seconds and retry
the same idempotency key. The gateway guarantees a fresh token after
that interval.

Three claims sit in those two sentences:

The wait interval is 30 seconds.
The same idempotency key is the recovery method.
The gateway guarantees a fresh token on that clock.

OpenAPI can confirm that 429 exists. It rarely confirms the bucket size. It almost never confirms a guarantee. A draft that invents the number will still read like a reference page.

The fix is not "write better prompts." The fix is a source map. Every numeric or temporal sentence must point at a file a human already owns. If the pointer is missing, CI cuts the sentence.

Two classes of sentences

Split the page before anyone asks a model to write it.

Restatable. These sentences restate structure that already exists in machine-readable form. A model may draft them. A checker still needs a pointer, but the pointer may be the spec.

Path, method, and operationId
Required vs optional fields
Enum members listed in the schema
Status codes declared on the operation
Example JSON that validates against the schema

Must-cite. These sentences assert behavior in time, quantity, or commitment. A model may phrase them only after a human-owned claims file already contains the value. If the value is absent, the sentence does not ship.

Timeouts, retries, backoff, and jitter
Rate limits, quotas, burst sizes, and token buckets
p95 / p99 language, SLAs, and "usually" intervals
Deprecation and sunset dates
Idempotency semantics that are not in the spec
Any use of guarantee, always, never, or we promise

The test is mechanical. If a sentence still makes sense after you delete every digit and every time unit, it might be restatable. If the sentence collapses, it was a promise.

A source-map convention

Keep the map in the Markdown, next to the sentence, not in a side spreadsheet. HTML comments survive most static-site pipelines and are easy to grep.

<!-- src: openapi.yaml#/paths/~1widgets/get/responses/429 -->
A `429 Too Many Requests` response means the client exceeded the
declared rate limit for this operation.

<!-- src: ops-claims.yaml#retry_after_seconds -->
Retry after `5` seconds. Send the original `Idempotency-Key`.

<!-- human-owned:start -->
Support will not extend a burst window during an incident.
<!-- human-owned:end -->

Rules for the convention:

One src comment owns the following paragraph, not the whole file.
Allowed files are an allowlist: openapi.yaml and ops-claims.yaml in this workflow.
Keys after # must exist in the claims file when the source is ops-claims.yaml.
human-owned blocks are opaque to the model. Regeneration must not rewrite them.
Bare URLs, Slack messages, and "as discussed" are not sources.

This is a proposed convention, not a standard. Label it that way in the repo so later editors do not treat the comments as prose.

The human-owned claims file

Put numbers where on-call already looks. Do not hide them inside a prompt.

# ops-claims.yaml
# Human-owned. Models may read. Models may not edit.
claims:
  retry_after_seconds: 5
  rate_limit_per_minute: 120
  burst_size: 20
  request_timeout_ms: 2500
  idempotency_key_ttl_seconds: 86400
  deprecation:
    widgets_v1_sunset: "2026-12-01"
forbidden_in_draft:
  - guarantee
  - guaranteed
  - we promise
  - SLA
  - five nines

The file is small on purpose. If a number cannot fit here, it does not belong in a getting-started page. Product, SRE, and docs should change this file in a normal pull request. The model is a reader of the merge, not an author of the values.

Decision table

Sentence type	Model may draft wording?	Required `src`	Human must own the value
Field description copied from schema	Yes	`openapi.yaml`	No
Example body that validates	Yes	`openapi.yaml`	Review shape
HTTP status listed on the operation	Yes	`openapi.yaml`	No
Retry interval, timeout, burst, quota	Wording only	`ops-claims.yaml#key`	Yes
Sunset or deprecation date	Wording only	`ops-claims.yaml#key`	Yes
Idempotency TTL	Wording only	`ops-claims.yaml#key`	Yes
"We guarantee…" / SLA / five nines	No	n/a	Yes, inside `human-owned`
Unsourced digit or time unit	No	fail closed	Yes

If a cell says "fail closed," the checker exits non-zero. There is no warning mode for numbers. Warnings become documentation debt.

The checker

The script below is a proposed CI gate. It is not a benchmark and it has not been tuned against a private corpus. Run it on the Markdown you are about to publish.

#!/usr/bin/env python3
"""Fail when drafted API docs assert numbers without a source pointer."""

from __future__ import annotations

import pathlib
import re
import sys

import yaml

ROOT = pathlib.Path(__file__).resolve().parents[1]
DOCS = ROOT / "docs"
CLAIMS_PATH = ROOT / "ops-claims.yaml"
ALLOWED_FILES = {"openapi.yaml", "openapi.yml", "ops-claims.yaml"}

SRC_RE = re.compile(r"<!--\s*src:\s*([^\s#]+)(?:#([^\s]+))?\s*-->")
OWN_START = "<!-- human-owned:start -->"
OWN_END = "<!-- human-owned:end -->"
NUMBERISH = re.compile(
    r"(\b\d+(\.\d+)?\s*(ms|s|sec|secs|seconds|m|min|minutes|h|hours|days)\b)"
    r"|(\b(p95|p99|sla)\b)"
    r"|(\b(retry|timeout|rate[- ]?limit|quota|burst|idempoten)\w*\b)",
    re.I,
)
GUARANTEE = re.compile(r"\b(guarantee[ds]?|we promise|five nines)\b", re.I)


def load_claims() -> dict:
    data = yaml.safe_load(CLAIMS_PATH.read_text()) or {}
    return data.get("claims") or {}


def claim_exists(claims: dict, pointer: str) -> bool:
    cur: object = claims
    for part in pointer.split("."):
        if not isinstance(cur, dict) or part not in cur:
            return False
        cur = cur[part]
    return True


def paragraphs(text: str) -> list[tuple[str, bool]]:
    owned = False
    chunks: list[tuple[str, bool]] = []
    buf: list[str] = []

    def flush() -> None:
        if buf:
            chunks.append(("\n".join(buf).strip(), owned))
            buf.clear()

    for line in text.splitlines():
        if line.strip() == OWN_START:
            flush()
            owned = True
            continue
        if line.strip() == OWN_END:
            flush()
            owned = False
            continue
        if line.strip() == "":
            flush()
            continue
        buf.append(line)
    flush()
    return chunks


def check_file(path: pathlib.Path, claims: dict) -> list[str]:
    errors: list[str] = []
    pending_src: tuple[str, str | None] | None = None
    text = path.read_text()

    # Walk line-level only to bind src comments to the next paragraph.
    owned = False
    para_lines: list[str] = []
    start_line = 1

    def bind_and_reset(end_line: int) -> None:
        nonlocal pending_src, para_lines, start_line
        body = "\n".join(para_lines).strip()
        para_lines.clear()
        if not body:
            pending_src = None
            return
        if owned:
            pending_src = None
            start_line = end_line
            return
        if GUARANTEE.search(body):
            errors.append(
                f"{path}:{start_line}: guarantee language outside human-owned block"
            )
        if NUMBERISH.search(body):
            if pending_src is None:
                errors.append(
                    f"{path}:{start_line}: numeric/temporal claim has no src comment"
                )
            else:
                file_name, pointer = pending_src
                if file_name not in ALLOWED_FILES:
                    errors.append(
                        f"{path}:{start_line}: src file {file_name} is not allowlisted"
                    )
                elif file_name.startswith("ops-claims") and (
                    not pointer or not claim_exists(claims, pointer)
                ):
                    errors.append(
                        f"{path}:{start_line}: ops-claims pointer {pointer!r} missing"
                    )
        pending_src = None
        start_line = end_line

    for i, line in enumerate(text.splitlines(), start=1):
        if line.strip() == OWN_START:
            bind_and_reset(i)
            owned = True
            continue
        if line.strip() == OWN_END:
            bind_and_reset(i)
            owned = False
            continue
        src = SRC_RE.fullmatch(line.strip())
        if src:
            bind_and_reset(i)
            pending_src = (src.group(1), src.group(2))
            start_line = i
            continue
        if line.strip() == "":
            bind_and_reset(i)
            continue
        if not para_lines:
            start_line = i
        para_lines.append(line)
    bind_and_reset(len(text.splitlines()) + 1)
    return errors


def main() -> int:
    if not CLAIMS_PATH.exists():
        print(f"missing {CLAIMS_PATH}", file=sys.stderr)
        return 2
    claims = load_claims()
    errors: list[str] = []
    for path in sorted(DOCS.rglob("*.md")):
        errors.extend(check_file(path, claims))
    for err in errors:
        print(err, file=sys.stderr)
    return 1 if errors else 0


if __name__ == "__main__":
    raise SystemExit(main())

Install the one runtime dependency the script needs, then run it the same way CI will.

pip install pyyaml
python tools/provenance_check.py
echo $?

A non-zero exit is the product. Do not wrap it in a bot comment that reviewers can ignore. Docs PRs that only regenerate Markdown should still fail this gate.

Minimal CI hook

# .github/workflows/docs-provenance.yml
name: docs-provenance
on:
  pull_request:
    paths:
      - "docs/**"
      - "ops-claims.yaml"
      - "openapi.yaml"
jobs:
  check:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: "3.12"
      - run: pip install pyyaml
      - run: python tools/provenance_check.py

Path filters matter. If a claims value changes and the Markdown still cites the old number, that is a different bug. Add a second check later that the cited key's value actually appears in the paragraph. The script above does not do that yet. Call that gap out in the PR template so nobody assumes it does.

What the model is allowed to draft

Feed the model three inputs only:

The OpenAPI file, or a single operation slice of it.
ops-claims.yaml, read-only.
A page skeleton with empty restatable sections and already-filled human-owned blocks.

A proposed prompt, not a magic spell:

Draft the restatable sections of this API page.
Copy field names, types, and status codes from OpenAPI.
For any retry, timeout, rate limit, date, or idempotency TTL,
insert a paragraph only if ops-claims.yaml has the key.
Prefix that paragraph with <!-- src: ops-claims.yaml#key -->.
Do not invent digits. Do not write guarantee language.
Do not edit text between human-owned:start and human-owned:end.

The output is still a draft. A human accepts or deletes each must-cite paragraph. Regeneration may overwrite restatable sections. It must not overwrite human-owned blocks or ops-claims.yaml.

If you need a machine to run that draft pass, keep the machine off the claims file. Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode's free model access and free server option can host the draft pass against those three inputs. The provenance checker does not depend on that host. Remove the draft step and the gate still works.

A worked fail, then a pass

Failing page, docs/widgets.md:

# Widgets

Create a widget, then poll for completion.

On `429`, wait 30 seconds and retry. The platform guarantees a fresh
token after that wait.

Expected checker output:

docs/widgets.md:5: numeric/temporal claim has no src comment
docs/widgets.md:5: guarantee language outside human-owned block

Passing page:

# Widgets

<!-- src: openapi.yaml#/paths/~1widgets/post -->
`POST /widgets` accepts a JSON body and returns `201` or `429`.

<!-- src: ops-claims.yaml#retry_after_seconds -->
On `429`, wait `5` seconds and resend the same `Idempotency-Key`.

<!-- src: ops-claims.yaml#idempotency_key_ttl_seconds -->
Keys remain valid for `86400` seconds.

<!-- human-owned:start -->
Incident response will not raise burst size for a single tenant.
<!-- human-owned:end -->

The digits in the passing page are copies. That is the point. The model did not choose 5 or 86400. The claims file did.

Limitations

The gate is narrow. Treat these as known holes, not future slogans.

OpenAPI pointers are not resolved. A comment may cite a missing path and still pass if the source file name is allowlisted.
The checker does not confirm that the paragraph's digit equals the YAML value. A stale 30 can sit next to a retry_after_seconds: 5 pointer.
Time units in words without digits ("shortly", "eventually", "a moment") slip through NUMBERISH.
Tables, fenced code, and HTML <table> cells are not parsed as first-class claims.
Multi-paragraph lists after one src comment will over-bind. Keep claims in short paragraphs.
The script does not detect screenshots, mermaid timings, or Postman collections.
Forbidden words inside human-owned blocks are not scanned. That is intentional. It is also an abuse hatch.

Add value equality next if your corpus is small. Add OpenAPI pointer resolution if your spec is the only schema. Do not add a sentiment classifier. The bug is unsourced numbers, not tone.

Who should not use this

Skip the workflow if any of the following is true.

The API has no OpenAPI file and no other machine-readable contract. There is nothing restatable to draft.
Rate limits are tenant-specific and must not appear in public Markdown at all. Put them in an authenticated console, not a source map.
Legal or compliance text is generated. This gate is not a contract review.
The team cannot protect ops-claims.yaml from model writes. If the model can edit the claims, the map is theater.
Docs are a single giant HTML export with no paragraph structure. The comment convention will not survive.

For those cases, freeze the page. A frozen wrong number is still wrong, but it is attributable.

Ship the gate first

Start with one page that already caused a ticket. Add ops-claims.yaml with only the numbers on that page. Wrap the promises in human-owned blocks. Run the checker on a branch that still contains the invented interval. Confirm it fails. Then replace the interval with a cited copy of the claims file. Confirm it passes.

The model is optional after that. The source map is not. A restated schema can be regenerated. A number without a source should not leave the branch.

Envelope vs Clock: A Split Workflow for Async API Docs

Morgan Sun — Sun, 20 Sep 2026 11:13:27 +0000

A payments platform exposes POST /v1/exports. The handler returns 202 Accepted, a job_id, and a status_url. A doc generator then writes the page in one pass. The JSON envelope matches the OpenAPI file. The same page also says “poll every five seconds” and “files remain available for 30 days.”

Neither number lives in the schema. Neither number is asserted by a test. Support treats both sentences as contract. Clients poll at 5s. The status endpoint becomes the incident. The docs page still looks finished.

That is the failure this workflow isolates. Shape is extractable. Time is a promise.

Why async docs fail closed-looking

OpenAPI is strong on envelopes. It names fields, enums, and required keys. It is weak on clocks. Polling cadence, client timeouts, result retention, and webhook retry backoffs are operational promises. They are not properties of a JSON Schema object.

A model asked to “finish the page” will fill the silence. It prefers round numbers. It prefers advice that sounds like a runbook. Reviewers skim because the examples look real. The invented clock ships.

The industry pattern is familiar in 2026: generated text makes it cheap to appear to have done the SRE writing. The missing work is not prose. The missing work is who attests duration, retention, and retry.

Two documents, one URL

Keep a single public page if you must. Internally, treat it as two documents.

Envelope (a model may draft)

Request and response schemas
Required versus optional fields
Example payloads copied from fixtures
Job state names (queued, running, succeeded, failed)
Error object shape
Idempotency header names (not retry windows)

Clock (a human must own)

Recommended poll interval
Maximum poll duration / client timeout
Result retention and deletion
Webhook retry schedule and signing clock skew
Any “usually completes in…” sentence
Any SLA-like or billing-like duration

If a sentence contains a duration, a schedule, or a retention claim, it is clock. It does not belong in an unattended generation lane.

Artifact 1: an ownership manifest

Label this as a proposed layout. It is not a vendor format. Put it next to the OpenAPI file.

# docs/async-jobs.ownership.yaml
page: /docs/exports
source_schema: openapi/exports.yaml
fixture_dir: fixtures/exports/

envelope:
  owner: docs-generator
  may_regenerate: true
  sections:
    - heading: Submit an export
      proof: openapi/exports.yaml#/paths/~1v1~1exports/post
    - heading: Job status object
      proof: openapi/exports.yaml#/components/schemas/ExportJob
    - heading: Example status payload
      proof: fixtures/exports/status.succeeded.json

clock:
  owner: platform-oncall
  may_regenerate: false
  sections:
    - heading: Polling
      attested_by: sre-exports
      attested_on: 2026-09-18
    - heading: Result retention
      attested_by: sre-exports
      attested_on: 2026-09-18
    - heading: Webhook delivery
      attested_by: sre-exports
      attested_on: 2026-09-18

forbidden_in_envelope:
  - duration_units: [ms, s, sec, second, minute, hour, day, week]
  - phrases: ["poll every", "retry after", "kept for", "retained for", "usually completes"]

The manifest does three jobs. It names a proof artifact for every envelope heading. It names a human owner for every clock heading. It lists lexical patterns that must not appear in regenerated prose.

No proof, no regenerate. No attestation date, no publish.

Artifact 2: a time-claim checker

The checker is ordinary Python. Run it in CI against the rendered Markdown, not against the model transcript. Models drift. Files do not.

#!/usr/bin/env python3
"""Fail CI when clock claims leak into envelope sections. Proposed example."""
from __future__ import annotations

import re
import sys
from pathlib import Path

import yaml

DURATION = re.compile(
    r"\b(\d+(?:\.\d+)?)\s*(ms|milliseconds?|s|secs?|seconds?|"
    r"m|mins?|minutes?|h|hrs?|hours?|d|days?|w|weeks?)\b",
    re.I,
)
CLOCK_PHRASE = re.compile(
    r"poll every|retry after|kept for|retained for|usually completes|within \d",
    re.I,
)
HEADING = re.compile(r"^(#{2,3})\s+(.+?)\s*$", re.M)


def load_manifest(path: Path) -> dict:
    return yaml.safe_load(path.read_text())


def split_sections(markdown: str) -> list[tuple[str, str]]:
    parts = HEADING.split(markdown)
    sections = []
    # parts: preamble, hashes, title, body, hashes, title, body, ...
    i = 1
    while i + 2 < len(parts):
        title = parts[i + 1].strip()
        body = parts[i + 2]
        sections.append((title, body))
        i += 3
    return sections


def envelope_titles(manifest: dict) -> set[str]:
    return {row["heading"] for row in manifest["envelope"]["sections"]}


def clock_titles(manifest: dict) -> set[str]:
    return {row["heading"] for row in manifest["clock"]["sections"]}


def main(md_path: Path, manifest_path: Path) -> int:
    manifest = load_manifest(manifest_path)
    md = md_path.read_text()
    env = envelope_titles(manifest)
    clock = clock_titles(manifest)
    failures = []

    seen = {title for title, _ in split_sections(md)}
    for title in env | clock:
        if title not in seen:
            failures.append(f"missing heading: {title}")

    for title, body in split_sections(md):
        if title not in env:
            continue
        if DURATION.search(body) or CLOCK_PHRASE.search(body):
            failures.append(f"clock claim in envelope section '{title}'")

    for row in manifest["clock"]["sections"]:
        if not row.get("attested_by") or not row.get("attested_on"):
            failures.append(f"unattested clock section '{row['heading']}'")

    for row in failures:
        print(row, file=sys.stderr)
    return 1 if failures else 0


if __name__ == "__main__":
    sys.exit(main(Path(sys.argv[1]), Path(sys.argv[2])))

Install and run:

pip install pyyaml
python check_async_docs.py docs/exports.md docs/async-jobs.ownership.yaml

A non-zero exit is the contract. Do not parse model confidence. Do not ask the model whether it invented a number. The file either contains a duration in an envelope heading or it does not.

Envelope Markdown the checker will accept

## Submit an export

`POST /v1/exports` returns `202 Accepted` with `job_id` and `status_url`.
Both fields are required. See the schema component `ExportAccepted`.

## Job status object

`GET /v1/exports/{job_id}` returns `state`, `error`, and `result_href`.
`state` is one of `queued`, `running`, `succeeded`, `failed`.
`result_href` is present only when `state` is `succeeded`.

## Example status payload

json
{
"job_id": "exp_08c1",
"state": "succeeded",
"result_href": "/v1/exports/exp_08c1/content"
}


## Polling

<!-- clock: attested_by=sre-exports attested_on=2026-09-18 -->
Poll `status_url` no more than once every 15 seconds.
Stop after 15 minutes and open a support ticket with `job_id`.

## Result retention

<!-- clock: attested_by=sre-exports attested_on=2026-09-18 -->
Successful export files remain downloadable for 72 hours.
After that, `GET` on `result_href` returns `410 Gone`.

markdown

The first three headings can be regenerated from schema plus fixtures. The last two cannot. If a regenerate step rewrites Polling, CI fails on the missing attestation, even when the prose looks nicer.

Drafting protocol (envelope only)

Treat generation as a bounded transform, not as “write the page.”

Extract the schema slice named in proof.
Attach one fixture file, not a conversation history.
Instruct the model to describe fields that exist in those inputs.
Instruct it to omit intervals, retention, and “typical” latency.
Write output only under envelope headings.
Run check_async_docs.py.
Leave clock headings untouched unless attested_by is still on the hook.

A prompt that says “don’t invent numbers” is not a control. The checker is the control. The prompt is a courtesy to reduce failed CI runs.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. Free model access is enough to draft envelope headings from schema text. A free server option is enough to run the checker on every docs PR. Neither replaces the attestation line on a clock section. Do not feed clock headings to a model and then “see if it looks right.”

Decision table

Claim in the page	Source of truth	May a model draft it?	Ship gate
Field names and types	OpenAPI / JSON Schema	Yes	Schema diff
Example JSON	Checked-in fixture	Yes, copy only	Byte-compare to fixture
State enum values	Schema `enum`	Yes	Enum set equality
Poll interval	On-call runbook	No	Manifest attestation
Client timeout	On-call runbook	No	Manifest attestation
Result retention	Data-retention policy	No	Manifest attestation
Webhook retry backoff	Delivery config	No	Manifest attestation
“Usually takes ~N minutes”	None, unless measured	No	Ban in envelope lane

Copy the table into the repo. When a writer argues about a sentence, classify the row first. Arguments about tone come after classification.

What this does not catch

The checker is lexical. It will miss “poll frequently” with no unit. It will miss “we keep files around for a while.” It will miss a duration written as PT72H if you never taught it ISO-8601. Extend the regex only with patterns you have seen in your own corpus.

It also does not prove that an attested number is correct. A human can attest “poll every 15 seconds” on a fleet that 429s at that rate. Attestation is ownership, not load-test evidence. Pair this workflow with a status-endpoint budget test if polling advice is customer-facing.

Webhooks need a second proof artifact: the signing secret rotation note is clock-adjacent and legal-adjacent. Keep it out of envelope generation even when the payload schema is draftable.

Who should not use this

Skip the split if your API is synchronous and your docs never mention time. A 200 with a JSON body and no job object does not need a clock ledger.

Skip it if you have no owner who will attest. An empty attested_by field is a false gate. The checker will pass the moment someone types a name. Names without on-call rotation are decoration.

Skip it if legal or compliance text is mixed into the same headings as schema prose. Split those pages first. A model that is allowed to rewrite a retention sentence is a policy incident, not a docs incident.

A short working sequence

# 1. Freeze proofs
ls openapi/exports.yaml fixtures/exports/

# 2. Draft envelope headings only (local script or any model runner)
python draft_envelope.py --schema openapi/exports.yaml --out /tmp/envelope.md

# 3. Splice without touching clock headings
python splice_docs.py --base docs/exports.md --envelope /tmp/envelope.md

# 4. Gate
python check_async_docs.py docs/exports.md docs/async-jobs.ownership.yaml

The splice step is the easy one to get wrong. If your tool reflows the whole Markdown file, clock comments move or vanish. Prefer heading-bounded replacement. Replace by title, not by byte offset.

Envelope prose can be regenerated whenever the schema changes. Clock prose changes when an owner accepts a new promise. Those two frequencies are not the same. Docs pipelines that ignore the difference will keep publishing complete-looking pages with unearned clocks.

If you want a runner for the checker and a free model lane that is limited to envelope inputs, MonkeyCode is one place to try that split. The ownership manifest still has to live in your repo.

Unregistered Scopes Do Not Ship: A Registry Diff for AI-Drafted API Docs

Morgan Sun — Fri, 18 Sep 2026 07:28:12 +0000

Here is a failure mode that shows up in generated API guides. A partner integration failed on a Friday deploy. The public guide listed invoices:write:all. The identity service had never minted that scope.

The paragraph was fluent. The grant was imaginary. Completeness was the hazard, not spelling.

A sentence that names a scope, a rate, or a timeout is a runtime claim. Models copy the shape of real security schemes and then extend them. Fluent output is easy to mistake for a reviewed authorization model. Documentation is one of the places that mistake reaches customers.

Split the page before you split the prompt

Treat an API guide as two classes of text. One class can be regenerated when handlers change. The other class is a registry. If the model may extend the registry, it will.

What the model may draft

Endpoint purpose restated from an OpenAPI summary or a handler comment
Parameter narrative that does not introduce new identifiers
Happy-path sequences that only mention registered scopes
Cross-links to conceptual pages that already exist in the repo

What a human must own

Scope identifiers and the operations they authorize
Default-deny statements
Rate limits, quotas, pagination defaults, and timeouts
Token lifetime, refresh rules, and audience values
Public vs partner vs internal classification
Deprecation or breakage of a scope

The owned class is small. It is also the class that pages on-call when it is wrong.

A registry the generator may not extend

Keep a file the model can read and must not edit. Humans change it through the same review path as IAM code. Numbers without a source path are rumors. Clients hard-code rumours.

# auth-registry.yaml
# Owner: identity-platform. Regenerators may read, never append.
scopes:
  - id: orders:read
    grants: ["GET /v1/orders", "GET /v1/orders/{id}"]
  - id: orders:write
    grants: ["POST /v1/orders", "PATCH /v1/orders/{id}"]
  - id: orders:refund
    grants: ["POST /v1/orders/{id}/refunds"]

numeric_contracts:
  - key: rate_limit.merchant_default
    value: 100
    unit: requests_per_minute
    source: infra/gateway/ratelimit.yaml
  - key: timeout.orders_get
    value: 2.5
    unit: seconds
    source: services/orders/config.yaml
  - key: page_size.default
    value: 50
    unit: items
    source: libs/api/pagination.go

forbidden_claims:
  - "unlimited"
  - "no rate limit"
  - "full admin access"
  - "by default all users"

Cite a number by key, not by vibes. A passing sentence names a registered scope and points at a contract:

`GET /v1/orders` requires `orders:read`.

The merchant default rate limit is 100 requests per minute
[contract:rate_limit.merchant_default].

A failing sentence is the one that reached the partner:

Use `invoices:write:all` for bulk exports. There is no rate limit on this route.

Both are easy for a model to emit. Both should be impossible to merge.

Proposed workflow

This is a proposed pipeline, not a measured production study. The order matters more than the tooling.

Extract paths and security schemes from openapi.yaml.
Render a prompt that includes the registry as an allow-list, not as a suggestion.
Ask the model only for draftable sections, one operation at a time.
Concatenate drafts under headings humans already registered.
Run a lexical checker in CI. Fail the build on unknown scopes or uncited numbers.
Require a human approval on any registry diff, same as an IAM change.

Step 3 does not design authorization. It narrates an operation using names you already supplied. That is the only job a general model should have on this page.

Prompt shape (proposal)

Do not paste live tokens into the prompt. Pass identifiers only. The useful failure mode is a short refusal, not a confident new scope.

You draft API guide prose for one operation.
You may use only these scope ids: {scope_ids}
You may cite only these numeric contracts by key: {contract_keys}
If a permission or number is not in those lists, write UNREGISTERED and stop.
Do not invent scopes, roles, rate limits, timeouts, or page sizes.
Do not wrap the prose in JSON fences.

Operation:
{method} {path}
OpenAPI summary: {summary}
Registered scopes for this operation: {op_scopes}

UNREGISTERED is cheaper than a weekend of partner support. Keep the allow-list in git so two regenerations cannot drift apart.

A checker you can run locally

The script below is a proposed CI gate. It is lexical. It will not prove that your gateway enforces the matrix. It will prove that the markdown did not introduce identifiers the registry does not know.

#!/usr/bin/env python3
"""authdoc_check.py — fail if API markdown invents scopes or numbers."""

from __future__ import annotations

import argparse
import re
import sys
from pathlib import Path

import yaml

SCOPE_RE = re.compile(r"\b([a-z][a-z0-9_]{1,20}:[a-z][a-z0-9_:]{1,40})\b")
NUMBER_HINT_RE = re.compile(
    r"(?i)(rate\s*limit|quota|timeout|page\s*size|ttl|throttle|"
    r"requests?\s*per|retry-after)\D{0,40}(\d+(?:\.\d+)?)"
)
CITATION_RE = re.compile(r"\[contract:([a-z0-9_.-]+)\]")


def load_registry(path: Path) -> dict:
    data = yaml.safe_load(path.read_text())
    scopes = {row["id"] for row in data.get("scopes", [])}
    contracts = {row["key"]: row for row in data.get("numeric_contracts", [])}
    forbidden = [s.lower() for s in data.get("forbidden_claims", [])]
    return {"scopes": scopes, "contracts": contracts, "forbidden": forbidden}


def iter_markdown(root: Path):
    for p in sorted(root.rglob("*.md")):
        yield p, p.read_text(encoding="utf-8")


def check_file(path: Path, text: str, registry: dict) -> list[str]:
    errors = []
    allowed = registry["scopes"]
    for m in SCOPE_RE.finditer(text):
        token = m.group(1)
        if token not in allowed:
            line = text[: m.start()].count("\n") + 1
            errors.append(f"{path}:{line}: unregistered scope `{token}`")

    for m in NUMBER_HINT_RE.finditer(text):
        snippet = m.group(0)
        window_start = max(0, m.start() - 80)
        window = text[window_start : m.end() + 80]
        if not CITATION_RE.search(window):
            line = text[: m.start()].count("\n") + 1
            errors.append(
                f"{path}:{line}: numeric claim without [contract:key] "
                f"citation: {snippet!r}"
            )
        else:
            for key in CITATION_RE.findall(window):
                if key not in registry["contracts"]:
                    line = text[: m.start()].count("\n") + 1
                    errors.append(
                        f"{path}:{line}: unknown contract key `{key}`"
                    )

    lower = text.lower()
    for phrase in registry["forbidden"]:
        if phrase in lower:
            errors.append(f"{path}: forbidden claim {phrase!r}")
    return errors


def main() -> int:
    parser = argparse.ArgumentParser()
    parser.add_argument("--registry", type=Path, required=True)
    parser.add_argument("--docs", type=Path, required=True)
    args = parser.parse_args()

    registry = load_registry(args.registry)
    errors: list[str] = []
    for path, text in iter_markdown(args.docs):
        errors.extend(check_file(path, text, registry))

    if errors:
        print("authdoc_check: FAILED")
        print("\n".join(errors))
        return 1
    print("authdoc_check: ok")
    return 0


if __name__ == "__main__":
    sys.exit(main())

Install and run it against the guide tree:

pip install pyyaml
python authdoc_check.py --registry auth-registry.yaml --docs ./docs/api
echo $?   # 1 if the draft invented a scope or an uncited number

Proposed test plan

These fixtures are enough to keep the gate honest. They are not a substitute for gateway tests.

good.md uses only registered scopes and [contract:] citations. Expect exit 0.
bad-scope.md contains invoices:write:all. Expect exit 1 and the token in the log.
bad-number.md contains timeout of 30 seconds with no citation. Expect exit 1.
forbidden.md contains unlimited. Expect exit 1.
Change auth-registry.yaml in a separate PR from prose-only edits. Review that PR as IAM.

Wire the same command to CI

Drafts should be cheap to regenerate. Registry rows should not be. Path filters keep the two reviews apart.

# .github/workflows/authdocs.yml
name: authdocs
on:
  pull_request:
    paths:
      - "docs/api/**"
      - "auth-registry.yaml"
      - "openapi.yaml"
jobs:
  registry-diff:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: "3.12"
      - run: pip install pyyaml
      - run: python authdoc_check.py --registry auth-registry.yaml --docs ./docs/api

If auth-registry.yaml changes, identity owners review it. If only docs/api/** changes, the checker still runs. That split is the whole method.

Where a free model and a free server belong

The draft step is batchable. One operation per call. No tool use. No production credentials in the prompt.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

MonkeyCode provides free model access and a free server option, which is enough for this loop: registry in git, prompt per operation, markdown out, checker in CI. Park the generation job on the free server if the batch should not sit on a laptop. Keep the registry and the OpenAPI file in your own repo either way. The model is not the owner of record for a scope name or a number.

Do not send customer tokens, private keys, or live IAM dumps to any hosted model. Pass allow-listed identifiers only.

Decision table

Claim in the doc	May a model draft it?	Merge gate
Restated OpenAPI summary	Yes	Optional human skim
New scope identifier	No	Checker fail
Registered scope on the wrong path	No	Human review of `grants`
Rate / timeout / page size with `[contract:]`	Yes, copied	Checker + source still in registry
Rate / timeout / page size as free prose	No	Checker fail
"Unlimited" or "admin by default"	No	Forbidden-claim fail
Deprecating a scope	No	Registry PR + identity owners

Limitations

The checker does not execute requests. It does not confirm that the gateway, the token issuer, and the service agree. Homophones are caught: order:read versus orders:read. Semantic mistakes are not: orders:read on a delete handler still looks lexical-clean.

Numbers cited to a stale source path will pass until a human updates the registry. Freshness is an ownership problem. Sampling more model output will not fix a file nobody owns.

Who should not use this approach:

Teams with no IAM owner and no OpenAPI file. There is nothing to freeze.
Public-policy or compliance documents that require counsel. A regex is not counsel.
Pages that embed real credentials, customer ids, or internal hostnames in examples.
Organizations that treat generated pages as the system of record for authorization.

If you skip the registry, you are not saving time. You are publishing an identity product the identity team did not ship.

Fluent API docs are not evidence that the permission model was reviewed. The review is the registry diff. Everything else can be regenerated. A free model is sufficient for narration. It is not sufficient for naming powers.

The Diff Is Draftable. The Support Window Is Not.

Morgan Sun — Thu, 17 Sep 2026 05:34:29 +0000

A services team cut a Friday release with a generated changelog that read clean. The bullets matched the merged commits. The tone was calm. The version bump in the same file said minor.

Monday, two mobile clients 404'd on a query parameter the server had stopped accepting. The model had summarized the diff. Nobody had classified the change. The support window was a sentence the generator invented because empty cells look unfinished.

That failure is not a writing problem. It is an ownership problem. Release docs mix two kinds of content that look similar in Markdown and are not similar in risk.

Two facts a model can use, and two it cannot

Git, OpenAPI diffs, and package manifests are evidence. A model can read them and draft user-facing summaries. SemVer, sunset dates, and “who still has to migrate” are contracts. Contracts need a named owner, not a fluent paragraph.

A model can usually see:

commit subjects and file paths in a tag range
added, removed, or renamed HTTP paths when you pass a schema diff
dependency name/version pairs from a lockfile
test names that started failing in CI

A model cannot honestly own:

whether the change is a major, minor, or patch for your compatibility promise
the last date an old field remains supported
who pages if a partner has not migrated
the rollback command that is safe in production, not the one that looks right in a README

Treat those as frozen fields. If the generator fills them, CI should fail closed.

Diff Lane vs Contract Lane

Split the release note into two lanes. Keep both in the same file so reviewers see them together. Do not let the draft lane write keys in the contract lane.

Diff Lane (model may draft)

changes[]: short, cited bullets tied to commit SHAs or schema paths
touched_paths[]: files or OpenAPI operations from the extractor, not from memory
dependency_deltas[]: old → new versions copied from the lockfile diff

Contract Lane (human must own)

semver_bump: major | minor | patch
breaking: boolean that must match the bump
support_until: an ISO date, or null only when breaking is false
migration_owner: a real handle in your org
rollback: a command or git tag that exists in the repo
signed_by: the reviewer who accepts the contract

The rule is simple. Summaries regenerate. Contracts persist across regenerations.

Worked example: a parameter removal

Label: proposed example, not a production incident dump.

Extractor output (facts only):

openapi_diff:
  removed: GET /v1/orders?include_archived
  added:   GET /v1/orders?archive_state=
commits:
  - 9f2c1aa  drop include_archived query flag
  - 41ab003  add archive_state enum filter
lockfile: no runtime dependency change

A model may draft this Diff Lane:

## Changes (draft)
- Order listing no longer reads `include_archived`.
  Source: commit 9f2c1aa, OpenAPI removed query `include_archived`.
- Order listing accepts `archive_state`.
  Source: commit 41ab003, OpenAPI added query `archive_state`.

A human must still fill this Contract Lane. Empty is better than guessed.

contract:
  semver_bump: major
  breaking: true
  support_until: "2026-12-31"
  migration_owner: "@orders-api"
  rollback: "git checkout 2026.09.12-orders && helm rollback orders 14"
  signed_by: "alex"

If semver_bump is minor while a query parameter disappeared, the file is false even if every bullet is cited. Fluency does not make a compatibility promise.

Artifact: a freeze checker you can run in CI

Keep contract fields in YAML so a script can reject invented dates and unsigned bumps. The checker below is runnable as-is against a fixture. It does not call a model.

# freeze_changelog.py
# Proposed CI gate: validate Contract Lane. Does not generate prose.
from __future__ import annotations

import re
import sys
from datetime import date
from pathlib import Path

try:
    import yaml
except ImportError:
    sys.stderr.write("pip install pyyaml\n")
    raise

BUMPS = {"major", "minor", "patch"}
HANDLE = re.compile(r"^@[A-Za-z0-9][A-Za-z0-9\-_/]*$")
ISO = re.compile(r"^\d{4}-\d{2}-\d{2}$")

FORBIDDEN_IN_CONTRACT = {
    "tbd",
    "todo",
    "soon",
    "n/a",
    "unknown",
    "the model",
    "as needed",
}


def load(path: Path) -> dict:
    data = yaml.safe_load(path.read_text(encoding="utf-8"))
    if not isinstance(data, dict) or "contract" not in data:
        raise SystemExit(f"{path}: missing top-level 'contract'")
    return data


def fail(errors: list[str]) -> None:
    if errors:
        sys.stderr.write("Contract Lane failed:\n")
        for item in errors:
            sys.stderr.write(f"  - {item}\n")
        raise SystemExit(1)


def check_contract(doc: dict) -> list[str]:
    c = doc["contract"] or {}
    errors: list[str] = []
    bump = c.get("semver_bump")
    breaking = c.get("breaking")
    until = c.get("support_until")
    owner = c.get("migration_owner")
    rollback = c.get("rollback")
    signed = c.get("signed_by")

    if bump not in BUMPS:
        errors.append("semver_bump must be major|minor|patch")
    if not isinstance(breaking, bool):
        errors.append("breaking must be a boolean")
    if bump == "major" and breaking is False:
        errors.append("major bump requires breaking: true")
    if bump in {"minor", "patch"} and breaking is True:
        errors.append("breaking: true requires major bump")

    if breaking:
        if not isinstance(until, str) or not ISO.match(until):
            errors.append("breaking changes need support_until as YYYY-MM-DD")
        else:
            try:
                parsed = date.fromisoformat(until)
            except ValueError:
                errors.append("support_until is not a real calendar date")
            else:
                if parsed <= date.today():
                    errors.append("support_until must be in the future")
    elif until not in (None, ""):
        errors.append("support_until must be null when breaking is false")

    for key in ("migration_owner", "rollback", "signed_by"):
        val = c.get(key)
        if not isinstance(val, str) or not val.strip():
            errors.append(f"{key} is empty")
        else:
            low = val.strip().lower()
            if any(tok in low for tok in FORBIDDEN_IN_CONTRACT):
                errors.append(f"{key} looks like a placeholder, not an owner")

    if isinstance(owner, str) and owner.strip() and not HANDLE.match(owner.strip()):
        errors.append("migration_owner must be an @handle")
    if not isinstance(signed, str) or signed.strip().startswith("@ai"):
        errors.append("signed_by must be a human reviewer")
    if isinstance(rollback, str) and "helm rollback" not in rollback and "git " not in rollback:
        errors.append("rollback must be an executable git or helm command")

    changes = doc.get("changes") or []
    if not changes:
        errors.append("Diff Lane is empty; extract commits before drafting")
    for item in changes:
        if not item.get("source"):
            errors.append("every drafted bullet needs a source SHA or schema path")
    return errors


def main(argv: list[str]) -> None:
    if len(argv) != 2:
        raise SystemExit("usage: python freeze_changelog.py path/to/release.yaml")
    path = Path(argv[1])
    fail(check_contract(load(path)))
    print(f"ok: {path}")


if __name__ == "__main__":
    main(sys.argv)

Fixture that should pass:

# releases/2026-09-17.yaml
changes:
  - text: Order listing no longer reads include_archived.
    source: "commit:9f2c1aa"
  - text: Order listing accepts archive_state.
    source: "openapi:GET /v1/orders?archive_state"
contract:
  semver_bump: major
  breaking: true
  support_until: "2026-12-31"
  migration_owner: "@orders-api"
  rollback: "git checkout 2026.09.12-orders && helm rollback orders 14"
  signed_by: "alex"

Run it:

pip install pyyaml
python freeze_changelog.py releases/2026-09-17.yaml

Flip semver_bump to minor and run it again. The gate should refuse the file. That refusal is the documentation quality bar. A green bullet list is not.

Prompt boundary: fill Diff Lane only

Label: unexecuted prompt template. Pass extractor JSON in. Ask for YAML out. Name the keys the model must not emit.

You draft Diff Lane bullets only.
Input is JSON with commits[], openapi_diff, and lockfile_diff.
Write YAML with a top-level `changes` list.
Each item needs `text` and `source` copied from the input.
Do not emit `contract`, versions, dates, owners, or rollback commands.
If a fact is missing from input, omit the bullet. Do not guess.

Pipe the result into the human-edited file with a merge that preserves contract: untouched. A three-line shell sketch:

# Proposed local workflow. Review before wiring to CI secrets.
python extract_release_facts.py v2026.09.12 HEAD > /tmp/facts.json
# model_draft.sh is your wrapper; it must write only `changes:`
model_draft.sh /tmp/facts.json > /tmp/diff_lane.yaml
python merge_lanes.py releases/2026-09-17.yaml /tmp/diff_lane.yaml
python freeze_changelog.py releases/2026-09-17.yaml

merge_lanes.py should replace changes and leave contract byte-stable. If the model prints a support_until, drop the file. Do not parse it into the frozen map.

Decision table

Signal in the extractor	Diff Lane may say	Contract Lane must set
Query/path/field removed	“X is removed”, with source	`breaking: true`, `major`, future `support_until`
Field added, old field still served	“Y is added”	usually `minor`, `breaking: false`, `support_until: null`
Docs or comments only	“docs updated”, if you even ship it	`patch`
Lockfile patch of a transitive dep	name and versions from the diff	bump only if you promise that tree
CI test renamed	nothing unless behavior changed	do not bump from a rename
Empty extractor	no bullets	do not ship a changelog

The table is the review checklist. If a reviewer argues with the table, change the table in a docs PR. Do not special-case a single release in prose.

Where a hosted draft step fits

The freeze checker belongs in your repository. The draft step is the disposable part: it needs a model that can turn extractor JSON into changes[] and nothing else.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. If you do not want to stand up a private inference box for that draft job, MonkeyCode currently offers free model access and a free server option you can point the Diff Lane wrapper at. Keep Contract Lane signing on your CI runners either way. A hosted draft does not make a sunset date true.

Limitations

This workflow does not classify breaking changes for you. It only refuses unsigned or contradictory contracts. OpenAPI diffs miss behavior changes that never touch the schema: timeout shifts, pagination defaults, error payload shapes. Git subjects lie when people write “refactor” on a removal. The forbidden-token list is brittle; a model can emit a plausible date that is still wrong. The checker cannot see whether @orders-api is still the on-call rotation. rollback is a string match, not a dry-run against the cluster.

Do not treat a green freeze_changelog.py as a substitute for a staging consumer test. The script answers “did a human fill the contract keys.” It does not answer “will last quarter’s client still parse the response.”

Who should not use this

Skip the split if you ship a single private binary with no external clients and no compatibility promise. A personal changelog can stay a commit list. Skip it if legal or customer comms already owns a separate advisory and your repo notes are internal-only — duplicate sunset dates will drift. Skip it if your version numbers are calendar stamps with no SemVer meaning; then rename the bump field instead of pretending major applies. Teams that cannot name a migration_owner should stop generating release prose until ownership exists. A fluent draft without an owner is how Friday’s “minor” becomes Monday’s 404.

What to freeze next

After changelogs, the same split applies to deprecation banners in API reference pages and to “supported runtimes” tables in READMEs. The model may refresh the list of tested versions from CI badges. A human still owns the last date an old runtime is supported.

Start with one release YAML, one extractor, and the checker in CI. Leave the draft model interchangeable. The contract keys are the product.

Register the Heading: What AI May Draft in Docs, and What a Human Must Own

Morgan Sun — Wed, 16 Sep 2026 03:52:28 +0000

A payments partner voided the same invoice three times. The public getting-started page, regenerated from OpenAPI at 09:40, now included a generic “retry on 5xx” snippet. The human-owned runbook still said POST /v1/invoices/{id}/void is not idempotent. Both texts lived in docs/payments.md. The model did not invent a new policy. It filled a heading that had no owner.

That is the failure this article treats. Generated prose is cheap. Ownership is not. If a heading can be rewritten on every schema refresh, it must be classified as draftable. If a heading is a promise—retry rules, deprecation windows, PII handling, escalation paths—it must have a human owner and a hash that CI can refuse to silently change.

The rest of this piece is a proposed, reproducible method: a heading-level ownership registry, a stdlib checker, a slicer that keeps owned text out of the model prompt, and a decision table for what belongs on each side.

The incident, reduced to a file

Composite, but typical. One Markdown file. Two jobs. A generator rewrites “Getting started” from the spec. A human is supposed to own “Voiding an invoice.” After the morning regen, git blame pointed at the bot for both headings. On-call learned about the retry text from a partner ticket, not from review.

Markdown has headings. It does not have owners. Tools that “just regenerate the docs” treat the file as one blob. The blob is the bug.

Draftable vs owned is not a style choice

Classify by consequence, not by how confident the model sounds. A fluent paragraph about retries is still a promise if a customer will act on it.

Decision table

Doc surface	Model may draft?	Human must own?	Why	Failure if the model rewrites it
Parameter tables derived from OpenAPI	Yes	No	Spec is the source	Stale field names
Sandbox `curl` examples	Yes	Review examples that hit auth	Shape can regen	Token pasted into prod
Observed rate limits (informational)	Yes, if labeled current	No	Snapshot, not contract	Partners treat a snapshot as SLA
Contractual rate limits	No	Yes	Billing and abuse	Silent tightening or loosening
Retry / idempotency guidance	No	Yes	Side effects	Duplicate charges, double voids
Deprecation dates and sunset windows	No	Yes	Calendar is a promise	Early break, late break
Support hours and escalation	No	Yes	Staffing	Page the wrong rotation
PII, retention, subprocessors	No	Yes	Legal	Unreviewed processing claims
Architecture rationale	Yes	Edit for accuracy	Explanatory	Harmless drift, usually
Breaking-change language	No	Yes	Version contract	Accidental commitment

If a row can move money, data, or a page, it is owned. If a row can be rebuilt from a machine-readable source, it is draftable.

Artifact: a heading ownership registry

Proposed format. JSON, so the checker stays on the standard library. One registry file per Markdown doc, checked in next to it.

{
  "file": "docs/payments.md",
  "headings": [
    {
      "id": "getting-started",
      "match": "Getting started",
      "level": 2,
      "policy": "draft",
      "owner": null,
      "forbidden": ["SLA", "guarantee", "do not retry", "PII", "we will not"]
    },
    {
      "id": "auth-headers",
      "match": "Authentication headers",
      "level": 3,
      "policy": "draft",
      "owner": null,
      "forbidden": ["production secret", "never rotate"]
    },
    {
      "id": "voiding",
      "match": "Voiding an invoice",
      "level": 2,
      "policy": "own",
      "owner": "payments-oncall",
      "hash": "replace-me"
    },
    {
      "id": "deprecation",
      "match": "Deprecation window",
      "level": 2,
      "policy": "own",
      "owner": "api-council",
      "hash": "replace-me"
    }
  ]
}

Rules the registry encodes:

Every ## / ### in the doc must appear in headings.
Every registry row must match exactly one heading.
policy: own rows store a SHA-256 of the normalized section body.
policy: draft rows may change freely, but CI rejects forbidden phrases.
Changing an owned hash is a human action (sign_owned.py), not a regen side effect.

Normalize, then hash

Whitespace and trailing newlines will false-fail any lock. Normalize before hashing. Proposed function:

# ownership_lib.py
from __future__ import annotations

import hashlib
import json
import re
from pathlib import Path

HEADING_RE = re.compile(r"^(#{2,3})\s+(.+?)\s*$", re.M)


def normalize(text: str) -> str:
    lines = [line.rstrip() for line in text.replace("\r\n", "\n").split("\n")]
    while lines and lines[-1] == "":
        lines.pop()
    return "\n".join(lines) + "\n"


def sha256(text: str) -> str:
    return hashlib.sha256(normalize(text).encode("utf-8")).hexdigest()


def split_sections(markdown: str) -> list[dict]:
    matches = list(HEADING_RE.finditer(markdown))
    sections = []
    for i, match in enumerate(matches):
        start = match.start()
        end = matches[i + 1].start() if i + 1 < len(matches) else len(markdown)
        level = len(match.group(1))
        title = match.group(2).strip()
        sections.append({
            "level": level,
            "title": title,
            "body": markdown[start:end],
        })
    return sections


def load_registry(path: Path) -> dict:
    return json.loads(path.read_text(encoding="utf-8"))

Unexecuted example: if your renderer injects extra blank lines, keep normalize() as the only hash input. Do not hash the raw file.

Checker: fail closed on owned drift

# check_ownership.py
from __future__ import annotations

import argparse
import sys
from pathlib import Path

from ownership_lib import load_registry, sha256, split_sections


def main() -> int:
    parser = argparse.ArgumentParser()
    parser.add_argument("--registry", required=True)
    parser.add_argument("--root", default=".")
    args = parser.parse_args()

    registry = load_registry(Path(args.registry))
    doc_path = Path(args.root) / registry["file"]
    markdown = doc_path.read_text(encoding="utf-8")
    sections = split_sections(markdown)
    by_title = {(s["level"], s["title"]): s for s in sections}

    errors: list[str] = []
    seen = set()

    for row in registry["headings"]:
        key = (row["level"], row["match"])
        seen.add(key)
        section = by_title.get(key)
        if section is None:
            errors.append(f"missing heading: {row['match']}")
            continue
        if row["policy"] == "own":
            digest = sha256(section["body"])
            if digest != row.get("hash"):
                errors.append(
                    f"owned heading drifted: {row['match']} "
                    f"owner={row.get('owner')} expected={row.get('hash')} got={digest}"
                )
        elif row["policy"] == "draft":
            lower = section["body"].lower()
            for phrase in row.get("forbidden") or []:
                if phrase.lower() in lower:
                    errors.append(
                        f"draft heading contains owned language: {row['match']!r} -> {phrase!r}"
                    )
        else:
            errors.append(f"unknown policy on {row['id']}: {row['policy']}")

    for section in sections:
        key = (section["level"], section["title"])
        if key not in seen:
            errors.append(f"unregistered heading: {section['title']}")

    for err in errors:
        print(err, file=sys.stderr)
    if errors:
        print(f"{len(errors)} ownership error(s)", file=sys.stderr)
        return 1
    print(f"ok: {registry['file']} ({len(registry['headings'])} headings)")
    return 0


if __name__ == "__main__":
    sys.exit(main())

Run it the same way in CI and locally:

python check_ownership.py --registry docs/payments.ownership.json --root .

Expected first run: every own row fails until you sign. That is intended. Unsigned owned text is not “flexible.” It is unowned.

Sign only after a human edit

# sign_owned.py
from __future__ import annotations

import argparse
import json
from pathlib import Path

from ownership_lib import load_registry, sha256, split_sections


def main() -> None:
    parser = argparse.ArgumentParser()
    parser.add_argument("--registry", required=True)
    parser.add_argument("--root", default=".")
    parser.add_argument("--id", required=True, help="heading id to re-sign")
    args = parser.parse_args()

    path = Path(args.registry)
    registry = load_registry(path)
    markdown = (Path(args.root) / registry["file"]).read_text(encoding="utf-8")
    sections = {(s["level"], s["title"]): s for s in split_sections(markdown)}

    for row in registry["headings"]:
        if row["id"] != args.id:
            continue
        if row["policy"] != "own":
            raise SystemExit(f"{args.id} is not owned")
        section = sections[(row["level"], row["match"])]
        row["hash"] = sha256(section["body"])
        path.write_text(json.dumps(registry, indent=2) + "\n", encoding="utf-8")
        print(f"signed {args.id} -> {row['hash']}")
        return
    raise SystemExit(f"unknown id: {args.id}")


if __name__ == "__main__":
    main()

A regen job may rewrite draft headings and commit them. It must not invoke sign_owned.py. If the hash file changes in the same commit as a bot regen, review should reject the commit. That review rule is social. The checker is mechanical. You want both.

Slice owned text out of the prompt

Most “the model overwrote the runbook” bugs start with a full-file prompt. The model sees the promise, then rephrases it into the how-to. Stop giving it the promise.

# slice_draftable.py
from __future__ import annotations

import argparse
import json
from pathlib import Path

from ownership_lib import load_registry, split_sections


PROMPT_PREFIX = """You may rewrite only the sections below.
Do not add guarantees, SLAs, retry bans, PII rules, or deprecation dates.
Keep heading titles unchanged. Return Markdown sections only.
"""


def main() -> None:
    parser = argparse.ArgumentParser()
    parser.add_argument("--registry", required=True)
    parser.add_argument("--root", default=".")
    parser.add_argument("--out", required=True)
    args = parser.parse_args()

    registry = load_registry(Path(args.registry))
    markdown = (Path(args.root) / registry["file"]).read_text(encoding="utf-8")
    sections = {(s["level"], s["title"]): s for s in split_sections(markdown)}

    parts = [PROMPT_PREFIX]
    for row in registry["headings"]:
        if row["policy"] != "draft":
            continue
        section = sections[(row["level"], row["match"])]
        parts.append(section["body"].rstrip() + "\n")

    Path(args.out).write_text("\n".join(parts), encoding="utf-8")
    print(f"wrote {args.out}")


if __name__ == "__main__":
    main()

Proposed loop:

slice_draftable.py writes draft-packet.md.
A model rewrites only that packet.
A small merge script replaces draft headings by title, never owned ones.
check_ownership.py runs. Owned hashes must be unchanged. Draft forbidden phrases must be absent.
Humans update owned headings in a separate PR, then sign_owned.py.

The merge script is boring on purpose. Title match, then splice. If a title is missing, fail. Do not fuzzy-match a deprecation section into “Getting started.”

Where a free model pass fits

The draft packet is the only thing that should leave the repo. Owned sections stay local. That split is the method. The model is interchangeable.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. If you want the draft pass off your laptop, MonkeyCode is one open-source option with free model access and a free server option. Point it at draft-packet.md, not at docs/payments.md. The registry and the checker do not depend on that choice. They still run in CI if you swap the model later.

Keep product setup out of the owned headings. A getting-started draft can mention a sandbox base URL. It should not mention on-call aliases, retention days, or “safe to retry.”

Test plan (run these before you trust CI)

Label: unexecuted until you point it at a real file.

Coverage. Add an unregistered ## Shadow heading. Checker must exit 1.
Owned drift. Append a sentence to an own section without signing. Checker must exit 1 and print the new hash.
Sign path. Run sign_owned.py --id voiding. Checker must exit 0.
Draft poison. Insert we guarantee under a draft heading. Checker must exit 1.
Prompt isolation. Confirm draft-packet.md contains no owned heading titles from the registry.
Bot boundary. In a sample diff, a regen commit that touches hash must be treated as a review failure even if the checker was skipped.

If test 5 fails, stop generating. You are still pasting promises into the prompt.

Limitations

Heading policy is coarse. A promise buried in a bullet under “Overview” will be classified as draftable if that heading is draftable. Move promises under their own H2.

Hashing is brittle against prettier, link checkers that rewrite URLs, and locale-specific quotes. Normalize more aggressively, or freeze owned sections as separate files (voiding.md) and !include them. This article does not implement includes.

Forbidden-phrase lists are not a legal review. They catch accidental contract language in how-to text. They will miss a politely worded commitment that names no keyword.

The splitter only handles ## and ###. Front matter, HTML <h2>, and Setext headings are invisible to it. Extend HEADING_RE before you roll this out to a mixed corpus.

Who should not use this

Do not use a heading registry as the system of record for contracts, SOC reports, or anything a lawyer must sign. Generate those from a source that is already owned, or do not generate them.

Skip this if the entire README is a promise (status pages, incident policy, data-processing addenda). There is nothing draftable. A model has no lane.

Skip this if you have no CI gate. An unsigned JSON file next to the docs is documentation of intent, not enforcement.

Teams that already generate reference pages from OpenAPI and keep runbooks in a different repo may not need this. Their split is the directory. This method is for the common case: one Markdown file, two kinds of truth, one regen job that cannot tell them apart.

The model can draft the how. A human still owns the promise. Register the heading before the next OpenAPI refresh, not after the partner retries void.

Three Zones for AI-Drafted Docs — and a Linter That Enforces Them

Morgan Sun — Mon, 14 Sep 2026 21:37:14 +0000

Liquid syntax error: Unknown tag 'endraw'

Generate Error Catalogs Without Erasing On-Call Advice

Morgan Sun — Sun, 13 Sep 2026 19:56:24 +0000

The Friday regen looked successful. Every 4xx in payments/errors.py showed up on the public API page, each with a status code and a sample JSON body.

Saturday morning, support replayed a capture call because the new page treated 409 as a safe retry. It was not. The second capture settled a second time. This is a composite of a failure that appears whenever generated reference is treated as complete documentation.

Happy-path fields are cheap to regenerate. Remediation is not. The workflow below splits an error catalog into extractable facts a model may draft, and operational advice a human must own in a freeze file that CI can test.

The two halves of an error catalog

An error page that only lists codes is a dictionary. On-call needs a second half: retry class, blast radius, customer-visible wording, and whether a replay mutates money or data.

Those two halves do not share a lifecycle.

Regenerable: error code, HTTP status, identifier in source, short mechanical description, example payload shape.
Not regenerable: idempotency, retry class, paging rule, data-loss flag, compensation, language a customer will read.

If both halves live in one Markdown file, the next doc regen will keep the enum and drop the sentence that mattered. That is not a model quality problem. It is an ownership problem.

Ownership matrix

Use this table as the contract. If a field has no machine-readable source, it does not belong in the draft lane.

Field	Source of truth	Regen allowed	Owner
`code`	enum / constant in repo	yes	extractor
`http_status`	same source	yes	extractor
`source_ref`	file:line or symbol	yes	extractor
`mechanical_summary`	comment or identifier	yes	model draft
`example_payload`	schema or handler	yes	model draft
`idempotent`	freeze file only	no	human
`retry_class`	freeze file only	no	human
`page_on`	freeze file only	no	human
`customer_wording`	freeze file only	no	human
`data_loss`	freeze file only	no	human

retry_class is not a prose flourish. It is one of none, same_payload, backoff, or manual_only. If the model is allowed to invent that value from a handler name, it will guess backoff for anything that looks like a conflict.

Freeze file format

Keep generated output and human remediation in different paths. Proposed layout:

docs/
  generated/errors.catalog.json
  frozen/remediation.yml
  public/errors.md

remediation.yml is source, same as code. Proposed schema:

# docs/frozen/remediation.yml
# Human-owned. Regenerators must not write this file.
PAYMENT_CAPTURE_CONFLICT:
  idempotent: false
  retry_class: manual_only
  page_on: payments-oncall
  data_loss: duplicate_settlement
  customer_wording: >-
    Do not retry capture. Open a ticket with the payment intent id.
  last_reviewed: 2026-09-12
  reviewed_by: payments-api
INSUFFICIENT_FUNDS:
  idempotent: true
  retry_class: none
  page_on: null
  data_loss: none
  customer_wording: >-
    The method was declined. Ask the customer for another method.
  last_reviewed: 2026-09-10
  reviewed_by: payments-api

Missing keys are failures, not omissions the model can fill. A blank retry_class is how the Saturday capture happens again.

Extract from source, not from last week's Markdown

The draft lane should read the error enum, not the previous HTML. Proposed extractor against a small Python source of truth:

# payments/errors.py
from dataclasses import dataclass

@dataclass(frozen=True)
class ApiError:
    code: str
    http_status: int
    summary: str

PAYMENT_CAPTURE_CONFLICT = ApiError(
    "PAYMENT_CAPTURE_CONFLICT",
    409,
    "Capture rejected because the intent is not capturable in this state.",
)
INSUFFICIENT_FUNDS = ApiError(
    "INSUFFICIENT_FUNDS",
    402,
    "The processor declined the method for insufficient funds.",
)

CATALOG = (PAYMENT_CAPTURE_CONFLICT, INSUFFICIENT_FUNDS)

# tools/extract_errors.py — proposed, unexecuted in this article
import importlib.util
import json
from pathlib import Path

def load_catalog(path: str):
    spec = importlib.util.spec_from_file_location("errors", path)
    mod = importlib.util.module_from_spec(spec)
    spec.loader.exec_module(mod)
    rows = []
    for item in mod.CATALOG:
        rows.append({
            "code": item.code,
            "http_status": item.http_status,
            "mechanical_summary": item.summary,
            "source_ref": path,
        })
    return rows

def main():
    rows = load_catalog("payments/errors.py")
    out = Path("docs/generated/errors.catalog.json")
    out.parent.mkdir(parents=True, exist_ok=True)
    out.write_text(json.dumps(rows, indent=2) + "\n")
    print(f"wrote {len(rows)} errors to {out}")

if __name__ == "__main__":
    main()

Run it as a boring command, not as a chat instruction:

python tools/extract_errors.py
python tools/check_remediation.py

Extraction can be wrong if the enum is incomplete. It cannot silently drop a freeze-file paragraph. That is the point.

What the model may draft

After extraction, the model may write only regenerable fields. Label the following as a proposed prompt, not a measured eval.

You receive a JSON array of API errors with code, http_status,
mechanical_summary, and source_ref.

For each item, draft:
- example_payload: a JSON object with "error" and "code" keys
- mechanical_summary_rewritten: one sentence, no retry advice

Do not invent idempotency, retry policy, paging, compensation,
or customer-facing promises. If those appear in the summary,
strip them. Return JSON only.

The output lands in docs/generated/errors.catalog.json. It never lands in docs/frozen/remediation.yml.

A drafting job needs somewhere to run that is disposable. Disclosure: This article was prepared as part of MonkeyCode's product outreach. If you already have MonkeyCode's free model access and free server option, that pair is enough to run this draft lane as a batch job. It does not replace the freeze file or the checker. The product is optional; the split is not.

Merge is a program, not a paste

Public Markdown is a view over two files. Proposed merge:

# tools/render_errors.py — proposed
import json
from pathlib import Path
import yaml

def render():
    catalog = json.loads(Path("docs/generated/errors.catalog.json").read_text())
    frozen = yaml.safe_load(Path("docs/frozen/remediation.yml").read_text()) or {}
    lines = ["# Payments error catalog", ""]
    for row in catalog:
        code = row["code"]
        rem = frozen[code]
        lines += [
            f"## `{code}`",
            "",
            f"- HTTP status: `{row['http_status']}`",
            f"- Retry class: `{rem['retry_class']}`",
            f"- Idempotent: `{rem['idempotent']}`",
            "",
            rem["customer_wording"],
            "",
            "Example:",
            "",
            "```

json",
            json.dumps(row.get("example_payload", {"error": True, "code": code}), indent=2),
            "

```",
            "",
        ]
    Path("docs/public/errors.md").write_text("\n".join(lines))

if __name__ == "__main__":
    render()

If merge cannot find a freeze record, it must raise. Filling retry_class with backoff to keep the build green recreates the capture incident in CI clothing.

CI gate: the only test that matters

The test is not “does the page look complete.” The test is “would a full regen still print the irreversible warning.”

# tools/check_remediation.py — proposed
import json
import sys
from pathlib import Path
import yaml

REQUIRED = (
    "idempotent",
    "retry_class",
    "page_on",
    "data_loss",
    "customer_wording",
    "last_reviewed",
    "reviewed_by",
)
RETRY = {"none", "same_payload", "backoff", "manual_only"}

def main() -> int:
    catalog = json.loads(Path("docs/generated/errors.catalog.json").read_text())
    frozen_path = Path("docs/frozen/remediation.yml")
    frozen = yaml.safe_load(frozen_path.read_text()) or {}
    codes = [row["code"] for row in catalog]
    errors = []

    extra = sorted(set(frozen) - set(codes))
    missing = sorted(set(codes) - set(frozen))
    if extra:
        errors.append(f"freeze file has unknown codes: {extra}")
    if missing:
        errors.append(f"freeze file missing codes: {missing}")

    for code in codes:
        rec = frozen.get(code) or {}
        for key in REQUIRED:
            if key not in rec or rec[key] in ("", []):
                errors.append(f"{code}.{key} is empty")
        if rec.get("retry_class") not in RETRY:
            errors.append(f"{code}.retry_class is not in {sorted(RETRY)}")
        if rec.get("idempotent") is False and rec.get("retry_class") != "manual_only":
            errors.append(f"{code}: non-idempotent errors must be manual_only")

    # Guard against a generator writing the freeze path.
    text = frozen_path.read_text()
    if "AUTO-GENERATED" in text or "Drafted by" in text:
        errors.append("freeze file looks generated; refuse merge")

    for e in errors:
        print(e, file=sys.stderr)
    return 1 if errors else 0

if __name__ == "__main__":
    raise SystemExit(main())

Wire it so docs PRs cannot skip the gate:

# .github/workflows/error-catalog.yml — proposed
name: error-catalog
on:
  pull_request:
    paths:
      - "payments/errors.py"
      - "docs/**"
      - "tools/**"
jobs:
  check:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: pip install pyyaml
      - run: python tools/extract_errors.py
      - run: python tools/check_remediation.py

A useful extra assertion: if payments/errors.py changes and docs/frozen/remediation.yml does not, require a human to either add a key or record why the new code is not customer-visible. Silence is not review.

What this does not prove

The checker proves the remediation record still exists and is well-typed. It does not prove the advice is correct. manual_only can be wrong. page_on can point at a dead rotation. last_reviewed can be stale while still present.

Treat review dates as a separate policy. A 90-day age check is easy to add. It is still a policy, not evidence that on-call agrees.

Other limits:

If errors are raised as ad-hoc strings, extraction is theater. Put them in an enum first.
Customer wording in regulated payments, health, or identity flows still needs a reviewer who owns that risk. A freeze file is not counsel.
Localized public docs need a freeze record per locale, or a reviewed translation step. Do not let a draft job translate Do not retry capture unsupervised.
Example payloads can leak internal field names. Keep examples on the public schema, not on the log line from production.

Who should not use this

Skip this split if you have no machine-readable error list. A model inventing both the catalog and the retry policy is how you get confident, wrong runbooks.

Skip it if the document is the legal contract. Terms of service, data-processing language, and uptime commitments are not error catalogs.

Skip it if the team will not fail CI on an empty retry_class. A freeze file that reviewers “mean to fill in later” is a blog post in your repo.

Recap

Regenerate codes, status, and example shapes from source. Store retry class, idempotency, paging, data-loss, and customer wording in a file no generator may write. Merge them in code. Fail the build when a new error has no freeze record, or when a non-idempotent error is marked safe to retry.

The question to ask on every docs regen is narrow. After a clean generate, is the do-not-retry line still there?

Regen the Example, Freeze the Side Effect: Capability Cards for Internal APIs

Morgan Sun — Sat, 12 Sep 2026 17:56:27 +0000

An internal refunds page sat in the team wiki for six weeks. The curl example still parsed. The retry sentence did not.

On a Tuesday rollback, an on-call engineer followed a generated note that called POST /v2/refunds "safe to retry." The handler was not idempotent. Two credits posted against one charge. The example was only stale. The promise was invented.

Generated API docs fail on two clocks. Shape docs rot when fields move. Promise docs fail when a model writes operational guarantees that nobody signed. Mixing those clocks in a single Markdown file is the usual accident.

This article proposes a capability card: one short pair of files per internal endpoint or agent tool. The model may draft shape. A human must own side effects. A checker fails the build when the lanes mix. The workflow below is a proposal. Scripts are local examples, not production telemetry.

Two clocks, not one page

Treat each endpoint as two documents glued by a stable id.

Clock A — shape (regenerable)

Purpose line restated from the OpenAPI summary
Parameter table
Example request and response
Links to sibling paths

Clock B — promise (frozen until a human edits it)

Authentication and authorization
Side effects: money, email, deletes, lock state
Idempotency and retry
PII and retention
On-call owner
Deprecation window

Clock A can be rewritten whenever the spec hash changes. Clock B must survive that rewrite. If a model edits Clock B, the card is invalid even when the prose looks cleaner.

Agent tool catalogs make the split more urgent, not less. Teams now paste the same paragraphs into tool descriptions. A hallucinated retry line is no longer only an on-call footnote. It becomes a policy the agent will execute.

Ownership matrix

Field	Owner	Regen	Fail the build when
`summary`	model	yes	missing, or ignores the path
`params`	model	yes	name absent from inventory
`examples`	model	yes	method or path mismatch
`authn`	human	no	empty
`side_effects`	human	no	empty
`retry`	human	no	hedging language
`owner`	human	no	missing on-call id
`signed_at`	human	no	absent

Keep the matrix in git. Do not keep it only inside a prompt. Prompts drift. Tables in the repo do not, at least not silently.

File layout

Do not give the model a single file that also holds promises. Split directories.

cards/
  human/          # Clock B, humans only
    refunds.create.yaml
  draft/          # Clock A, model output only
    refunds.create.md
forbidden-draft.txt
inventory.json
capability_card_check.py

Human lane:

id: refunds.create
path: POST /v2/refunds
owner: payments-oncall
signed_at: 2026-09-12
authn: service-token + finance.refund.write
side_effects:
  - posts a credit to the ledger
  - emits refund.created
  - may send a customer email
retry: do-not-retry; read GET /v2/refunds/{id}
pii: customer_id, last4
deprecation: null

Draft lane:

<!-- CARD_ID: refunds.create -->
<!-- SPEC_HASH: demo -->
## POST /v2/refunds

Creates a refund against a captured charge.

### Parameters

| name | in | type | required |
| --- | --- | --- | --- |
| charge_id | body | string | yes |
| amount_cents | body | integer | yes |

### Example

curl -X POST https://api.internal.example/v2/refunds \
  -H "Authorization: Bearer $TOKEN" \
  -d '{"charge_id":"ch_123","amount_cents":1500}'

The example may be wrong tomorrow. That is acceptable if CI regenerates it from the spec. The retry line must not appear in the Markdown body.

Forbidden phrases in Clock A

A short linter beats another system prompt. Scan only cards/draft.

idempotent
safe to retry
no side effects
does not charge
guarantee
SLA
PII-free
never fails
exactly-once
can be called from a bot

If any phrase hits, discard the draft. Do not hand-edit the leak back into the .md file. The next regeneration will restore it.

Inventory from a reduced OpenAPI file

You do not need a full parser to start. A reduced spec is enough for the checker.

#!/usr/bin/env python3
"""inventory_from_openapi.py — proposed extractor, fixture-oriented."""
from __future__ import annotations

import json
import sys
from pathlib import Path

def main() -> int:
    spec = json.loads(Path(sys.argv[1]).read_text())
    endpoints = []
    for path, methods in (spec.get("paths") or {}).items():
        for method, op in methods.items():
            if method.startswith("x-") or not isinstance(op, dict):
                continue
            op_id = op.get("operationId") or f"{method}.{path}"
            endpoints.append(
                {
                    "id": op_id,
                    "path": f"{method.upper()} {path}",
                    "summary": op.get("summary") or "",
                }
            )
    Path("inventory.json").write_text(
        json.dumps({"endpoints": endpoints}, indent=2) + "\n"
    )
    print(f"wrote {len(endpoints)} endpoints")
    return 0

if __name__ == "__main__":
    raise SystemExit(main())

Run:

python3 inventory_from_openapi.py openapi.fixture.json

Every inventory id must have a matching cards/human/<id>.yaml. Missing YAML is a failed build, not a TODO comment.

Checker

The checker loads inventory, requires human fields, and rejects draft files that contain frozen concerns.

#!/usr/bin/env python3
"""capability_card_check.py — fail if draft and promise lanes mix."""
from __future__ import annotations

import hashlib
import json
import re
import sys
from pathlib import Path

try:
    import yaml
except ImportError:
    print("pip install pyyaml", file=sys.stderr)
    sys.exit(2)

FORBIDDEN = [
    ln.strip().lower()
    for ln in Path("forbidden-draft.txt").read_text().splitlines()
    if ln.strip()
]
HUMAN_REQUIRED = ("owner", "signed_at", "authn", "side_effects", "retry")

def load_inventory(path: Path) -> dict:
    data = json.loads(path.read_text())
    return {item["id"]: item for item in data["endpoints"]}

def check_human(card: dict, endpoint: dict, errors: list[str]) -> None:
    cid = card.get("id", "<missing>")
    for key in HUMAN_REQUIRED:
        if not card.get(key):
            errors.append(f"{cid}: missing human field {key}")
    if card.get("path") != endpoint.get("path"):
        errors.append(f"{cid}: path does not match inventory")

def check_draft(md: str, card_id: str, errors: list[str]) -> None:
    low = md.lower()
    for phrase in FORBIDDEN:
        if phrase in low:
            errors.append(f"{card_id}: draft lane contains '{phrase}'")
    if re.search(r"^retry:", md, re.M | re.I):
        errors.append(f"{card_id}: retry belongs in cards/human")
    if "CARD_ID:" in md and card_id not in md:
        errors.append(f"{card_id}: CARD_ID mismatch")

def main() -> int:
    human_dir = Path("cards/human")
    draft_dir = Path("cards/draft")
    inventory = load_inventory(Path("inventory.json"))
    errors: list[str] = []
    yaml_ids = set()
    for yml in sorted(human_dir.glob("*.yaml")):
        card = yaml.safe_load(yml.read_text()) or {}
        cid = card.get("id")
        if not cid:
            errors.append(f"{yml.name}: no id")
            continue
        yaml_ids.add(cid)
        if cid not in inventory:
            errors.append(f"{cid}: not in inventory")
            continue
        check_human(card, inventory[cid], errors)
        md_path = draft_dir / f"{cid}.md"
        if not md_path.exists():
            errors.append(f"{cid}: missing draft markdown")
            continue
        check_draft(md_path.read_text(), cid, errors)
    for mid in sorted(set(inventory) - yaml_ids):
        errors.append(f"{mid}: inventory path has no signed card")
    spec_hash = hashlib.sha256(Path("inventory.json").read_bytes()).hexdigest()[:12]
    if errors:
        print("FAIL")
        print("\n".join(errors))
        return 1
    print(f"ok {len(yaml_ids)} cards inventory={spec_hash}")
    return 0

if __name__ == "__main__":
    raise SystemExit(main())

pip install pyyaml
python3 capability_card_check.py

A red build is the control. Green prose in a wiki is not.

Test plan (fixture, not a live service)

Drop retry: do-not-retry from the YAML. Expect fail on missing human field retry.
Add "safe to retry" under the curl example. Expect fail on forbidden phrase.
Add an inventory endpoint with no YAML. Expect fail on unsigned card.
Restore a clean pair. Expect ok 1 cards.
Point the model at cards/human. If it writes YAML, treat that as a pipeline bug, not a content fix.

Do not run step 5 against production credentials. The draft job should have write access only to cards/draft.

Draft loop

When inventory.json changes, regenerate only cards/draft/*.md. Leave YAML untouched.

Using inventory.json and the OpenAPI snippet, rewrite cards/draft/<id>.md.
Do not mention retry, auth, SLA, PII, or side effects.
Keep the CARD_ID comment.
Output Markdown only.

If the checker fails, discard the file and rerun. Do not patch the leak by hand in the draft lane.

The job is mechanical: spec fragment to table and curl. A long agent loop is unnecessary. A hosted free model is enough for Clock A. Clock B still needs a named human.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

MonkeyCode is an open-source project with free model access and a free server option. Those two properties fit this workflow in a narrow way. The model drafts Clock A. The server can run capability_card_check.py on a schedule so a closed laptop does not skip the gate. Neither property fills owner or signed_at. If a run rewrites YAML, that is an incident.

Limits and who should skip this

The checker does not prove the YAML is true. A human can sign a wrong retry policy.
Inventory is a file you maintain. This is not traffic discovery.
Public customer docs, legal copy, and localization are out of scope.
The linter is string-based. It will miss a paraphrased promise that avoids the phrase list. Extend the list when you find a miss. Do not pretend the list is complete.
It does not make an agent understand the API. It only stops the agent-facing page from claiming a retry nobody signed.

Skip the approach if you maintain one public README and no money-moving or delete paths. Skip it if you cannot name an on-call owner per path. A card without owner is another wiki page.

If examples and promises are already split in review, the useful next step is a CI checker, not a longer prompt. Free model access plus a free server is enough to keep the draft lane regenerable. The frozen lane still needs a person.

Cite-or-Freeze: Split AI Docs Into Cited Claims and Frozen Commands

Morgan Sun — Fri, 11 Sep 2026 15:38:17 +0000

The on-call page landed at 02:14, and the first link in the runbook still said to restart worker-b. The repo had renamed that process six days earlier. Generated Markdown still rendered. It simply pointed operators at a binary that was gone.

That failure is not a grammar problem. Models are fluent. Fluency hides drift. The dangerous sections are the ones people copy into a shell at 2 a.m., not the product-overview paragraph that nobody treats as a contract.

This article proposes a split pipeline. The model may draft narrative that cites a source symbol. A human must freeze recovery commands, warnings, and any sentence that cannot point at a file. A small linter enforces the split. If every product mention below disappeared, the gate would still be usable.

The failure mode is citation, not tone

Happy-path READMEs usually survive a regenerate. Troubleshooting does not. Error catalogs do not. Runbooks do not. Those pages mix two kinds of text:

Derived claims — “UploadService retries with backoff.” This can be checked against code.
Operational commitments — “Page #platform-oncall after two failed rollbacks.” This is a human policy. No symbol in src/ proves it.

A single “AI wrote the docs” pass flattens both. The first kind should be cheap to refresh. The second kind should be expensive to change.

Ownership split, in one table

Doc region	Model may draft?	Required artifact	Human must freeze?
Architecture overview	Yes, with cites	`path#symbol` on each factual sentence	No
Public endpoint list	Yes, from the spec	OpenAPI operationId or proto RPC name	No
Code sample that compiles	Draft only inside a fixture	Extracted file under `docs/fixtures/`	Human owns the fixture
Shell recovery steps	No	Fenced block marked `freeze:command`	Yes
Severity, paging, SLA language	No	Fenced block marked `freeze:policy`	Yes
Security warnings	No	`freeze:warning`	Yes
Changelog “why we did this”	Draft ok	Cite the PR or issue id	Human owns the “why”

The rule is narrow. If a sentence asserts a fact about the repo, it needs a cite. If a sentence tells a human to do something irreversible, it needs a freeze block. Everything else is optional color and should not be in the critical path.

The cite-or-freeze contract

Mark draftable sections with a lane comment. Mark frozen blocks with an info string on the fence. Keep both in ordinary Markdown so reviews stay readable.

<!-- lane: draftable -->
`InventoryWorker` reads from `jobs` and writes `inventory_events`.
<!-- cite: src/workers/inventory.py#InventoryWorker -->

<!-- lane: frozen -->

freeze:command
systemctl restart inventory-worker
journalctl -u inventory-worker -n 200

yaml

Three invariants follow:

Every factual sentence in a draftable lane has a following cite: comment.
A cite: target resolves to a path that exists, and the fragment exists in that file.
A freeze:* block is byte-identical to the last human-approved copy in docs/freeze/.

The model never writes into docs/freeze/. CI compares, it does not “improve.”

Artifact: a claim map plus a linter

The example below is a self-contained proposal. Run it on the fixtures in this article. It is not a production scanner and it does not parse every language.

1. Claim map

# docs/claim_map.yaml
version: 1
draftable_globs:
  - "docs/product/**/*.md"
  - "docs/api/overview.md"
frozen_globs:
  - "docs/runbooks/**/*.md"
  - "docs/security/**/*.md"
cite_required_lanes:
  - draftable
frozen_kinds:
  - command
  - policy
  - warning
index:
  roots:
    - src
    - proto
    - openapi.yaml
allow_missing_fragment_languages:
  - md

2. Sample source and a lying doc

# src/workers/inventory.py
class InventoryWorker:
    QUEUE = "jobs"
    OUT_STREAM = "inventory_events"
    MAX_ATTEMPTS = 5

<!-- file: docs/product/inventory.md -->
<!-- lane: draftable -->
`InventoryWorker` reads `jobs` and emits `inventory_events`.
<!-- cite: src/workers/inventory.py#InventoryWorker -->

Retries stop after three attempts.
<!-- cite: src/workers/inventory.py#MAX_ATTEMPTS -->

The second cite is the interesting case. The symbol exists. The sentence is still wrong: the code says 5, the prose says three. Citation is necessary, not sufficient. A later section treats that gap.

3. Linter (runnable example)

#!/usr/bin/env python3
"""claim_lint.py — cite-or-freeze gate. Proposal / example, not a full parser."""
from __future__ import annotations

import pathlib
import re
import sys

CITE = re.compile(r"<!--\s*cite:\s*([^#\s]+)(?:#([^\s]+))?\s*-->")
LANE = re.compile(r"<!--\s*lane:\s*(draftable|frozen)\s*-->")
FREEZE = re.compile(r"^```

freeze:(command|policy|warning)\s*$", re.M)
FACTISH = re.compile(
    r"`[^`]+`|\b(retries|timeout|port|queue|endpoint|restart)\b",
    re.I,
)

def iter_sentences(block: str) -> list[str]:
    parts = re.split(r"(?<=[.!?])\s+", block.strip())
    return [p for p in parts if p]

def file_has_fragment(path: pathlib.Path, fragment: str | None) -> bool:
    if not path.exists():
        return False
    text = path.read_text(encoding="utf-8")
    if not fragment:
        return True
    return fragment in text

def lint(md_path: pathlib.Path, repo: pathlib.Path) -> list[str]:
    text = md_path.read_text(encoding="utf-8")
    errors: list[str] = []
    lane = "unknown"
    for raw_line in text.splitlines():
        m = LANE.search(raw_line)
        if m:
            lane = m.group(1)

    if lane == "draftable":
        sentences = iter_sentences(re.sub(r"```.*?```", "", text, flags=re.S))
        cites = list(CITE.finditer(text))
        fact_rows = [s for s in sentences if FACTISH.search(s)]
        if len(cites) < len(fact_rows):
            errors.append(
                f"{md_path}: {len(fact_rows)} fact-like sentences, {len(cites)} cites"
            )
        for c in cites:
            rel, frag = c.group(1), c.group(2)
            target = repo / rel
            if not file_has_fragment(target, frag):
                errors.append(f"{md_path}: missing cite {rel}#{frag or ''}")

    for kind in FREEZE.findall(text):
        freeze_copy = repo / "docs" / "freeze" / f"{md_path.stem}.{kind}.txt"
        if not freeze_copy.exists():
            errors.append(f"{md_path}: no frozen snapshot for {kind}")
            continue
        fenced = re.search(
            rf"`{% endraw %}{% raw %}``freeze:{kind}\n(.*?)``{% endraw %}{% raw %}`", text, flags=re.S
        )
        if fenced and fenced.group(1) != freeze_copy.read_text(encoding="utf-8"):
            errors.append(f"{md_path}: freeze:{kind} drifted from docs/freeze/")

    if lane == "frozen" and CITE.search(text) is None and not FREEZE.search(text):
        errors.append(f"{md_path}: frozen lane has neither cite nor freeze block")
    return errors

def main() -> int:
    repo = pathlib.Path.cwd()
    docs = repo / "docs"
    failures: list[str] = []
    for md in docs.rglob("*.md"):
        failures.extend(lint(md, repo))
    for item in failures:
        print(item)
    return 1 if failures else 0

if __name__ == "__main__":
    sys.exit(main())
```

### 4. Commands

``{% endraw %}{% raw %}`bash
chmod +x claim_lint.py
python3 claim_lint.py
mkdir -p docs/freeze docs/product src/workers
# After a human edits a recovery block:
python3 - <<'PY'
from pathlib import Path
import re, sys
p = Path("docs/runbooks/inventory.md")
text = p.read_text()
for kind in ("command", "policy", "warning"):
    m = re.search(rf"`{% endraw %}{% raw %}``freeze:{kind}\n(.*?)``{% endraw %}{% raw %}`", text, re.S)
    if m:
        Path(f"docs/freeze/{p.stem}.{kind}.txt").write_text(m.group(1))
print("freeze snapshots written")
PY
```

A green run means every draftable fact has a pointer, and every freeze block matches the snapshot. A red run means regenerate is blocked. That is the point.

## What the model may draft

Keep the prompt boring. Feed it the claim map, the source files listed in `index.roots`, and the current Markdown with freeze blocks stripped. Ask for sentences, not for commands.

``{% endraw %}{% raw %}`text
Draft only <!-- lane: draftable --> sections.
Each factual sentence must be followed by <!-- cite: path#symbol -->.
Do not invent symbols. If the source does not contain the fact, omit the sentence.
Do not modify ```freeze:*``` blocks.
Do not write paging, rollback, or credential instructions.
```

The output is a patch against draftable lanes. Reviewers read the patch plus the linter diff, not a 2,000-word regenerate.

## What a human must own

Freeze these even when the model is confident:

- Any command that mutates state (`restart`, `kubectl delete`, `DROP`, `chmod`, token rotation).
- Severity labels and paging targets.
- Numbers that are policy, not code: customer-facing RTO, support hours, “we will notify.”
- Threat text. A wrong warning is worse than a missing adjective.
- Examples that need secrets, production hostnames, or customer data shapes.

A useful review question: “If this sentence is wrong, does someone run a command or trust a guarantee?” If yes, freeze it. If no, cite it.

## Numeric claims need a second gate

The `MAX_ATTEMPTS` example shows the remaining hole. The fragment exists, so `claim_lint.py` passes, while the prose still says “three.” Add a narrow check for integer literals next to known names.

``{% endraw %}{% raw %}`python
# proposal: numeric_cite.py — unexecuted extension, same fixtures
import ast, pathlib, re

ASSIGN = re.compile(r"^(\w+)\s*=\s*(\d+)", re.M)

def constants_in(path: pathlib.Path) -> dict[str, str]:
    text = path.read_text(encoding="utf-8")
    found = dict(ASSIGN.findall(text))
    return found

def prose_numbers(sentence: str) -> list[str]:
    words = {
        "one": "1", "two": "2", "three": "3", "four": "4",
        "five": "5", "six": "6", "seven": "7", "eight": "8",
        "nine": "9", "ten": "10",
    }
    out = re.findall(r"\b\d+\b", sentence)
    for w, n in words.items():
        if re.search(rf"\b{w}\b", sentence, re.I):
            out.append(n)
    return out
```

Wire it only for cites whose fragment is a constant. If the sentence contains a number and it is not in `{value, value-1}` of that constant, fail. Do not try to NLP the whole paragraph. The cheap check catches the common lie: “retries three times” next to `MAX_ATTEMPTS = 5`.

## Where a free drafting host fits

The generate step is bursty and disposable. The gate belongs in CI on the repo you already trust. Those two jobs should not share credentials.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. If you need a drafting loop without standing up a paid GPU box, MonkeyCode’s free model access and free server option can host the generate step that rewrites `draftable` lanes. Keep `docs/freeze/` and `claim_lint.py` on your own runner. The model never needs permission to overwrite recovery commands.

## Limitations

- **Citation is not truth.** A symbol can exist and the sentence can still invert its meaning. The numeric extra-gate covers a subset. Semantic inversion (“does not retry”) still needs a human.
- **Fragment matching is string search.** `InventoryWorker` in a comment will satisfy the linter. Generated comments inside source can launder a bad cite. Own the source comments too.
- **Freeze snapshots fight formatting.** Re-wrapping a shell block fails CI. That is intended. Do not auto-format freeze files.
- **This will not satisfy legal or compliance review.** Terms of service, privacy notices, and SOC narratives need a different control plane.
- **Monorepos with generated code** can cite generated stubs that lag the spec. Index the spec, not the stub, when both exist.
- **No performance claims are implied.** The script walks Markdown and does substring checks. That is the whole method.

## Who should not use this

Skip the pipeline if the repo has no stable symbols (a pure design-docs tree), if every page is already hand-written and rarely regenerated, or if on-call runbooks live in a pager tool you cannot lint. Do not use it as a reason to skip incident review. A green linter after an outage only means the cites still resolve.

## A short adoption path

1. Pick one runbook and one overview page. Do not start with the whole `docs/` tree.
2. Move commands into `freeze:command` and snapshot them.
3. Add cites only to sentences that name a type, queue, flag, or numeric constant.
4. Run `claim_lint.py` in CI as a required check on `docs/**`.
5. Allow the model to patch `draftable` lanes. Reject any patch that touches `docs/freeze/`.

The on-call page at 02:14 does not need a better adjective. It needs a command block that cannot be regenerated by accident, and a narrative that cannot mention a worker the tree no longer contains. Cite the symbol. Freeze the command. Leave the rest of the prose cheap.