Unregistered Scopes Do Not Ship: A Registry Diff for AI-Drafted API Docs

Here is a failure mode that shows up in generated API guides. A partner integration failed on a Friday deploy. The public guide listed invoices:write:all. The identity service had never minted that scope.

The paragraph was fluent. The grant was imaginary. Completeness was the hazard, not spelling.

A sentence that names a scope, a rate, or a timeout is a runtime claim. Models copy the shape of real security schemes and then extend them. Fluent output is easy to mistake for a reviewed authorization model. Documentation is one of the places that mistake reaches customers.

Split the page before you split the prompt

Treat an API guide as two classes of text. One class can be regenerated when handlers change. The other class is a registry. If the model may extend the registry, it will.

What the model may draft

• Endpoint purpose restated from an OpenAPI summary or a handler comment
• Parameter narrative that does not introduce new identifiers
• Happy-path sequences that only mention registered scopes
• Cross-links to conceptual pages that already exist in the repo

What a human must own

• Scope identifiers and the operations they authorize
• Default-deny statements
• Rate limits, quotas, pagination defaults, and timeouts
• Token lifetime, refresh rules, and audience values
• Public vs partner vs internal classification
• Deprecation or breakage of a scope

The owned class is small. It is also the class that pages on-call when it is wrong.

A registry the generator may not extend

Keep a file the model can read and must not edit. Humans change it through the same review path as IAM code. Numbers without a source path are rumors. Clients hard-code rumours.

# auth-registry.yaml
# Owner: identity-platform. Regenerators may read, never append.
scopes:
  - id: orders:read
    grants: ["GET /v1/orders", "GET /v1/orders/{id}"]
  - id: orders:write
    grants: ["POST /v1/orders", "PATCH /v1/orders/{id}"]
  - id: orders:refund
    grants: ["POST /v1/orders/{id}/refunds"]

numeric_contracts:
  - key: rate_limit.merchant_default
    value: 100
    unit: requests_per_minute
    source: infra/gateway/ratelimit.yaml
  - key: timeout.orders_get
    value: 2.5
    unit: seconds
    source: services/orders/config.yaml
  - key: page_size.default
    value: 50
    unit: items
    source: libs/api/pagination.go

forbidden_claims:
  - "unlimited"
  - "no rate limit"
  - "full admin access"
  - "by default all users"

Cite a number by key, not by vibes. A passing sentence names a registered scope and points at a contract:

`GET /v1/orders` requires `orders:read`.

The merchant default rate limit is 100 requests per minute
[contract:rate_limit.merchant_default].

A failing sentence is the one that reached the partner:

Use `invoices:write:all` for bulk exports. There is no rate limit on this route.

Both are easy for a model to emit. Both should be impossible to merge.

Proposed workflow

This is a proposed pipeline, not a measured production study. The order matters more than the tooling.

1. Extract paths and security schemes from openapi.yaml.
2. Render a prompt that includes the registry as an allow-list, not as a suggestion.
3. Ask the model only for draftable sections, one operation at a time.
4. Concatenate drafts under headings humans already registered.
5. Run a lexical checker in CI. Fail the build on unknown scopes or uncited numbers.
6. Require a human approval on any registry diff, same as an IAM change.

Step 3 does not design authorization. It narrates an operation using names you already supplied. That is the only job a general model should have on this page.

Prompt shape (proposal)

Do not paste live tokens into the prompt. Pass identifiers only. The useful failure mode is a short refusal, not a confident new scope.

You draft API guide prose for one operation.
You may use only these scope ids: {scope_ids}
You may cite only these numeric contracts by key: {contract_keys}
If a permission or number is not in those lists, write UNREGISTERED and stop.
Do not invent scopes, roles, rate limits, timeouts, or page sizes.
Do not wrap the prose in JSON fences.

Operation:
{method} {path}
OpenAPI summary: {summary}
Registered scopes for this operation: {op_scopes}

UNREGISTERED is cheaper than a weekend of partner support. Keep the allow-list in git so two regenerations cannot drift apart.

A checker you can run locally

The script below is a proposed CI gate. It is lexical. It will not prove that your gateway enforces the matrix. It will prove that the markdown did not introduce identifiers the registry does not know.

#!/usr/bin/env python3
"""authdoc_check.py — fail if API markdown invents scopes or numbers."""

from __future__ import annotations

import argparse
import re
import sys
from pathlib import Path

import yaml

SCOPE_RE = re.compile(r"\b([a-z][a-z0-9_]{1,20}:[a-z][a-z0-9_:]{1,40})\b")
NUMBER_HINT_RE = re.compile(
    r"(?i)(rate\s*limit|quota|timeout|page\s*size|ttl|throttle|"
    r"requests?\s*per|retry-after)\D{0,40}(\d+(?:\.\d+)?)"
)
CITATION_RE = re.compile(r"\[contract:([a-z0-9_.-]+)\]")


def load_registry(path: Path) -> dict:
    data = yaml.safe_load(path.read_text())
    scopes = {row["id"] for row in data.get("scopes", [])}
    contracts = {row["key"]: row for row in data.get("numeric_contracts", [])}
    forbidden = [s.lower() for s in data.get("forbidden_claims", [])]
    return {"scopes": scopes, "contracts": contracts, "forbidden": forbidden}


def iter_markdown(root: Path):
    for p in sorted(root.rglob("*.md")):
        yield p, p.read_text(encoding="utf-8")


def check_file(path: Path, text: str, registry: dict) -> list[str]:
    errors = []
    allowed = registry["scopes"]
    for m in SCOPE_RE.finditer(text):
        token = m.group(1)
        if token not in allowed:
            line = text[: m.start()].count("\n") + 1
            errors.append(f"{path}:{line}: unregistered scope `{token}`")

    for m in NUMBER_HINT_RE.finditer(text):
        snippet = m.group(0)
        window_start = max(0, m.start() - 80)
        window = text[window_start : m.end() + 80]
        if not CITATION_RE.search(window):
            line = text[: m.start()].count("\n") + 1
            errors.append(
                f"{path}:{line}: numeric claim without [contract:key] "
                f"citation: {snippet!r}"
            )
        else:
            for key in CITATION_RE.findall(window):
                if key not in registry["contracts"]:
                    line = text[: m.start()].count("\n") + 1
                    errors.append(
                        f"{path}:{line}: unknown contract key `{key}`"
                    )

    lower = text.lower()
    for phrase in registry["forbidden"]:
        if phrase in lower:
            errors.append(f"{path}: forbidden claim {phrase!r}")
    return errors


def main() -> int:
    parser = argparse.ArgumentParser()
    parser.add_argument("--registry", type=Path, required=True)
    parser.add_argument("--docs", type=Path, required=True)
    args = parser.parse_args()

    registry = load_registry(args.registry)
    errors: list[str] = []
    for path, text in iter_markdown(args.docs):
        errors.extend(check_file(path, text, registry))

    if errors:
        print("authdoc_check: FAILED")
        print("\n".join(errors))
        return 1
    print("authdoc_check: ok")
    return 0


if __name__ == "__main__":
    sys.exit(main())

Install and run it against the guide tree:

pip install pyyaml
python authdoc_check.py --registry auth-registry.yaml --docs ./docs/api
echo $?   # 1 if the draft invented a scope or an uncited number

Proposed test plan

These fixtures are enough to keep the gate honest. They are not a substitute for gateway tests.

1. good.md uses only registered scopes and [contract:] citations. Expect exit 0.
2. bad-scope.md contains invoices:write:all. Expect exit 1 and the token in the log.
3. bad-number.md contains timeout of 30 seconds with no citation. Expect exit 1.
4. forbidden.md contains unlimited. Expect exit 1.
5. Change auth-registry.yaml in a separate PR from prose-only edits. Review that PR as IAM.

Wire the same command to CI

Drafts should be cheap to regenerate. Registry rows should not be. Path filters keep the two reviews apart.

# .github/workflows/authdocs.yml
name: authdocs
on:
  pull_request:
    paths:
      - "docs/api/**"
      - "auth-registry.yaml"
      - "openapi.yaml"
jobs:
  registry-diff:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: "3.12"
      - run: pip install pyyaml
      - run: python authdoc_check.py --registry auth-registry.yaml --docs ./docs/api

If auth-registry.yaml changes, identity owners review it. If only docs/api/** changes, the checker still runs. That split is the whole method.

Where a free model and a free server belong

The draft step is batchable. One operation per call. No tool use. No production credentials in the prompt.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

MonkeyCode provides free model access and a free server option, which is enough for this loop: registry in git, prompt per operation, markdown out, checker in CI. Park the generation job on the free server if the batch should not sit on a laptop. Keep the registry and the OpenAPI file in your own repo either way. The model is not the owner of record for a scope name or a number.

Do not send customer tokens, private keys, or live IAM dumps to any hosted model. Pass allow-listed identifiers only.

Decision table

Claim in the doc	May a model draft it?	Merge gate
Restated OpenAPI summary	Yes	Optional human skim
New scope identifier	No	Checker fail
Registered scope on the wrong path	No	Human review of grants
Rate / timeout / page size with [contract:]	Yes, copied	Checker + source still in registry
Rate / timeout / page size as free prose	No	Checker fail
"Unlimited" or "admin by default"	No	Forbidden-claim fail
Deprecating a scope	No	Registry PR + identity owners

Limitations

The checker does not execute requests. It does not confirm that the gateway, the token issuer, and the service agree. Homophones are caught: order:read versus orders:read. Semantic mistakes are not: orders:read on a delete handler still looks lexical-clean.

Numbers cited to a stale source path will pass until a human updates the registry. Freshness is an ownership problem. Sampling more model output will not fix a file nobody owns.

Who should not use this approach:

• Teams with no IAM owner and no OpenAPI file. There is nothing to freeze.
• Public-policy or compliance documents that require counsel. A regex is not counsel.
• Pages that embed real credentials, customer ids, or internal hostnames in examples.
• Organizations that treat generated pages as the system of record for authorization.

If you skip the registry, you are not saving time. You are publishing an identity product the identity team did not ship.

Fluent API docs are not evidence that the permission model was reviewed. The review is the registry diff. Everything else can be regenerated. A free model is sufficient for narration. It is not sufficient for naming powers.