Put a Kind Checker Between Extracted Facts and Published Docs

Generated API documentation fails when a model invents implications that no schema or test can support. The practical fix is a kind checker between extracted facts and the published Markdown tree. Treat that checker like a type system for claims, not like a style linter for tone. Models may render only proven restatements; humans must sign promises, recommendations, and support boundaries.

Why chat drafts leak product promises

Chat-based documentation generation treats missing evidence as a creative opportunity rather than a hard stop. The model completes a story about uptime, migrations, and after-hours support because the prompt asked for helpful prose. Reviewers then debate tone while an unsigned reliability claim lands in the getting-started guide. A kind checker rejects that class of sentence before a renderer is allowed to run.

Agent-style assistants amplify the leak because they optimize for a finished page instead of a typed claim set. The defect is not awkward wording or weak examples in the happy path. The defect is a product promise that no test, schema, or runbook can currently defend. Kind safety is therefore a release concern, not a writing preference.

Three lanes, one published tree

Keep documentation work in three lanes that share one output tree and never share write permission. The extract lane reads OpenAPI files, recorded fixtures, and command catalogs, then emits typed fact nodes with source spans. The render lane may turn only allowlisted nodes into English paragraphs, tables, and parameter lists. The sign lane is a human-owned file set for implications that no extractor can prove.

Published docs are a merge of render output and signed files, never a single chat transcript. If a heading has no kind in the manifest, the pipeline fails closed and leaves that heading unpublished. This is slower than pasting a repository into a prompt, and that slowness is the point. Unsigned claims should be more expensive than awkward phrasing.

Claim-kind decision table

Use a table as the policy artifact, not as decoration after the model has already drafted. Every heading in docs/manifest.yml must map to one kind before any renderer starts. The table below is a worked example for a public HTTP API, not a universal ontology.

Kind	Required evidence	Writer	Unsigned publish
field_restatement	OpenAPI path, method, status, JSON Schema pointer	render lane	fail
error_restatement	named error code plus fixture or schema examples	render lane	fail
command_restatement	CLI help dump or test that asserts the flag	render lane	fail
worked_example	hashed fixture body that still matches CI	render lane after hash check	fail
procedure_order	commands that exit 0 in a docs smoke job	human confirms order; model may only copy commands	fail
implication	none extractable	sign lane only	fail
recommendation	none extractable	sign lane only	fail
support_boundary	named owner in CODEOWNERS or a runbook path	sign lane only	fail

The useful rule is narrow: if evidence can be recovered from a file, the model may pretty-print it. If evidence cannot be recovered, the heading is blank until a human signs a sibling file. Do not add a fourth kind called helpful_guess. Guessing is how SLAs appear in tutorials.

A typed documentation IR

Label the following schema as a proposal you can run locally, not as a vendor format. Each fact is a node with a kind, a source span, and a payload the renderer is allowed to restate. The kind checker refuses unknown kinds, missing spans, and payloads that drifted from the source file.

# proposal: docs_ir.py — run locally as a kind checker, not as a chat wrapper
from __future__ import annotations

from dataclasses import dataclass
from enum import Enum
from pathlib import Path
from typing import Any
import json
import hashlib

RENDERABLE = {"field_restatement", "error_restatement", "command_restatement", "worked_example"}
HUMAN_ONLY = {"implication", "recommendation", "support_boundary", "procedure_order"}

class KindError(ValueError):
    pass

class ClaimKind(str, Enum):
    FIELD = "field_restatement"
    ERROR = "error_restatement"
    COMMAND = "command_restatement"
    EXAMPLE = "worked_example"
    PROCEDURE = "procedure_order"
    IMPLICATION = "implication"
    RECOMMEND = "recommendation"
    SUPPORT = "support_boundary"

@dataclass(frozen=True)
class SourceSpan:
    path: str
    pointer: str
    digest: str

@dataclass(frozen=True)
class FactNode:
    id: str
    kind: ClaimKind
    span: SourceSpan
    payload: dict[str, Any]

def sha256_text(text: str) -> str:
    return hashlib.sha256(text.encode("utf-8")).hexdigest()

def load_source(path: str) -> str:
    return Path(path).read_text(encoding="utf-8")

def check_node(node: FactNode) -> None:
    source = load_source(node.span.path)
    actual = sha256_text(source)
    if actual != node.span.digest:
        raise KindError(f"{node.id}: source digest drifted for {node.span.path}")
    if node.kind.value in RENDERABLE and not node.span.pointer:
        raise KindError(f"{node.id}: renderable node needs a source pointer")
    if node.kind.value in HUMAN_ONLY:
        raise KindError(f"{node.id}: human-only kind leaked into extract output")

def check_bundle(path: str) -> list[FactNode]:
    raw = json.loads(Path(path).read_text(encoding="utf-8"))
    nodes = []
    for item in raw["nodes"]:
        node = FactNode(
            id=item["id"],
            kind=ClaimKind(item["kind"]),
            span=SourceSpan(**item["span"]),
            payload=item["payload"],
        )
        check_node(node)
        nodes.append(node)
    return nodes

The digest is the entire source file in this minimal example, which is conservative and easy to explain in review. Production extractors can hash a canonicalized JSON Pointer region instead, but the rule does not change. If the file moved, the node is stale, and the renderer must not emit yesterday's field list.

Worked example: extract, check, render, sign

The next commands are a local tutorial for a tiny payments API, not a measured production benchmark. Put OpenAPI beside tests, then refuse to open a model prompt until kind_check exits 0. Human implication files live under docs/signed/ and are the only place recommendations may appear.

# unlabeled example layout
openapi.yaml
fixtures/create_payment.200.json
docs/manifest.yml
docs/signed/support.md
docs/signed/when-to-use.md

# docs/manifest.yml
headings:
  - id: create-payment-fields
    title: Create Payment fields
    kind: field_restatement
    source: openapi.yaml#/paths/~1payments/post
  - id: create-payment-errors
    title: Create Payment errors
    kind: error_restatement
    source: openapi.yaml#/components/responses/PaymentError
  - id: when-to-use-payments
    title: When to use this API
    kind: recommendation
    signed: docs/signed/when-to-use.md
  - id: support-hours
    title: Support boundary
    kind: support_boundary
    signed: docs/signed/support.md

# proposal: render_lane.py — pretty-print allowlisted nodes only
from __future__ import annotations

from pathlib import Path
import json

from docs_ir import RENDERABLE, check_bundle

TEMPLATE = """### {title}

- Method: `{method}`
- Path: `{path}`
- Required JSON fields: {fields}
- Documented error codes: {errors}
"""

def render_node(node) -> str:
    if node.kind.value not in RENDERABLE:
        raise RuntimeError(f"refusing to render {node.id} kind={node.kind.value}")
    payload = node.payload
    fields = ", ".join(f"`{name}`" for name in payload.get("required", []))
    errors = ", ".join(f"`{code}`" for code in payload.get("error_codes", []))
    return TEMPLATE.format(
        title=payload["title"],
        method=payload.get("method", ""),
        path=payload.get("path", ""),
        fields=fields or "(none listed in schema)",
        errors=errors or "(none listed in schema)",
    )

def main() -> None:
    nodes = check_bundle("facts.json")
    parts = [render_node(node) for node in nodes]
    Path("docs/generated/reference.md").write_text("\n".join(parts), encoding="utf-8")

if __name__ == "__main__":
    main()

A deterministic template is enough when the payload is already typed, and that is the preferred default. A model is optional in the render lane only as a printer for tables that templates make ugly, such as long oneOf branches. The printer still receives nodes, not the repository, and it still cannot see docs/signed/.

# proposal: sign_check.py — fail CI if generated files contain human-only headings
from __future__ import annotations

from pathlib import Path
import re
import sys
import yaml

FORBIDDEN = re.compile(
    r"\b(SLA|uptime|we recommend|production-ready|24\/7|guaranteed)\b",
    re.I,
)

def main() -> int:
    manifest = yaml.safe_load(Path("docs/manifest.yml").read_text(encoding="utf-8"))
    generated = Path("docs/generated").read_text(encoding="utf-8") if False else ""
    text = "\n".join(p.read_text(encoding="utf-8") for p in Path("docs/generated").glob("*.md"))
    human_titles = {
        row["title"] for row in manifest["headings"] if row["kind"] in
        {"implication", "recommendation", "support_boundary"}
    }
    for title in human_titles:
        if title in text:
            print(f"unsigned heading leaked into generated output: {title}")
            return 1
    if FORBIDDEN.search(text):
        print("generated docs contain promise-like wording; move it to docs/signed/")
        return 1
    return 0

if __name__ == "__main__":
    sys.exit(main())

The regex is a backstop, not the type system. Kind metadata remains the source of truth, and the word list only catches accidental promise verbs that escaped a restatement. Tune the list to your product language, then keep it boring on purpose.

Numbered pipeline

1. Freeze extractable inputs in version control: OpenAPI, fixtures, CLI dumps, and the heading manifest with kinds.
2. Run extractors in CI so every fact node carries a path, pointer, and digest before any prose exists.
3. Kind-check the bundle and fail on drifted digests, unknown kinds, or human-only kinds in extract output.
4. Render allowlisted nodes with a template first; call a model only when a table is unreadable as a template.
5. Merge docs/generated/ with docs/signed/ in a publish job that forbids generated files from carrying signed headings.
6. Review diffs by kind: restatement diffs should look like schema diffs, while implication diffs need a named owner.

python extract_facts.py --openapi openapi.yaml --fixtures fixtures --out facts.json
python -c "from docs_ir import check_bundle; check_bundle('facts.json')"
python render_lane.py
python sign_check.py

Those commands should be ordinary CI steps beside unit tests, not a separate writing ritual. If step three fails, nobody gets a draft to "just tidy." Tidying is how implications re-enter through the render lane.

Where a free render job fits

The render lane is the only step that can move off-box without enlarging the product contract, because it pretty-prints nodes that already passed a kind checker. MonkeyCode's free model access and free server option are relevant there as a disposable printer, not as an author of support policy. Disclosure: This article was prepared as part of MonkeyCode's product outreach. Extraction, kind checking, and signing should still run against your own repository, where OpenAPI and tests already have owners.

Keep the remote job stateless: upload facts.json, receive Markdown for renderable kinds, and discard the workspace. Do not send docs/signed/, incident runbooks, or customer names to the printer. If the printer is unavailable, templates still publish field lists, which is the correct degraded mode.

Limitations, and who should not use this

Kind checking does not make a weak schema honest, and it cannot recover intent that engineers never wrote down. OpenAPI that omits error bodies will produce thin restatements, which is preferable to invented error catalogs, but it will annoy teams that wanted narrative coverage. The digest check also fails noisily during legitimate refactors, so the extract lane must rerun in the same change as the schema edit.

Skip this pipeline if the document is an opinion essay, a design rationale, or an incident review with no extractable surface. Skip it if you cannot name an owner for support_boundary files, because the checker will only freeze blank headings. Skip it if your API is still a moving prototype with no fixture corpus, since the render lane will republish churn as if it were a contract.

Do not use a model to fill blank signed files "for now." Temporary implications become durable the first time a customer quote them. The blank heading is the honest public artifact until a human can defend the sentence in support.

What this changes in review

Reviewers should ask one question per hunk: is this restating a span, or is this committing the product. Restatement hunks can be accepted when the digest matches and the kind is renderable. Implication hunks need a person, a date, and a follow-up path when the promise breaks. That split is the entire system; the code only makes the split expensive to ignore.

If you already emit OpenAPI and fixtures in CI, adding a kind checker is a smaller change than retraining writers to prompt more carefully. Careful prompts still guess. Kind-checked nodes cannot.