Model-drafted documentation stays useful only when every paragraph binds to a machine-checkable source. Unbound claims read like product judgment, and that judgment still belongs to a named human owner. Review queues do not repair missing sources; they merely delay a merge that still lacks evidence. The workable gate is therefore blunt: without a citation, a model does not draft the paragraph.
Cheap generation has not reduced documentation debt. It has changed the shape of that debt into fluent pages that cannot be traced. Reference text can be expanded from OpenAPI files, protobuf comments, and failing or passing tests. Policy text cannot, because no repository object states when a feature is appropriate, what support will promise, or which breakage customers must absorb.
The failure mode: fluent pages with no source
A model will happily write a parameter table, a retry story, and a support promise in the same tone. Readers cannot see which sentences came from a schema and which sentences were inferred. Reviewers then argue about style while the dangerous material is the uncited guarantee. That mix is how generated docs become un-revertible: the page looks complete, so nobody can point at the unsupported clause.
Bindable sources share one property. A program can resolve them to a file, a pointer, or a test node identifier after the draft lands. Human-owned claims share the opposite property. They encode a decision that is not present in the tree, even when the prose is short and confident.
Bindable sources versus human-owned claims
Treat the following matrix as a drafting contract, not as a writing-quality rubric. If a section type has no resolver, it is out of scope for the model pass. If a section type has a resolver, the model may draft, and continuous integration must prove the citations still exist.
| Section type | Checkable source | Model may draft | Human must own |
|---|---|---|---|
| Path and field tables | OpenAPI or protobuf path | Yes, with citations | Review only |
| Status and error catalogs | Error enums plus assertion tests | Yes, with citations | Review only |
| Runnable request examples | Contract tests or recorded fixtures | Yes, with citations | Review only |
| Default and limit numbers | Config keys or schema default
|
Yes, with citations | Review only |
| When to use, when to avoid | None | No | Yes |
| Support, SLA, and severity language | None | No | Yes |
| Breaking-change customer impact | Changelog plus owner note | Partial facts only | Narrative |
| Security and privacy guarantees | None unless a published control ID | No | Yes |
| Architecture rationale | Existing ADR identifier | Only quoting the ADR | Interpretation |
The middle column is the entire method. A citation is not a footnote for readers; it is a merge predicate for the repository. Paragraphs that cannot satisfy that predicate are not “review-gated model drafts.” They are human drafts that happen to live in the same file.
A six-step source-binding workflow
Follow the steps in order. Skipping the inventory produces a prompt that asks the model to invent the missing policy voice, which this gate is designed to prevent.
- Inventory every heading that ships in the next docs release, and tag each heading as
bound,facts-only, orhuman. - List resolvers the repository already has: OpenAPI pointers, proto field paths, pytest node ids, ADR filenames, and config keys.
- Forbid the model from emitting a
boundparagraph that lacks a source comment immediately above it. - Allow
facts-onlyheadings to receive generated bullets only when each bullet cites a changelog entry or issue identifier. - Leave
humanheadings as stubs with an owner field, and fail the build if model output appears under those stubs. - Re-resolve every citation on pull request, and reject the change when a pointer, test name, or schema path has disappeared.
The output of step one should be a small map committed beside the docs tree. Keep it boring and explicit so a later author can see why a heading was ineligible for generation.
# docs/_source_map.yml
pages:
- path: docs/reference/widgets.md
headings:
- id: list-widgets
mode: bound
resolvers: [openapi, pytest]
- id: widget-errors
mode: bound
resolvers: [enum, pytest]
- id: when-to-use-widgets
mode: human
owner: api-experience
- id: support-expectations
mode: human
owner: support-lead
- id: 1.14-breaking-changes
mode: facts-only
resolvers: [changelog]
Citation convention the validator can parse
Use HTML comments so rendered pages stay clean while the gate still sees the binding. One comment documents one claim cluster, not an entire file. If a later edit splits a cluster, split the comments as well.
<!-- source: openapi:paths./v1/widgets.get.parameters[name=limit] -->
<!-- source: test:tests/api/test_widgets.py::test_list_widgets_honors_limit -->
`limit` is optional, defaults to 20, and is rejected above 100.
<!-- source: enum:widget.v1.WidgetError -->
<!-- source: test:tests/api/test_widgets.py::test_unknown_widget_is_404 -->
Unknown widget identifiers return `404` with code `WIDGET_NOT_FOUND`.
<!-- human-owned: api-experience -->
## When to use widgets
_Stub. Do not generate. Owner writes the qualification rules._
A free-form “Sources: the API” sentence is not a citation. The resolver has to point at an object that python or a JSON pointer library can load. If the team cannot name that object before drafting, the heading is human-owned by definition.
Validator the merge can run
The script below is a starting artifact, not a measured production benchmark. It fails closed: unknown modes, missing owners, and unresolved pointers are errors. Label it as a proposal you should run against your own schema files before adopting it.
#!/usr/bin/env python3
"""Fail docs PRs when model-drafted paragraphs lack resolvable sources."""
from __future__ import annotations
import pathlib
import re
import sys
import yaml
ROOT = pathlib.Path(__file__).resolve().parents[1]
SOURCE_RE = re.compile(r"<!-- source: ([a-z]+):([^ >]+) -->")
HUMAN_RE = re.compile(r"<!-- human-owned: ([a-z0-9-]+) -->")
MODEL_MARK = re.compile(r"<!-- generated -->")
def load_map():
return yaml.safe_load((ROOT / "docs/_source_map.yml").read_text())
def openapi_exists(pointer: str) -> bool:
spec = yaml.safe_load((ROOT / "openapi.yaml").read_text())
node = spec
for part in pointer.split("."):
part = part.replace("/", "/") if False else part
key = part
if isinstance(node, dict) and key in node:
node = node[key]
continue
return False
return True
def pytest_exists(nodeid: str) -> bool:
path, _, name = nodeid.partition("::")
text = (ROOT / path).read_text()
return f"def {name}(" in text
def changelog_exists(entry: str) -> bool:
return entry in (ROOT / "CHANGELOG.md").read_text()
RESOLVERS = {
"openapi": openapi_exists,
"test": pytest_exists,
"changelog": changelog_exists,
}
def main() -> int:
errors = []
mapping = {p["path"]: p for p in load_map()["pages"]}
for path, page in mapping.items():
text = (ROOT / path).read_text()
if MODEL_MARK.search(text):
human_headings = {h["id"] for h in page["headings"] if h["mode"] == "human"}
for heading in human_headings:
if f"# {heading}" in text.lower():
errors.append(f"{path}: generated mark near human heading {heading}")
for kind, ref in SOURCE_RE.findall(text):
fn = RESOLVERS.get(kind)
if fn is None or not fn(ref):
errors.append(f"{path}: unresolved {kind}:{ref}")
if HUMAN_RE.search(text) is None and any(
h["mode"] == "human" for h in page["headings"]
):
errors.append(f"{path}: human heading missing owner marker")
for item in errors:
print(item)
return 1 if errors else 0
if __name__ == "__main__":
sys.exit(main())
Wire it so the docs job cannot skip the map. An empty-run still loads the YAML, walks every listed path, and exits non-zero when a cited test was renamed.
python tools/check_doc_sources.py
pytest tests/api -q --collect-only
test -f openapi.yaml && python -c "import yaml,pathlib; yaml.safe_load(pathlib.Path('openapi.yaml').read_text())"
# .github/workflows/doc-sources.yml
name: doc-sources
on:
pull_request:
paths: ["docs/**", "openapi.yaml", "tests/api/**", "CHANGELOG.md"]
jobs:
bind:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: "3.12"
- run: pip install pyyaml pytest
- run: python tools/check_doc_sources.py
What the model is allowed to receive
The prompt is a filter, not a personality. Feed only the heading, the allowed resolvers, and the source excerpts. Do not feed adjacent human stubs, because those stubs are the usual contamination path into SLA language.
You draft the heading `list-widgets` only.
You may use the OpenAPI fragment and the pytest source provided below.
Every paragraph you emit must start with one or more `<!-- source: ... -->` comments.
If a sentence is not entailed by those sources, omit the sentence.
Do not write when-to-use guidance, support promises, or migration judgment.
Do not fill headings marked human-owned.
That constraint is the difference between a reference expander and a silent product manager. Teams that skip the omit rule will get complete-looking pages again, and the validator will only catch missing pointers, not extra policy.
Disclosure: This article was prepared as part of MonkeyCode's product outreach. A drafting pass that is limited to bound headings can run against MonkeyCode's free model access, and the validator plus the excerpt job can sit on the free server option so local laptops are not the only runners. Neither option changes the ownership line: unbound headings still do not go to the model, and the merge still depends on resolvers in this repository rather than on a chat transcript.
Limitations, and who should not use this
This workflow does not make conceptual guides cheaper. It makes reference surfaces safer to generate because those surfaces already have machine-readable shadows. It will not detect a schema that is wrong but still present, and it will not detect a test that asserts yesterday's behavior. Stale sources produce stale citations that still pass the gate.
Do not apply the method to legal pages, incident customer letters, pricing, or threat models. Those documents fail as documents when a model supplies the missing noun, even if a reviewer later trims adjectives. Do not apply it when OpenAPI is generated from the same draft pipeline, because the citation then points at another model output. Do not apply it when no owner name can be placed on a human heading; an unowned stub is how policy text re-enters through a later “just fill this” prompt.
Teams without a committed schema or contract-test folder should skip generation entirely and write the reference by hand. The cost of a missing citation is not a messy page. The cost is a sentence that reads like a guarantee and cannot be blamed on any file.
The durable output is not more documentation. It is a split you can enforce: models expand checkable sources, and humans keep every claim that has no object behind it. If that split is too strict for a given page, the page was never a candidate for a model draft.
Top comments (0)