Generated documentation fails when a model invents purpose, guarantees, or audience guidance the repository never states. Treat every drafted sentence as a claim that must point at a file, test, or config the repo already contains. If the path is missing, the sentence is not documentation; it is speculation that should be cut before review. This article describes an extractive workflow that classifies claims, drafts only what the tree can prove, and rejects the rest with a small verifier.
The method is meant for teams that already generate README fragments, API field lists, or runbook command blocks from code. It is not a style guide and it does not score writing quality. It answers a narrower question: which sentences may a model emit, and which sentences a human must own because no file can back them.
Why extractive drafting beats plausible prose
Model drafts often sound complete because they fill missing intent with industry defaults. A handler that returns 409 becomes “safe for retries.” A YAML flag becomes “recommended for production.” Those sentences read well in a pull request and fail later when operators treat them as contract. Reviewers who only skim tone will miss the gap, because the gap is factual, not grammatical.
A cite-or-cut gate moves the check earlier. Each sentence in a generated section must carry a source pointer such as path:line or a test name. The verifier then confirms the file exists and, when a token is supplied, that the token still appears in that file. Unsupported sentences never reach the human judgment pass.
That split also keeps cheap model capacity useful. Extractive listing is repetitive and local. Intent, policy, and unsupported inference remain human work even when drafting is free.
Three claim classes, not three writing styles
Classify each candidate sentence before any model runs. The class decides whether a draft is allowed, not how polished the paragraph should sound.
- EXTRACT. The sentence restates a name, type, default, flag, status code, or command that already exists in the tree. A model may draft it only with a citation.
- DERIVE. The sentence combines two or more EXTRACT facts, such as “this flag changes that timeout.” A human must confirm the combination; a model may propose a stub, never the conclusion.
- INTENT. The sentence states why the system exists, who should run it, what is supported, or what will not change. Humans write INTENT with no model body in the file.
Keep the labels on the outline, not in the published page. Readers should see documentation. Maintainers should see a map of what was allowed to be machine-drafted.
| Claim class | Model may draft body? | Required citation | Human owns | Typical examples |
|---|---|---|---|---|
| EXTRACT | Yes, after sources resolve | File path plus token or line | Final wording only | Flag names, defaults, routes, error codes |
| DERIVE | Stub bullets only | Two or more EXTRACT sources | The causal sentence | “A implies B”, ordering, compatibility |
| INTENT | No | None accepted | Entire paragraph | Support promises, audience, “always/never” |
The table is the policy. If a sentence cannot be placed in one row, treat it as INTENT and keep the model out.
Numbered workflow
Use the same sequence on every documentation pull request that includes model output. Skipping a step reintroduces unsupported prose.
- Freeze the outline. List heading paths and mark each heading EXTRACT, DERIVE, or INTENT before any prompt runs.
- Collect source tokens. For EXTRACT headings, write the file paths and the literals the draft must mention, such as flag names or status codes.
- Draft only EXTRACT bodies. Send the model the outline, the tokens, and a hard rule: no sentence without a citation comment.
- Run the verifier. Reject the draft if any sentence lacks a source, if a path is missing, or if a token is absent.
- Human-write INTENT and confirm DERIVE. Do this in a separate commit so the generated hunk stays reviewable.
-
Empty-run the published commands. Copy every fenced command from the page into a dry-run or
--helpcheck on a throwaway environment.
The order matters. Classification before drafting prevents the model from choosing its own authority. Verification before human polish prevents reviewers from editing fiction into nicer fiction.
Artifact: a claim map and a cite-or-cut check
Store the map next to the page, not in a chat transcript. A small YAML file is enough and stays greppable in review.
# docs/_claims/http-retry.yml
page: docs/http-retry.md
headings:
- path: "HTTP retry / Flags"
class: EXTRACT
sources:
- { file: "cmd/server/flags.go", token: "-retry-max" }
- { file: "cmd/server/flags.go", token: "-retry-backoff" }
- path: "HTTP retry / Interaction"
class: DERIVE
sources:
- { file: "cmd/server/flags.go", token: "-retry-max" }
- { file: "internal/client/retry.go", token: "Backoff" }
human_owns: "whether backoff applies when retry-max is 0"
- path: "HTTP retry / Support"
class: INTENT
sources: []
human_owns: "what is supported in GA and what operators may assume"
Generated EXTRACT paragraphs should carry HTML comments the verifier can read and the renderer can ignore.
`-retry-max` caps follow-up attempts after a failed HTTP call.
<!-- source: cmd/server/flags.go token:-retry-max -->
`-retry-backoff` sets the delay strategy between those attempts.
<!-- source: cmd/server/flags.go token:-retry-backoff -->
The check script below is labeled as an unexecuted example. Adapt paths to the repo; do not treat the snippet as a measured production suite.
#!/usr/bin/env python3
"""cite_or_cut.py — reject model doc drafts that lack resolvable sources."""
from __future__ import annotations
import pathlib
import re
import sys
ROOT = pathlib.Path(__file__).resolve().parents[1]
SOURCE_RE = re.compile(
r"<!--\s*source:\s*(?P<file>\S+)\s+token:(?P<token>\S+)\s*-->"
)
SENTENCE_RE = re.compile(r"[^.\n]+\.")
def extract_blocks(markdown: str) -> list[tuple[str, str | None]]:
lines = markdown.splitlines()
blocks: list[tuple[str, str | None]] = []
i = 0
while i < len(lines):
line = lines[i].strip()
if not line or line.startswith("#") or line.startswith("<!--"):
i += 1
continue
sentence = line
source = None
if i + 1 < len(lines):
match = SOURCE_RE.search(lines[i + 1])
if match:
source = f"{match.group('file')}::{match.group('token')}"
i += 1
blocks.append((sentence, source))
i += 1
return blocks
def main(doc: pathlib.Path) -> int:
text = doc.read_text(encoding="utf-8")
failures = []
for sentence, source in extract_blocks(text):
if sentence.startswith("`") is False and "http" not in sentence.lower():
# INTENT-looking modal verbs without a source are hard failures.
if re.search(r"\b(always|never|must|should|support)\b", sentence, re.I) and not source:
failures.append(f"INTENT-like sentence lacks source: {sentence}")
continue
if not source:
failures.append(f"no citation: {sentence}")
continue
rel, token = source.split("::", 1)
path = ROOT / rel
if not path.is_file():
failures.append(f"missing file {rel} for: {sentence}")
continue
if token not in path.read_text(encoding="utf-8", errors="replace"):
failures.append(f"token {token!r} absent in {rel} for: {sentence}")
for item in failures:
print(item)
print(f"{len(failures)} cite-or-cut failures in {doc}")
return 1 if failures else 0
if __name__ == "__main__":
sys.exit(main(pathlib.Path(sys.argv[1])))
Wire it to a docs job so a generated section cannot merge on green prose alone.
python3 tools/cite_or_cut.py docs/http-retry.md
test -z "$(git diff --name-only docs/_claims)" || echo "claim map changed; re-review INTENT rows"
For command blocks, add a second empty-run that does not need a model at all.
# Proposal: extract fenced bash from the page and run --help only.
awk '/^```
bash$/{p=1;next}/^
```$/{p=0}p' docs/http-retry.md > /tmp/doc-cmds.sh
# Review the file by hand, then:
# bash -n /tmp/doc-cmds.sh
Where a free model and free server belong
Extractive drafting is repetitive: list flags, quote defaults, and attach citations. That work can sit on a free model tier if the prompt is barred from INTENT headings and the verifier is mandatory. A free server option is useful as the place that runs cite_or_cut.py and the command dry-run against a checkout, not as an unattended publisher of pages.
Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode's free model access and free server option are relevant here only as a drafting and checking lane. They do not classify INTENT, they do not certify support language, and this article does not claim named models, quotas, hardware, duration, or benchmark wins. If those lanes are unavailable, the same YAML map and verifier still apply with any local model or with a human typist filling EXTRACT rows.
Keep the product out of the page title and out of the published docs. The operational rule is the claim map. The model is a stenographer for EXTRACT rows, and the server is a cheap place to fail those rows in public CI-shaped jobs.
Limitations and who should skip this
The verifier proves presence, not correctness of meaning. A token can appear in a file for reasons unrelated to the sentence, so DERIVE and INTENT still need a person. Line numbers drift; prefer stable tokens over brittle file:line pairs unless the pipeline rewrites citations on each commit. Generated tables that summarize runtime behavior still need a test or a captured --help dump, because source comments cannot see production configuration.
Do not use cite-or-cut as a substitute for legal, security, or GA support review. Do not apply it to incident narratives, threat models, or pricing pages, where the important sentences are INTENT by definition. Do not point a model at customer data, secrets, or unpublished incidents in order to “find sources.” If the repository cannot prove a sentence, the honest output is silence, not a smoother paragraph.
Teams that publish one-page READMEs with no flags or APIs will find the map heavier than the page. Teams that already require human authorship for every heading do not need a model in the loop. In those cases, write the INTENT first and skip generation entirely.
The durable output is not more documentation. It is a shorter page whose remaining sentences can be traced to the tree, plus a clear list of paragraphs no model is allowed to touch.
Top comments (0)