AI Drafted the Docs. Your Job Is Decisions, Not Prose.
When a language model drafts documentation, the bottleneck shifts from writing to reviewing, and most review habits were built for scarce text. Teams respond by reading generated prose line by line, which spends attention on wording the model can regenerate in seconds. The better split separates labor by kind rather than by time: the model produces candidates, a script extracts decision points, and the human answers those decisions. This article shows a review-brief pipeline that turns every generated section into a small list of ownership decisions a merge gate can enforce.
Why line-by-line review fails
Documentation review traditionally assumes that text is scarce enough to justify close reading. Generated text breaks that assumption, because the cost of regenerating a paragraph approaches zero while the cost of verifying a claim stays constant. Two failure patterns appear in teams that review AI drafts the old way. The first is volume acceptance, where long documents get merged because the act of reading felt like work. The second is review theater, where humans edit punctuation while wrong parameter names survive into production.
The workflow has four steps, and each one produces a concrete artifact.
Step 1: Define ownership tiers before drafting
The pipeline starts with a YAML file that declares what each documentation section may contain. The file is the contract between the drafting model and the human reviewers, and it lives in the repository next to the docs it governs. Three tiers cover most projects, and the exact names matter less than the boundary they draw.
# docs/ownership.yaml
sections:
quickstart:
tier: model_draft # model may draft freely
api_reference:
tier: model_draft_verified # model drafts, human verifies claims
security:
tier: human_owned # no generated prose accepted
migration_guide:
tier: human_owned
model_draft sections are disposable prose that the team can regenerate at any time, so review stays light. model_draft_verified sections contain claims about behavior, parameters, or migrations, and the model must mark anything it cannot confirm. human_owned sections carry decisions about security, compatibility, and product intent, and no generated text may enter them. Each human-owned section carries a marker like <!-- owner: @handle -->, which the script treats as evidence that a person actually authored it.
Step 2: Draft with a bounded prompt
Each generation run starts from the ownership file instead of an open-ended instruction, and the prompt tells the model which sections to draft and which to leave empty. MonkeyCode's free model access keeps this drafting step inexpensive, and its free server option is relevant when the brief generator needs a scheduled home. Disclosure: This article was prepared as part of MonkeyCode's product outreach. Neither option changes the review design, because every draft remains a hypothesis until a human decides otherwise.
You are drafting docs for the repository at <repo path>.
Read docs/ownership.yaml.
Draft only sections with tier "model_draft" or
"model_draft_verified"; leave human_owned sections empty.
Never invent API names or flags. For any fact you cannot
confirm, append @UNRESOLVED in the text.
The prompt has one rule that matters: the model marks uncertainty instead of hiding it. An @UNRESOLVED marker converts a silent hallucination risk into a visible item that the review step can find mechanically. This boundary is also what the human owns, because only the human can decide whether a claim matches the product's actual behavior.
Step 3: Generate a review brief, not a rewritten document
The central artifact of this workflow is the review brief, a short list of decisions extracted from the generated document. The script below reads the ownership file, finds each registered section, and emits exactly the items that require human judgment. Running it takes seconds, so the team can regenerate the brief after every draft iteration.
# scripts/review_brief.py
import re, sys, yaml
from pathlib import Path
ownership_path = Path("docs/ownership.yaml")
doc_path = Path(sys.argv[1] if len(sys.argv) > 1 else "README.md")
ownership = yaml.safe_load(ownership_path.read_text())
doc_text = doc_path.read_text()
decisions = []
for name, rule in ownership["sections"].items():
match = re.search(
rf"^##?\s+{re.escape(name)}\s*$", doc_text, re.MULTILINE | re.IGNORECASE
)
if not match:
decisions.append(f"- [ ] {name}: section missing; decide whether it is needed")
continue
start = match.end()
end = doc_text.find("\n## ", start)
body = doc_text[start:end if end != -1 else len(doc_text)]
if rule["tier"] == "human_owned" and "owner:" not in body:
decisions.append(f"- [ ] {name}: human-owned section has no owner marker")
if rule["tier"] == "model_draft_verified":
for line in body.splitlines():
if "@UNRESOLVED" in line:
decisions.append(f"- [ ] {name}: resolve uncertain claim -> {line.strip()}")
print("# Review decisions\n")
for item in decisions:
print(item)
print(f"\n{len(decisions)} decisions open. Answer all before merge.")
Each decision line points at a concrete section and names the missing fact. An unresolved reference, an absent owner marker, or a missing section becomes a checkbox instead of a vague comment. Run the script after every draft:
python scripts/review_brief.py docs/generated/README.md > REVIEW_BRIEF.md
Step 4: Make the merge gate check decisions, not prose
Humans answer the brief by marking each box with approve, delete, or adjust, and the answers live in the same file as the decisions they resolve. A CI job then fails the build while any box remains unchecked, so the gate enforces completion rather than subjective quality. That distinction keeps the human role measurable without pretending that prose quality can be scored by a machine.
# .github/workflows/docs-review.yml (excerpt)
- name: Check open review decisions
run: |
if grep -qE '^\s*- \[ \]' REVIEW_BRIEF.md; then
echo "Open review decisions remain"
exit 1
fi
The three review modes differ on attention cost and error coverage, which explains why the decision list usually wins. Line-by-line review spends attention on wording, while the brief redirects the same attention toward claims that only a person can confirm.
| Mode | Attention cost | Catches semantic errors | Merge gate |
|---|---|---|---|
| Line-by-line review | Highest | Low, attention goes to prose | Subjective approval |
| No review | None | None | None |
| Review-brief gate | Low | Medium, targets claims | Checks decision completion |
What the gate cannot do
The review brief verifies that decisions exist, not that the decisions are correct, and a reviewer can check every box while a hallucinated API name still ships. The @UNRESOLVED marker reduces that risk, but only if reviewers actually investigate flagged lines instead of clearing them. Teams without a named documentation owner tend to treat the brief as one more backlog item, and the gate will fail builds until someone claims that work.
Who should not use this workflow
Do not adopt this pipeline for regulated documentation where every sentence must be human-authored, or for solo projects where the writer and the reviewer are the same person. Small codebases often keep their API knowledge in the team's head, and the tier file plus CI checks become overhead with no measurable return. The pipeline pays off when doc volume grows faster than review capacity, which is exactly the situation generated drafts create.
The useful question about generated documentation is not whether a human wrote every sentence, but which parts required a human decision. Once the pipeline separates drafts from decisions, the review workload shrinks to the items only a person can resolve, and the diff becomes small enough to read for real. If your team reviews model drafts, run this script on the next one and count how many decisions survive a full read.
Top comments (0)