DEV Community: Morgan Sun

The Model May Draft the Prose. The Repo Must Confirm the Facts.

Morgan Sun — Tue, 20 Oct 2026 09:35:13 +0000

A doc said the retry limit defaults to 5. The code said 3. The env var in the doc had been renamed two releases earlier, so anyone who copy-pasted the snippet got a silent no-op instead of an error.

Nothing about that draft looked wrong. The sentences were clean, the structure was better than what we had, and the reviewer approved it in four minutes because reading prose is easy and checking facts is work.

That is the actual failure mode with AI-drafted documentation. Not hallucinated paragraphs — unverified specifics wearing good grammar.

The last week of DEV's front page leaned hard into AI fatigue and vibe-coding regret. Fair. But the useful response isn't refusing the tool; it's narrowing what the tool is allowed to produce and making the rest mechanically checkable.

Split the doc before you generate it

Before any prompt runs, decide two things per section: is this prose the model may draft, and are the facts in it something a machine can confirm against the repo?

Sections where the answer is "no" on the second question are human-owned, always.

Section	Model drafts?	Evidence source	Why
Overview / what it does	Yes	Repo README + module docstrings	Low risk, easy to correct
Local setup steps	Yes	`scripts/`, `Makefile`, CI config	Commands can be executed
Config reference	Yes	Code that reads the env vars	Names and defaults are greppable
API surface summary	Yes, with limits	Route definitions	Paths and methods are greppable
Security model	No	Human security owner	Wrong text is a liability
Billing, quotas, limits	No	Product owner, dated source	Numbers change without code
Deprecation / sunset dates	No	Release manager	A guess here breaks customers
SLA and support tiers	No	Contracts	Not derivable from code

The rule that survives contact with reality: if the fact lives outside the repository, the model does not get to state it. No amount of retrieval fixes a source of truth you don't control.

Artifact: a claim checker that fails the build

Here is the piece I keep coming back to. A scope file declares which headings may be drafted, and a small standard-library script extracts every hard claim — env vars, CLI flags, repo paths, version strings — and requires each one to appear somewhere in the repo.

The code below is a template, and the output further down is illustrative. It runs on the Python standard library plus git grep; nothing else.

docs/ai-scope.json:

{
  "draftable": ["Overview", "Local setup", "Configuration reference", "API surface"],
  "human_owned": ["Security model", "Billing and quotas", "Deprecation", "SLA"],
  "fail_on_missing": ["env_var", "flag", "path"],
  "warn_on_missing": ["version"]
}

tools/claim_check.py:

#!/usr/bin/env python3
"""Fail a model-drafted doc that asserts repo facts the repo cannot confirm."""
import json, re, subprocess, sys
from pathlib import Path

REPO = Path(__file__).resolve().parents[1]
SCOPE = json.loads((REPO / "docs/ai-scope.json").read_text())

PATTERNS = {
    "env_var": re.compile(r"`([A-Z][A-Z0-9]*(?:_[A-Z0-9]+)+)`"),
    "flag":    re.compile(r"`(--[a-z][a-z0-9-]+)`"),
    "path":    re.compile(r"`((?:src|docs|scripts|config)/[\w./-]+)`"),
    "version": re.compile(r"\bv?(\d+\.\d+\.\d+)\b"),
}

def grep(needle: str):
    out = subprocess.run(["git", "grep", "-n", "-F", "--", needle],
                         cwd=REPO, capture_output=True, text=True)
    return [l for l in out.stdout.splitlines() if l.strip()]

def sections(text: str):
    """Yield (heading, body) for each H2."""
    current, buf = None, []
    for line in text.splitlines():
        if line.startswith("## "):
            if current is not None:
                yield current, "\n".join(buf)
            current, buf = line[3:].strip(), []
        elif current is not None:
            buf.append(line)
    if current is not None:
        yield current, "\n".join(buf)

def main(path: str) -> int:
    draft = Path(path).read_text()
    failures, warnings = [], []

    for heading, body in sections(draft):
        if any(h.lower() in heading.lower() for h in SCOPE["human_owned"]):
            failures.append(f"human-owned section present: {heading!r}")
            continue
        if not any(h.lower() in heading.lower() for h in SCOPE["draftable"]):
            failures.append(f"heading outside the draftable scope: {heading!r}")
            continue
        for kind, rx in PATTERNS.items():
            for claim in sorted(set(rx.findall(body))):
                hits = grep(claim)
                if hits:
                    print(f"OK   {kind:8} {claim:32} {hits[0].split(':')[0]}")
                elif kind in SCOPE["fail_on_missing"]:
                    failures.append(f"unverified {kind}: {claim} (section {heading!r})")
                else:
                    warnings.append(f"unverified {kind}: {claim} (section {heading!r})")

    for w in warnings:
        print(f"WARN {w}")
    for f in failures:
        print(f"FAIL {f}")
    return 1 if failures else 0

if __name__ == "__main__":
    sys.exit(main(sys.argv[1]))

Run it on the draft before a human ever reads it:

python3 tools/claim_check.py docs/drafts/configuration.md; echo "exit=$?"

Illustrative output for the retry-limit draft that started this post:

OK   env_var  LOG_LEVEL                        src/config.py
OVER env_var  RETRY_MAX_ATTEMPTS               src/retry.py
OK   flag     --dry-run                        src/cli.py
OK   path     scripts/seed.py                  scripts/seed.py
WARN version  2.4.1                            (no match)
FAIL unverified env_var: RETRY_MAX_ATTEMPTS (section 'Configuration reference')
FAIL human-owned section present: 'Billing and quotas'
exit=1

OVER is an overlay your checker will need anyway: the string exists, but not in the section the draft claims. Catching the rename is the whole point — the doc and the code disagree, and the checker refuses to let that pass.

Why a grep and not a model

Because the verdict has to be boring. A second model grading the first model's facts gives you two opinions and no evidence. git grep gives you a file, a line, and an exit code — something a reviewer can check in five seconds and a pipeline can gate on.

Use the model for the part it is genuinely good at: turning the diff, the config read, and the module docstrings into readable prose. Use the repo for the part prose cannot be trusted with.

Wiring it into a real loop

Generate into docs/drafts/ only. Never generate directly into the published tree.
Run claim_check.py; fix the draft or fix the docs, never the rule.
Human review starts at the FAIL/WARN list, so attention lands on facts instead of adjectives.
Promote to docs/ on merge; keep the draft path in the PR body for traceability.

Step 2 is where a scheduled job earns its keep. A nightly run over every draft plus every recently changed config module catches drift before a reader does — and this is where free model access and a free server option become practical rather than theoretical: the generation step is a batch job you don't want to pay per-seat for, and the verification step is cheap enough to run on a small always-on box.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. I'm describing the workflow I'd use; the project advertises free model access and a free server option, and those terms are set by its operators, so check the current ones before you depend on them.

Limitations, stated plainly

Presence is not truth. A grep hit proves the string exists, not that the sentence about it is correct. Defaults, ordering, and conditional behavior still need a human.
Monorepos produce noise. Vendored code and generated files will "confirm" claims that have nothing to do with your service. Keep an ignore list and review it monthly.
External claims are out of scope. If the doc describes a third-party API, no amount of repo grepping helps. Those sections belong in the human-owned list.
Scope files rot too. A new heading will fail the run, which is correct — but it means someone must update ai-scope.json deliberately.
Free tiers are unstable by nature. Batch jobs should degrade to "draft only, no generation" rather than blocking a release when a quota or server is unavailable.

Who should not use this

Skip it if your docs are mostly prose about why rather than how — architecture rationale, decision records, onboarding narrative. There are few hard claims to check and the overhead won't pay back.

Also skip it if your source of truth is a wiki outside the repo, or if nobody can own the scope file. An unmaintained gate becomes a skipped gate, and a skipped gate is worse than none because it looks like assurance.

If you do have a config surface, a CLI, and a repo you can grep, the split is worth the afternoon: let the model draft the setup steps and the config table, let claim_check.py argue with it, and keep the security model and the pricing page in human hands where they belong.

Docs Rot Unevenly: Rank Sections by Drift Before You Regenerate Them

Morgan Sun — Mon, 19 Oct 2026 08:20:42 +0000

A docs PR touched 41 files last quarter. Two reviewers approved it in an afternoon. Three weeks later a support engineer pasted a config example from that same page into a customer ticket, and the flag in it had been renamed twice since.

That wasn't a hallucination. The model wrote a correct sentence about the code as it existed on the day it ran. The failure was scheduling: nobody regenerated that section after the rename, because nothing in the pipeline knew the section had become expensive to leave alone.

Most AI doc pipelines I've seen obsess over the wrong axis. They argue about whether the model may write, and skip the question of when a section deserves another cycle. This post is about the second question.

The unit is the section, not the file

A page about database migrations contains one paragraph that decays weekly and five that decay yearly. Regenerating the whole file to fix the paragraph is how you get churn-heavy diffs that reviewers stop reading.

So make the section addressable. A one-line HTML comment above the heading carries the scope and the blast radius:

## Rotating the ingest key

<!-- scope: services/ingest/**, infra/keys/** | risk: high -->

1. Drain the consumer group...

The marker is inert in rendered Markdown, diffable, and greppable. scope lists the globs whose changes make this text suspect. risk is a human judgment about what happens when the text is wrong, not about how complex the text is.

Score drift with git, not with vibes

The script below ranks every marked section by how much its scope has moved in a window. It shells out to git log, so there is no index to maintain and no service to keep alive.

#!/usr/bin/env python3
"""rank_doc_drift.py - rank doc sections by staleness risk."""
from __future__ import annotations

import argparse
import json
import re
import subprocess
from dataclasses import asdict, dataclass
from pathlib import Path

MARKER_RE = re.compile(
    r"<!--\s*scope:\s*(?P<globs>[^|>]+?)\s*(?:\|\s*risk:\s*(?P<risk>low|med|high))?\s*-->"
)
HEADING_RE = re.compile(r"^(?P<level>#{2,3})\s+(?P<title>.+?)\s*$")

RISK_WEIGHT = {"low": 1.0, "med": 3.0, "high": 8.0}


@dataclass
class Section:
    doc: str
    heading: str
    globs: list[str]
    risk: str = "low"
    commits: int = 0
    files: int = 0

    @property
    def score(self) -> float:
        # Starting weights. Tune them against your own incident history.
        return round(RISK_WEIGHT[self.risk] * (self.commits + 0.5 * self.files), 2)


def parse_doc(path: Path) -> list[Section]:
    sections: list[Section] = []
    current: Section | None = None

    for line in path.read_text(encoding="utf-8").splitlines():
        marker = MARKER_RE.search(line)
        if marker and current is not None:
            current.globs = [g.strip() for g in marker["globs"].split(",") if g.strip()]
            current.risk = marker["risk"] or "med"
            continue

        heading = HEADING_RE.match(line)
        if heading:
            current = Section(doc=str(path), heading=heading["title"], globs=[])
            sections.append(current)

    return [s for s in sections if s.globs]


def git(*args: str) -> str:
    return subprocess.run(
        ["git", *args], capture_output=True, text=True, check=True
    ).stdout


def measure(section: Section, since: str) -> None:
    hashes = git("log", f"--since={since}", "--format=%H", "--", *section.globs).split()
    section.commits = len(hashes)
    if not hashes:
        return
    names = git(
        "log", f"--since={since}", "--name-only", "--format=", "--", *section.globs
    )
    section.files = len({n for n in names.split() if n})


def main() -> int:
    ap = argparse.ArgumentParser()
    ap.add_argument("docs", nargs="+", type=Path)
    ap.add_argument("--since", default="90 days ago")
    ap.add_argument("--check", action="store_true")
    ap.add_argument("--max-high", type=int, default=0)
    ap.add_argument("--json", action="store_true")
    args = ap.parse_args()

    sections = [s for doc in args.docs for s in parse_doc(doc)]
    for s in sections:
        measure(s, args.since)
    sections.sort(key=lambda s: s.score, reverse=True)

    if args.json:
        print(json.dumps([{**asdict(s), "score": s.score} for s in sections], indent=2))
    else:
        for s in sections:
            print(f"{s.score:7.2f} {s.risk:4} {s.commits:3}c {s.files:3}f  {s.heading}  ({s.doc})")

    if args.check:
        hot = [s for s in sections if s.risk == "high" and s.commits > args.max_high]
        if hot:
            print(f"\n{len(hot)} high-risk section(s) drifted past budget:")
            for s in hot:
                print(f"  - {s.doc} :: {s.heading}")
            return 1
    return 0


if __name__ == "__main__":
    raise SystemExit(main())

Run it, then gate on it:

python3 rank_doc_drift.py docs/ --since="30 days ago"

# fail the build when a high-risk section moved at all
python3 rank_doc_drift.py docs/ --since="14 days ago" --check --max-high 0

Two properties matter here. The score is a ranking, not a measurement of truth, so do not print it as a quality metric. And 0.5 * files is deliberately weaker than commit count: forty commits to one file is a stronger staleness signal than one commit to forty files.

Three lanes, three levels of trust

The score decides which lane a section enters. The risk label decides what the model is allowed to produce inside that lane.

Risk	Model may draft	Human must own	Merge rule
`low`	Full prose, rewritten freely	A read-through	1 reviewer
`med`	Prose plus a patch against current text; every literal (path, flag, default) must appear in the diff	Each literal value, verified against source	2 reviewers, diff vs code
`high`	Headings and a list of questions only	All commands, numbers, warnings, ordering	Section owner signs off; replay test required

A high section is anything where a wrong sentence causes data loss, a security decision, or a billing surprise. Auth flows, deletion paths, migration cutovers, quota behavior. Those sections are exactly where a fluent paragraph is most dangerous, because it reads like it was verified.

The prompt contract that keeps the lane honest

Give the model less freedom than you think it needs. The draft request carries four things and nothing else:

The current section text, verbatim.
The git log --patch for the scope window, truncated to the hunks that touch exported symbols or config keys.
An instruction to output a unified diff against the section, not a rewritten file.
A constraint list: no identifiers absent from the patch, no new commands, no removed warnings, no changed defaults.

Point 3 is the load-bearing one. A patch is reviewable at a glance and cannot silently delete the paragraph a support engineer depends on. A full rewrite can.

Where the free tier actually fits

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The queue script decides what to regenerate; something still has to hold the checkout, run the loop, and produce drafts. That is a small, bursty, stateless job, which is a good shape for cheap infrastructure: the operator describes MonkeyCode as offering free model access and a free server option, including a stated ten-million-token allowance.

Treat that allowance as a budget, not a guarantee. Measure your own cycle cost before you plan a schedule — a section regenerate-and-diff cycle on a medium doc page plausibly lands in the low thousands of tokens, which means the arithmetic is about how often you sweep, not about whether you can. Verify the current terms rather than designing around a number you read in a post, including this one.

A nightly loop on the free server is ~30 lines:

set -euo pipefail
git clone --depth 200 "$DOCS_REMOTE" work && cd work
python3 rank_doc_drift.py docs/ --since="7 days ago" --json > queue.json
python3 drain_queue.py queue.json --lane-budget low=20,med=5

drain_queue.py is yours to write: take the top N per lane, call the model, write patches to a scratch branch, open one PR per lane. Keep the three lanes as three separate PRs so a high review never blocks a low typo fix.

A test plan you can run this week

The script is easy. Trusting it is the work.

Replay. Take last quarter's doc commits. For each regenerated section, check whether the pre-regeneration text was already wrong at that commit. The ratio you get is your baseline staleness rate — record it, don't estimate it.
False demand. Count sections the ranker flags that a human reviewer judges fine as-is. A high false-demand rate means your globs are too broad, not that the model is bad.
Time to detection. For each real doc error found, how many days between the code change and the fix? This is the number the whole workflow exists to shrink.
Literal accuracy. In med sections, sample the changed literals and verify against source. Report it as a count, not a percentage of "accuracy" in general.

Run all four before and after enabling the loop on the same repo. Anything else is a demo.

Where this breaks

External systems. Git churn says nothing about a third-party API you document. Those sections need a dated source link and a manual review cadence, not a commit counter.
Monorepos. Unscoped globs make every section look hot. Scope to the narrowest directory that actually owns the symbol.
Docs as contract. If your docs carry legal or compliance weight, no model should produce the diff, and the reviewer is not optional.
Marker discipline. A section without a marker is invisible to the ranker, and invisible sections are the ones that rot silently. Add a lint that fails on unmarked ## headings in governed directories.
Risk inflation. If everything is high, the queue is useless. Cap high at some fraction of sections and make the label a conversation, not a reflex.

Who should skip this

If your docs live in a wiki with no git history, the ranker has no signal — start by exporting to Markdown in a repo. If the whole docs tree is under five pages, read it monthly and save yourself the pipeline. And if your team already struggles to get one reviewer on a docs PR, adding a second lane-specific reviewer will not help; fix the review capacity first.

The uncomfortable part of generated documentation isn't the writing. It's admitting that a paragraph can be accurate, approved, and still be wrong six weeks later because nobody scheduled it. Rank the sections. Then decide who signs.

If you run the ranker on your own tree, the interesting output isn't the score list — it's the risk labels you had to invent because none of the three fit. That's usually where the real documentation debt is hiding.

Anchor Coverage: A CI Gate for Docs a Model May Draft but Never Own

Morgan Sun — Sun, 18 Oct 2026 07:02:02 +0000

Consider a docs PR that adds a retry section. The prose is tidy, the table is aligned, and one line reads: "Set --retry-limit to 5 to cap retries." The flag was renamed two releases ago, and nothing in CI noticed, because the pipeline lints Markdown formatting, not meaning.

That example is illustrative, not a customer story. The shape of it is common though: the review cost of generated prose is paid by humans reading sentences one at a time, and that cost does not shrink when generation gets cheaper.

DEV discussion over the last few weeks has swung between enthusiasm for what models can produce and fatigue from reviewing the output. Both reactions point at the same missing piece — a mechanical check between "the model wrote it" and "a human approved it."

Reviewing prose does not scale, checking anchors does

Most "AI docs" workflows fail in one of two boring ways. Either the model is allowed to touch files it should not, or it is allowed to state facts nobody can trace. Both are diff-shaped problems, and diffs can be gated.

The gate I use asks a narrow question: for every drafted paragraph, is there a machine-readable artifact behind it? Not "is it correct" — that is still a human judgment. Just "is it traceable," which is a question a script can answer in under a second.

Three lanes: extract, draft, own

Split every documentation change into three lanes before any model sees the file.

Extract — facts pulled from sources that already exist: --help output, an OpenAPI schema, a config reference, exit codes in a test fixture. A script owns this lane.
Draft — connective prose that explains extracted facts in order, with examples drawn from the fixtures. A model may own this lane, inside delimited blocks only.
Own — commitments a human makes: support windows, security posture, deprecation dates, pricing, anything a reader could hold you to. A model may not enter this lane at all.

The important part is not the labels. It is that lanes 1 and 3 are enforced by tooling, so lane 2 can be regenerated freely without a reviewer having to re-audit the whole file.

The artifact: `anchor_gate.py`

Drafts are written only between markers. Every fact inside a draft block must carry an anchor like {{claim:cli.retry-limit}}. The gate resolves anchors against claims.json and refuses naked numbers.

#!/usr/bin/env python3
"""anchor_gate.py - fail the build when drafted prose outruns its evidence.

Usage:
    python anchor_gate.py docs/ claims.json --max-orphans 0.20

Exit codes: 0 = pass, 1 = gate failed, 2 = bad input.
"""
from __future__ import annotations

import json
import re
import sys
from pathlib import Path

START = re.compile(r"<!--\s*draft:start\s+id=([\w.-]+)\s*-->")
END = re.compile(r"<!--\s*draft:end\s*-->")
ANCHOR = re.compile(r"\{\{claim:([\w.-]+)\}\}")
NUMBER = re.compile(r"\d[\d,_]*(?:\.\d+)?")


def draft_blocks(text: str) -> list[tuple[str, list[str]]]:
    blocks, block_id, buf = [], None, []
    for line in text.splitlines():
        start = START.search(line)
        if start:
            block_id, buf = start.group(1), []
            continue
        if END.search(line) and block_id is not None:
            blocks.append((block_id, buf))
            block_id = None
            continue
        if block_id is not None:
            buf.append(line)
    return blocks


def paragraphs(lines: list[str]) -> list[str]:
    out, buf = [], []
    for line in lines + [""]:
        if line.strip():
            buf.append(line.strip())
        elif buf:
            out.append(" ".join(buf))
            buf = []
    return out


def main() -> int:
    if len(sys.argv) < 3:
        print(__doc__)
        return 2
    docs_dir, claims_path = Path(sys.argv[1]), Path(sys.argv[2])
    claims = json.loads(claims_path.read_text(encoding="utf-8"))
    budget = 0.20
    if "--max-orphans" in sys.argv:
        budget = float(sys.argv[sys.argv.index("--max-orphans") + 1])

    total = orphans = 0
    errors: list[str] = []

    for md in sorted(docs_dir.rglob("*.md")):
        for block_id, lines in draft_blocks(md.read_text(encoding="utf-8")):
            for para in paragraphs(lines):
                total += 1
                ids = ANCHOR.findall(para)
                unknown = sorted({i for i in ids if i not in claims})
                if unknown:
                    errors.append(f"{md.name}:{block_id}: unknown claim(s) {unknown}")
                if not ids:
                    orphans += 1
                    if NUMBER.search(para):
                        errors.append(
                            f"{md.name}:{block_id}: number without an anchor -> {para[:60]!r}"
                        )

    rate = orphans / total if total else 0.0
    print(f"draft paragraphs: {total}  orphans: {orphans}  rate: {rate:.2f}")
    if rate > budget:
        errors.append(f"orphan rate {rate:.2f} exceeds budget {budget:.2f}")
    for err in errors:
        print(f"FAIL {err}")
    return 1 if errors else 0


if __name__ == "__main__":
    sys.exit(main())

The orphan budget matters more than the hard errors. Prose always contains sentences with no fact in them — transitions, framing, a definition. A 20% ceiling keeps the gate from becoming a lint that everyone disables in week two, while still catching the paragraph that quietly invents a default.

Generate `claims.json`, do not type it

A hand-maintained claims file rots exactly as fast as the docs it is meant to protect. Derive it from the same artifacts the code already trusts.

{
  "cli.retry-limit": { "value": "3", "source": "app.py --help" },
  "schema.timeout_ms": { "value": "1500", "source": "openapi.yaml#/components/schemas/Retry" }
}

A twenty-line extractor over argparse help text and schema keys is usually enough to start. If a fact cannot be extracted, that is a signal: either it belongs in the own lane, or the source of truth is missing and the doc should not claim it yet.

The list a model must never write

Write this list into the repo as human_owned.txt and have the generator refuse to touch those sections.

Content	Model may draft?	Human owner
Flag names, defaults, exit codes	Yes, anchored	Extraction script
Example invocations from fixtures	Yes, anchored	CI fixture suite
Deprecation and support windows	No	Product or support owner
Security, data retention, compliance	No	Security owner
Pricing, quotas, licensing	No	Whoever signs the contract
"Why we chose this design" narrative	Draft only, must be edited	The engineer who decided

That last row is where most teams get it wrong. A model can produce a plausible rationale for a design it has never seen, and plausible rationale is the hardest kind of wrong to review.

Running the draft pass where it costs nothing to re-run

Regeneration is the whole point of this workflow: when the schema changes, you re-draft instead of hand-patching. That only stays true if the draft pass is cheap enough to run on every PR rather than monthly.

MonkeyCode is one option here — the project is described as open source, with free model access and a free server option, which is what makes a per-PR draft pass financially uninteresting to argue about. The operator of this account states a free tier of 10 million tokens; treat that as a starting figure to verify against the project's current terms, not a capacity plan.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The free server option fits the other half of the loop: the gate and the extractor need somewhere to run as a check. A draft job that calls a model, writes only inside marked blocks, then runs anchor_gate.py is a small, boring workload — which is exactly what you want on a free box rather than on the cluster that runs production traffic.

Estimate your own burn instead of trusting a round number. Token counts vary by tokenizer, but a rough planning heuristic for English prose plus code is four characters per token:

def rough_tokens(text: str) -> int:
    # planning heuristic only; measure with your model's tokenizer
    return len(text) // 4

Run that over the prompts for your five largest docs, multiply by regenerations per week, and you have a number you can defend. If it does not fit a free tier, gate the draft pass to changed pages only — the extractor already tells you which anchors moved.

A four-case test plan for the gate itself

A gate nobody tests becomes a gate that passes everything.

Clean block — one paragraph, one valid anchor, no numbers outside it. Expect exit 0.
Unknown anchor — {{claim:cli.ghost-flag}} not in claims.json. Expect exit 1 with the block id.
Naked number — "retries default to 5" with no anchor. Expect exit 1.
Honest orphan — a framing sentence with no facts, budget set to 0.0. Expect exit 1 only on the rate line, which proves the budget is actually read.

Add a fifth once you trust it: regenerate the same block twice with the same inputs and diff the output. Non-determinism is not a bug, but a block that changes wording every run will churn your git history and train reviewers to ignore the diff.

Limitations, and who should skip this

Anchor coverage measures traceability, not truth. A wrong extractor produces confidently anchored lies, and the gate will pass them.

The approach also assumes a machine-readable source of truth exists. If your product's behavior is defined by a PDF, a support inbox, or one person's memory, the extract lane is empty and this workflow has nothing to stand on.

Skip it if your docs are primarily narrative — architecture decisions, incident write-ups, tutorials where the value is the author's judgment. And skip it if a formal review process already binds every sentence to a signer; adding anchors on top is duplicate work with extra syntax.

Start with one directory

Pick the docs directory with the most flags and defaults, write the marker comments, and run the gate in report-only mode for a week. The number it prints — orphan rate — is the most useful documentation metric I know of that a script can compute without understanding a single sentence.

If you try the per-PR draft pass, run it on a small page set first and keep the human-owned list longer than you think you need.

Who Owns This Paragraph? An Ownership Manifest for AI-Drafted Docs

Morgan Sun — Thu, 15 Oct 2026 11:16:47 +0000

Here is a scenario that repeats in most repositories that let a model write documentation. A pull request adds a runbook section: rotate the API credential, then restart the worker. The diff is clean, the headings match the style guide, and the reviewer approves it in ninety seconds.

Six weeks later the steps are wrong. Rotation moved behind a queue, and the text now describes a procedure that no longer exists. Nobody can name the person who was supposed to catch that. The paragraph has an author. It has no owner.

That gap is the interesting part. Style is cheap to automate; accountability is not. What follows is a workflow that keeps generated prose draftable while keeping ownership human, plus the check that enforces the boundary in CI.

Generated docs rarely fail on prose

They fail on four smaller things:

Unverified claims. A number, endpoint, or step that was plausible in the draft and never checked against code.
Missing ownership. No named person or team is on the hook when the claim goes stale.
Silent drift. The source of truth moved; the document did not.
Unbounded scope. The model was allowed to write about customer impact, pricing, or compliance because nobody drew the line.

Better prompts do not fix any of these. A contract next to the document does.

Classify blocks, not whole files

Treat a document as a set of addressable blocks. Each block gets one of three classes:

draftable — the model writes the first pass. Required evidence: a source_ref pointing into code, config, or schema.
attestable — the model drafts a skeleton, a human confirms each step actually ran. Required evidence: a human_attestation reference.
prohibited — the model emits nothing here. Legal, pricing, customer impact, incident postmortems, and anything with an external commitment live in this class.

A three-state enum is boring on purpose. It is easy to review and easy to test, and both of those matter more than expressiveness.

Mark blocks in the source

Blocks are delimited with HTML comments so they stay invisible when rendered:

<!-- block:rotation-steps -->
1. Drain the worker pool.
2. Rotate the credential in the secret store.
3. Restart the worker and confirm the health endpoint.
<!-- /block -->

The manifest then lists every block, its class, its owner, and its evidence:

# docs/ownership.yaml
version: 1
documents:
  - path: docs/ops/rotate-api-credentials.md
    blocks:
      - id: overview
        class: draftable
        owner: '@ops-docs'
        evidence: source_ref
        ref: 'services/auth/src/rotate.ts#L40-L96'
      - id: rotation-steps
        class: attestable
        owner: '@ops-oncall'
        evidence: human_attestation
        ref: 'ops-review-log#R-2214'
      - id: customer-impact
        class: prohibited
        owner: '@support-lead'
        evidence: human_attestation
        ref: 'reachability-review#R-88'

Paths and review-log identifiers above are illustrative placeholders; substitute your own.

The gate

One script is enough to turn the manifest into a failing check:

#!/usr/bin/env python3
'''check_ownership.py — fail when a docs block has no owner or no evidence.'''
from __future__ import annotations

import json
import sys
from pathlib import Path

import yaml  # pip install pyyaml

REQUIRED = {'id', 'class', 'owner', 'evidence'}
CLASSES = {'draftable', 'attestable', 'prohibited'}
EVIDENCE = {'source_ref', 'command_output', 'human_attestation'}
NEEDS_REF = {'draftable', 'attestable'}


def block_ids(markdown: str) -> set[str]:
    '''Collect block markers: <!-- block:id --> ... <!-- /block -->'''
    found = set()
    for raw in markdown.splitlines():
        line = raw.strip()
        if line.startswith('<!-- block:') and line.endswith('-->'):
            found.add(line[len('<!-- block:'):-3].strip())
    return found


def check(manifest_path: Path) -> list[str]:
    errors: list[str] = []
    data = yaml.safe_load(manifest_path.read_text(encoding='utf-8'))
    if data.get('version') != 1:
        errors.append('manifest: unsupported version')

    for doc in data.get('documents', []):
        path = Path(doc['path'])
        if not path.exists():
            errors.append(f'{path}: declared but missing')
            continue

        declared = {b['id']: b for b in doc.get('blocks', [])}
        present = block_ids(path.read_text(encoding='utf-8'))

        for orphan in sorted(present - declared.keys()):
            errors.append(f'{path}: block {orphan!r} missing from manifest')
        for ghost in sorted(declared.keys() - present):
            errors.append(f'{path}: manifest declares {ghost!r}, no such block')

        for bid, block in declared.items():
            missing = REQUIRED - block.keys()
            if missing:
                errors.append(f'{path}#{bid}: missing {sorted(missing)}')
                continue
            cls = block['class']
            ev = block['evidence']
            if cls not in CLASSES:
                errors.append(f'{path}#{bid}: unknown class {cls!r}')
            if ev not in EVIDENCE:
                errors.append(f'{path}#{bid}: unknown evidence {ev!r}')
            if cls in NEEDS_REF and not block.get('ref'):
                errors.append(f'{path}#{bid}: class {cls} needs a ref')
    return errors


if __name__ == '__main__':
    target = Path(sys.argv[1] if len(sys.argv) > 1 else 'docs/ownership.yaml')
    failures = check(target)
    print(json.dumps({'failures': failures}, indent=2))
    sys.exit(1 if failures else 0)

A pytest wrapper makes it part of the normal test run:

# tests/test_docs_ownership.py
import subprocess
import sys


def test_ownership_manifest_is_satisfied():
    result = subprocess.run(
        [sys.executable, 'tools/check_ownership.py', 'docs/ownership.yaml'],
        capture_output=True,
        text=True,
    )
    assert result.returncode == 0, result.stdout + result.stderr

And the CI job is small enough to read in one screen:

name: docs-ownership
on:
  pull_request:
    paths:
      - 'docs/**'
      - 'tools/check_ownership.py'
      - 'tests/test_docs_ownership.py'
jobs:
  check:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: '3.12'
      - run: pip install pyyaml pytest
      - run: python tools/check_ownership.py docs/ownership.yaml
      - run: pytest -q tests/test_docs_ownership.py

The decision table

Block class	Model may	Human must	Evidence key	CI behaviour
draftable	write the first pass	verify every factual claim against `ref`	`source_ref`	fails without `ref`
attestable	draft the skeleton only	run each step and record the attestation	`human_attestation`	fails without `ref`
prohibited	nothing	write it or approve it explicitly	`human_attestation`	review only

One honest limit: detecting that a model touched a prohibited block requires your generator to record provenance. If it cannot, drop that claim and let review carry it rather than pretending the check proves it.

Where a free tier actually helps

The manifest makes the draft step cheap and the gate non-negotiable. That split is where free infrastructure helps instead of getting in the way.

Drafting runs are high-volume and low-risk: one call per block, output only ever landing in draftable and attestable blocks. MonkeyCode's free model access fits that shape, because the artifact is still a first pass a human signs.
The gate is the opposite: short, deterministic, and it has to run on every contributor's machine or in CI. A free server option lets you run the checker from a hosted environment instead of asking each contributor to install Python and PyYAML locally.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Two caveats that matter more than the convenience. Quotas, limits, and availability on any free tier change, so check the current plan before routing a pipeline through it, and never treat a free tier as a production SLA. A cheap draft does not reduce what the human still owns.

Limitations and who should not use this

If your docs come from a schema, generate them instead. OpenAPI, protobuf, and Terraform can emit reference pages with no ownership manifest at all. Manifests are for prose.
If a document has exactly one maintainer, the manifest is overhead. A single owner field in front matter is enough.
If you cannot name an owner for a block, that is the finding. Do not invent a name to make CI green; mark the block prohibited and escalate.
Regulated or contractual text — SLOs, security commitments, pricing — should not sit in a generated pipeline no matter how good the check is.
The checker is structural, not semantic. It proves a block has an owner and that a reference exists. It cannot prove the reference still says what the paragraph claims; that needs a link check and a human read.

Start with three blocks

You do not need to retrofit an entire docs tree. Pick one runbook, mark three blocks, write three manifest entries, and run the script once. If it passes on the first attempt, the manifest is usually too vague to be useful.

The rule worth keeping is short: the model may draft anything a human will read, and it may never own the sentence someone will act on. Keep the gate yours, and a free first pass is a reasonable place to begin.

Allowlist the Command Before the Generated Runbook Ships

Morgan Sun — Tue, 13 Oct 2026 00:40:32 +0000

A support page for a stuck queue worker reached review in a composite scenario, not a customer report. The draft read cleanly. It promised recovery within five minutes, named a restart command, and assigned no owner.

That is the failure this workflow targets. Fluent prose is not a binding. A binding is a claim a person will stand behind after the page is published.

What breaks when draft and binding share one file

Generated troubleshooting pages mix two kinds of text. Descriptive lines are cheap to regenerate. Commitments change what on-call will run, or what a reader is told.

Reviewers who read for tone miss the second kind. The page looks finished because the sentences are finished. The operational commitments are not.

Three unsigned items cause most of the damage in this pattern. A time promise has no measurement behind it. A shell command was never added to an approved set. An escalation owner is left as "the team" or "TBD."

None of those require a new model. They require a split, then a check that refuses to publish when the split is violated.

What the model may draft

Treat model output as a proposal bundle. It can save a first-pass hour, and it is still unsigned. If a sentence would change an action or a promise, it does not belong in that bundle.

The draft lane may include:

A symptom summary, marked source: model and claim_type: description.
Candidate checks, each with approval: pending and no production command.
A glossary line that cites an existing doc id, instead of inventing a definition.
A suggested outline: detect, contain, verify, escalate.
Synthetic log lines, labeled synthetic: true, so they are not pasted into an incident channel as evidence.

The model may not select severity, customer impact, or the command production is allowed to run. Those fields stay empty until a human writes them in a separate file.

What a human must own

A named person owns the binding ledger. A bot identity does not. The author of the prompt does not, unless that same person reviews the ledger under a normal change process.

Binding	Human owns it because	Reject if
`severity`	It pages people and sets tempo	Empty, or any of `todo`, `tbd`, `model`
`customer_promise`	It is an external commitment	A time window with no verifier
`escalation_owner`	Someone must answer when a step fails	A nickname with no individual id
`environment_scope`	Commands are not universal	Missing, or `all` without an exception list
`verified_by` / `verified_at`	Freshness is a recorded fact	Absent, or older than your own policy window
`command` + `approval`	Execution is the hazardous part	Approval is not `human-approved`, or the string is off the allowlist

The human does not have to write the symptom paragraph. They do have to refuse publication until the table is complete. That is a smaller job than rewriting the page, and it is the job that stops a confident overclaim.

A checker you can run before publish

The files below are a proposal. They were not executed against a production repository for this article. Copy them into a scratch directory if you want a local result.

runbook.json:

{
  "title": "Queue worker stuck in draining",
  "bindings": {
    "severity": "sev-3",
    "customer_promise": "none — internal only",
    "escalation_owner": "oncall-queue-primary",
    "environment_scope": "staging",
    "verified_by": "docs-owner",
    "verified_at": "2026-10-13T00:00:00Z"
  },
  "steps": [
    {
      "id": "s1",
      "source": "model",
      "claim_type": "description",
      "approval": "pending",
      "text": "Workers may report draining when the consumer loop stops fetching."
    },
    {
      "id": "s2",
      "source": "human",
      "claim_type": "check",
      "approval": "human-approved",
      "command": "systemctl is-active queue-worker"
    }
  ]
}

allowlist.json:

{
  "commands": [
    "systemctl is-active queue-worker",
    "curl -fsS http://127.0.0.1:8080/health"
  ]
}

check_runbook.py:

#!/usr/bin/env python3
"""Proposal checker: unsigned bindings and off-allowlist commands fail the run."""

from __future__ import annotations

import json
import sys
from pathlib import Path

OWNED = (
    "severity",
    "customer_promise",
    "escalation_owner",
    "environment_scope",
    "verified_by",
    "verified_at",
)
PLACEHOLDERS = {"", "todo", "tbd", "unknown", "model", "n/a"}


def load(path: Path) -> dict:
    return json.loads(path.read_text(encoding="utf-8"))


def check(doc: dict, allow: dict) -> list[str]:
    errors: list[str] = []
    bindings = doc.get("bindings") or {}
    for key in OWNED:
        value = bindings.get(key)
        text = value.strip().lower() if isinstance(value, str) else ""
        if text in PLACEHOLDERS:
            errors.append(f"missing human binding: {key}")
    allowed = set(allow.get("commands") or [])
    for step in doc.get("steps") or []:
        command = step.get("command")
        if not command:
            if step.get("claim_type") == "promise" and step.get("source") == "model":
                errors.append(f"model authored a promise in {step.get('id')}")
            continue
        if step.get("approval") != "human-approved":
            errors.append(f"command not human-approved: {command}")
        if command not in allowed:
            errors.append(f"command outside allowlist: {command}")
    return errors


def main() -> int:
    if len(sys.argv) != 3:
        print("usage: check_runbook.py RUNBOOK.json ALLOWLIST.json", file=sys.stderr)
        return 2
    errors = check(load(Path(sys.argv[1])), load(Path(sys.argv[2])))
    if errors:
        print("\n".join(errors))
        return 1
    print("bindings present; commands allowlisted")
    return 0


if __name__ == "__main__":
    raise SystemExit(main())

Run the proposal checker with:

python3 check_runbook.py runbook.json allowlist.json

A passing run prints one line and exits 0. Change approval on s2 to pending, or point command at any string that is not on the allowlist, and the process should exit 1. The checker does not debate whether a command is clever. It only asks whether a human already named that exact string.

How to wire the split

Keep generation and ownership in separate files. Mixing them makes review look like copy editing, which is how unsigned promises survive.

Generate or hand-write draft.md with symptom text only. No severity, no recovery window, and no shell text that is not already on the allowlist.
Store bindings in runbook.json. A human edits that file. The generation step does not get write access to it.
Keep allowlist.json in the repository that owns production change control. Updates use the same review as other operational changes.
Run check_runbook.py on every documentation pull request. Fail the job when the exit code is 1.
Publish the rendered page only from a commit where the checker passed, and where verified_by matches the code owner for that path.

If a step needs a new command, stop and add that command to the allowlist in a reviewed change first. Then reference it. Do not let a draft introduce execution text "just for the example."

Where a free draft lane fits

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The checker does not need a model. The draft lane does, if you want proposal wording without spending the human hour on a blank page. Operator-supplied availability for that lane is free model access plus a free server option: enough, as stated, to draft draft.md and run this Python check in one scratch place. This article does not name models, quotas, hardware, or a time window. Those details are not established here, and they should be rechecked before anyone depends on them. Use the server as a scratch runner for the checker, not as a production control plane.

Remove the product and the ledger still stands. The split is the method. The host is optional.

A short test plan

Use this plan before you trust the script in CI. It is a plan, not a recorded lab result, and it does not prove that a passing page is operationally correct.

Fixture A: the sample JSON above. Expect exit 0.
Fixture B: set customer_promise to tbd. Expect exit 1 and a missing-binding line.
Fixture C: set s2.approval to pending. Expect a not-approved line.
Fixture D: add a step whose command is curl -fsS http://127.0.0.1:8080/health. Expect exit 0, because that exact string is allowlisted.
Fixture E: add systemctl restart queue-worker even with human-approved. Expect exit 1 until the allowlist changes in review.
Fixture F: a model step with claim_type set to promise and no command. Expect exit 1.

Record the six exit codes in the pull request. Do not record a model confidence score. The score is not a binding, and it should not sit next to verified_by.

Limitations

The script matches strings. It will not notice a promise hidden in text, such as a recovery window written as ordinary prose, unless you add a separate phrase check. That extension is still not proof that the prose is true.

An allowlist can ratify a bad command. Human approval is a control, not a guarantee. If reviewers stamp the file without reading the command, the checker will faithfully ship the stamp.

verified_at is not compared with a clock in the sample. Add a maximum age in your own policy if you need one. Do not copy a freshness number from an article and treat it as your service objective.

Free access, if you use it, is not a capacity plan. No throughput, retention, or uptime figure is claimed here. A scratch run that passes today says nothing about next month's offer.

Who should skip this

Skip it if nobody can be named in verified_by. A ledger without an owner is a form with empty signature lines.

Skip it for live incident command. This is a pre-publish gate, not a console for an outage. During an incident, use the bridge process you already have.

Skip it for customer legal notices, security advisories, and any page that would include exploit detail or credential handling. Those need specialist review. A JSON allowlist is the wrong instrument, and this workflow must not be stretched into attack procedures.

Skip it if the goal is for a model to choose severity or invent a recovery time. That goal is the opposite of the split.

Close

Write the symptom in the draft. Put the promise, the owner, the scope, and the command in the ledger. Publish when the checker exits 0, not when the paragraph sounds done.

If you want a scratch place to try that draft lane before you attach it to CI, the free model access and free server option noted above are enough to start. Recheck what is actually offered on the day you try it, and keep the binding file under human review either way.

Classify the Environment Before the Walkthrough Is Drafted

Morgan Sun — Sun, 11 Oct 2026 21:09:59 +0000

Liquid syntax error: Unknown tag 'endraw'

Draft the Migration Note. Refuse the Unsigned Number.

Morgan Sun — Sat, 10 Oct 2026 17:38:07 +0000

Liquid syntax error: Unknown tag 'endraw'

The Model May Draft the Example. It May Not Bind the Client.

Morgan Sun — Fri, 09 Oct 2026 13:40:06 +0000

Consider a generated reference page that tells callers to retry HTTP 429 for 30 seconds. The OpenAPI file lists 429 as a possible status. It does not define a wait, a budget, or a guarantee. The sentence is fluent, and it is also a contract the service never signed.

That mix is easy to miss once a model writes the whole page. Examples and obligations share one voice. Readers cannot tell which lines illustrate a fixture and which lines bind a client.

This workflow marks the difference before generation starts. A model may draft illustrative text. A human must own every binding sentence.

Two sentence classes, one page

Treat each paragraph as one of two classes before you ask for a draft.

Illustrative text shows a shape. It can include a sample request, a short walkthrough, or a field gloss that restates the schema. It can be regenerated when the fixture changes.

Binding text creates an obligation. Modal verbs such as MUST and SHALL do this. So do invented status rules, retry windows, rate numbers, deprecation dates, auth lifetimes, and availability promises. Those lines survive a regen only if a named owner re-approves them.

Mixing the classes is the actual bug. A polished page can still be wrong when one binding sentence was inferred rather than copied from a source.

What the model may draft

Keep the model inside a marked block. Inside that block it may:

Restate a field that already exists in the schema, using plain language.
Expand a recorded HTTP fixture into a walkthrough, without adding headers or statuses the fixture does not contain.
Propose a shorter introduction for an example.
Flag awkward phrasing for a human to accept or drop.

It may not invent a status code, a quota, a clock, or a compatibility promise. If the schema is silent, the draft stays silent. Silence is a valid output.

The fixture is the source for the draft, not the model's memory of similar APIs.

{
  "request": {"method": "GET", "path": "/orders"},
  "response": {
    "status": 200,
    "body": [{"id": "o_1", "status": "paid"}]
  }
}

A faithful draft says that this call returned one paid order. It does not add a 429 branch, a retry loop, or a note about typical latency.

What a human must own

A human owner signs a separate block. That block is the only place for normative claims:

Required and forbidden client behavior.
Auth scopes and token lifetimes.
Error semantics that are not a pure copy of the schema.
Rate, retry, timeout, and deprecation statements.
Any number that is not copied from a primary spec or fixture.

Keep the owner block short. If it turns into a second essay, normative claims are hiding in prose again. Move examples back to the model block and leave the rule behind.

Markers the checker can see

HTML comments stay out of the rendered page and remain visible to a script.

<!-- claim-class: model -->
`list_orders` returned one paid order in `orders.fixture.json`.
The `status` field is the enum value `paid` from that fixture.
<!-- /claim-class: model -->

<!-- claim-class: owner -->
Callers MUST send `Idempotency-Key` on POST /orders.
A missing key is rejected with 400. No retry window is defined in this page.
<!-- /claim-class: owner -->

The first block can be rewritten on the next regen. The second block stays frozen until the owner edits it. Deleting the owner block is a failed page, not a cleanup.

A claim-class checker

The script is a proposal. It is not a measured benchmark, and it does not prove the owner block is true. It fails the check when binding language appears inside a model block, or when the owner block is missing.

#!/usr/bin/env python3
"""Fail if a model-owned block contains binding language. Proposal only."""
from __future__ import annotations

import re
import sys
from pathlib import Path

OPEN = "<!-- claim-class: model -->"
CLOSE = "<!-- /claim-class: model -->"
OWNER = "<!-- claim-class: owner -->"

BINDING = re.compile(
    r"\b(MUST|SHALL|REQUIRED|MUST NOT|SHALL NOT)\b"
    r"|retry (?:for|after) \d+"
    r"|rate limit(?:ed)? to \d+"
    r"|deprecated (?:on|as of) \d{4}"
    r"|expires after \d+"
    r"|guaranteed (?:latency|uptime)"
    r"|\bSLA\b"
    r"|returns? (?:HTTP )?status \d{3}",
    re.IGNORECASE,
)

def blocks(text: str) -> list[tuple[int, str]]:
    found = []
    start = 0
    while True:
        i = text.find(OPEN, start)
        if i < 0:
            return found
        j = text.find(CLOSE, i)
        if j < 0:
            raise SystemExit("unclosed model block")
        found.append((text[:i].count("\n") + 1, text[i + len(OPEN):j]))
        start = j + len(CLOSE)

def main(path: str) -> int:
    text = Path(path).read_text(encoding="utf-8")
    if OWNER not in text:
        print("missing owner block")
        return 2
    hits = 0
    for line, body in blocks(text):
        for match in BINDING.finditer(body):
            hits += 1
            clip = body[max(0, match.start() - 24): match.end() + 24]
            print(f"{path}:{line}: binding language in model block: {clip!r}")
    return 1 if hits else 0

if __name__ == "__main__":
    if len(sys.argv) != 2:
        raise SystemExit("usage: claim_class_gate.py PAGE.md")
    raise SystemExit(main(sys.argv[1]))

Run the checker against the page file, not against the model transcript.

python3 claim_class_gate.py docs/orders.md
echo "exit=$?"

Exit codes

Exit 0 means no binding pattern was found in model blocks and an owner block exists. Exit 1 means a binding pattern leaked into a draft. Exit 2 means nobody owns the page. Treat exit 2 as a process failure even when the prose looks finished.

Decision table

Use the table as a review aid, not as a style guide.

Sentence	Class	Who writes it	On regen
The fixture shows one paid order.	Illustrative	Model, then a skim	Rewrite allowed
Field `status` matches the fixture enum.	Illustrative, if copied	Model	Rewrite only after the fixture changes
Callers MUST send Idempotency-Key.	Binding	Human	Frozen until the owner edits
Retry 429 for 30 seconds.	Binding	Human, only if a spec says so	Reject when unsourced
Typical latency feels low.	Neither	Nobody	Delete

The last row is the one teams skip. Vague performance talk is not an example and not a rule. Delete it. Do not promote it into the owner block to make the checker quiet.

Where the draft pass runs

The split pays off only if redrafting the illustrative block is cheap. Disclosure: This article was prepared as part of MonkeyCode's product outreach. In this workflow the illustrative block can be drafted with MonkeyCode's free model access, and the checker can run on the free server option so a doc change does not need a separate paid runner.

Both points are availability claims supplied for this article. They are not a named model, a token quota, a hardware profile, or a statement that the offer is permanent. Confirm the current terms before a pipeline depends on them. The checker is ordinary Python. The server is a place to run it, not a source of truth for the API.

If you want a low-cost place to practice the split, start with one endpoint, draft only the model block, and keep the owner block in the same pull request.

Test plan

The cases below are a plan, not a report of an executed run. Apply them to a fixture repo before you trust the gate.

Illustrative block plus a real owner block: expect exit 0.
The same page with MUST inside the model block: expect exit 1 and a line reference.
No owner comment: expect exit 2, even if every sentence is accurate.
Owner block contains MUST: expect exit 0. Binding language is allowed there.
Model block says returns status 429 while the fixture has no 429: expect exit 1.
Model block copies a status line that also appears in the schema: still expect exit 1. Status rules stay with the owner so the checker does not guess intent.
Unclosed model comment: expect a hard error, not a pass.
Phrase expires after 1 hour inside the model block: expect exit 1 after the pattern list includes that shape.

Case 6 is strict on purpose. A duplicated status line looks harmless and still drifts when the schema moves. One home for it is enough.

A failure, then the fix

Put the sample sentence back into the model block: clients MUST retry 429 responses for 30 seconds. The checker returns exit 1. The fix is not to soften MUST into "should". Softening hides the same invention.

Delete the sentence if no spec defines a retry. If a spec does define one, copy that rule into the owner block and name the spec section beside it. The model block should then describe only the recorded call.

A quieter leak looks like this: authentication uses a bearer token that expires after one hour. There is no MUST. A short pattern list can miss it. The hour is still a binding number. Add the pattern once you see the leak, and keep the number in the owner block only when the auth spec states it.

That is the maintenance loop. The regex grows from leaks you observed. It does not begin as a complete grammar of obligation.

Limitations

The regex is a tripwire, not a parser. It misses soft obligations such as "clients should generally wait." It also flags harmless uses of "required" in a field gloss. Review the hits. Do not auto-delete them.

Some publishers strip HTML comments on import. If that happens, store the class in a sidecar and point the checker at the sidecar. A missing marker should fail closed.

A signed owner block can still be false. The gate checks placement, not truth. Pair it with the spec or fixture you already treat as canonical. Do not ask the model to confirm the owner block by rewriting it in a smoother tone.

Free model access will vary with page size and with whatever limits are current. Do not design the loop around an assumed token budget. If a page does not fit, split the illustrative block. Do not ask the model to compress binding rules into the draft to save space.

Who should not use this

Skip the gate when you have no schema and no fixture. The model will fill the silence with plausible obligations, and the checker catches only the loud ones.

Skip it for security advisories, legal terms, and incident reports. Those pages are binding from the first sentence. A model draft is the wrong first move.

Skip it when the same person authors, reviews, and publishes with no second read. The owner comment becomes theater. The gate helps when someone else can reject a binding sentence without rewriting the example.

Keep Replay Outcomes Out of the Generated Page

Morgan Sun — Thu, 08 Oct 2026 09:37:12 +0000

A checkout service documented Idempotency-Key on POST /payments. Overnight, a generator rewrote the page from the OpenAPI description fields. The new paragraph said a repeat call "always returns the original response body."

A merchant retried a timeout. The API answered 409 with code: replay_conflict and a short error body, not the first 201. Support quoted the page. Engineering quoted the handler.

The page lost. The retry script lost with it. This sequence is a composite, built from a familiar docs failure, not a log held by this account.

Two jobs, one URL

An API page orients the caller, and it also makes promises. Orientation is the header name, the placement, a fake sample key, and a link onward. A promise is the replay outcome: status code, body identity, key scope, retention, and whether a second call can create a second charge.

Models draft orientation well. They do not own promises. Regen makes that split sharp. Guide sentences may change every release. Contract sentences stay until a named human edits them.

The composite regen touched 11 sentences. Six were orientation. Five asserted replay behavior. Those five are the defect.

What the model may draft

Put orientation in guide.md. The model may create and replace that file.

Header placement, and a fake sample such as pay_8f3c
A request skeleton: method, path, content type
A pointer sentence: replay behavior lives in freeze.md
Name restatements that add no outcome (Idempotency-Key, 409, replay_conflict)
A "where to look" list for logs and request ids

A changelog line that says the guide was regenerated is also a draft. A changelog line that says behavior changed is not.

What a human must own

Put promises in freeze.md. A human API owner edits that file. The regen job must not have write access to it, and the prompt must not ask the model to refresh it.

The owner signs six rows. Each row names a source, or it says unspecified.

Key scope: account, merchant, or global.
Retention, and what an expired key does on the next call.
Match rule: which request parts are compared on replay.
Success replay: status code, and whether the body is the stored body.
Conflict replay: status, error code, and the charge or ledger effect.
In-flight rule: what the second caller receives while the first call is open.

unspecified may be published. A guessed "always" may not.

HTTP defines 409 as Conflict in RFC 9110. It does not require that code for idempotent replay. Choosing 409 is an API decision, so cite the handler or the spec revision in the owner mark when you choose it. See RFC 9110, status 409.

Decision table

Apply this on every idempotency docs change. The right-hand count is the composite page only. It is not a multi-repo study. Retention and status examples are illustrative claims, not measured defaults.

Sentence job	Example line	Writer	Survives regen?	Composite count
Name the header	Send Idempotency-Key on POST /payments.	Model	No	2
Show a sample	curl skeleton, fake key	Model	No	2
Point at the contract	See freeze.md for outcomes.	Model	No	2
Scope the key	Unique per account, not global.	Human	Yes	1
Retention	Keys are stored for 24 hours.	Human	Yes	1
Success replay	Same status and stored body.	Human	Yes	1
Conflict replay	409 replay_conflict, no second charge.	Human	Yes	1
In-flight	409 in_flight until the first call ends.	Human	Yes	1

Five promise rows. None of them belong in a generated description.

Worked files

Copy these as a starting proposal. They are not a dump from a production repository.

guide.md:

## Calling the endpoint

Send `Idempotency-Key` on `POST /payments`. Use a new key for a new payment intent. Reuse the key only when retrying that same intent.

Replay status codes, body identity, retention, and charging rules are not described here. Read freeze.md. If freeze.md says unspecified, do not infer a retry policy.

curl -X POST https://api.example.test/payments \
  -H "Idempotency-Key: pay_8f3c" \
  -H "Content-Type: application/json" \
  -d '{"amount": 1000, "currency": "USD"}'

freeze.md:

<!-- owner: payments-api; freeze: replay-rule; source: unspecified -->

- scope: unspecified
- retention: unspecified
- match: unspecified
- success replay: unspecified
- conflict: unspecified
- in-flight: unspecified
- charge: unspecified

A freeze file full of gaps is a valid publish. A fluent guide that closes those gaps is not.

Boundary checker

This script is a proposal. It is not a benchmark, and it has no corpus score. It fails closed on contract verbs in guide.md, and on a freeze.md with no owner mark.

#!/usr/bin/env python3
"""Proposal: flag contract verbs in generated idempotency guides."""

import re
import sys
from pathlib import Path

CONTRACT = re.compile(
    r"\b(always|guaranteed|exactly[- ]once|byte-for-byte|"
    r"identical response|safe to retry|never charges|"
    r"same body|no second charge)\b",
    re.I,
)
OWNER = re.compile(
    r"<!--\s*owner:\s*[^;>]+;\s*freeze:\s*replay-rule;\s*source:\s*[^>]+-->"
)

def main(root: Path) -> int:
    guide = (root / "guide.md").read_text(encoding="utf-8")
    freeze = (root / "freeze.md").read_text(encoding="utf-8")
    errors = []
    for i, line in enumerate(guide.splitlines(), 1):
        if CONTRACT.search(line):
            errors.append(f"guide.md:{i}: contract verb in draftable file")
    if not OWNER.search(freeze):
        errors.append("freeze.md: missing owner/freeze/source mark")
    if re.search(r"\bunspecified\b", freeze, re.I) and CONTRACT.search(guide):
        errors.append("guide asserts an outcome while freeze says unspecified")
    print("\n".join(errors) if errors else "ok")
    return 1 if errors else 0

if __name__ == "__main__":
    sys.exit(main(Path(sys.argv[1])))

python3 check_replay_boundary.py docs/idempotency
git diff -- docs/idempotency/freeze.md

The first command checks sentence class. Exit 0 with ok is expected for the honest sample. Exit 1 is expected if guide.md says "always returns the original response". The git diff is the human step: if freeze.md moved, the named owner reviews that hunk before anything else.

What the checker does not prove

A green run does not show that the service matches freeze.md. Pair the docs check with a handler test owned by the service repo. Keep that test out of the docs generator.

If production returns 200 on replay while freeze.md says 409, file a service bug. Do not rewrite the page to match production in an unattended regen.

Review order

Read the freeze.md diff. No owner, no merge.
Reject a single unattended commit that edits freeze.md and guide.md together.
Run the checker and store the exit code on the PR.
Confirm sample keys are fake. pay_8f3c is acceptable. A live key is not.
When a row stops saying unspecified, update the source: mark in that same commit.

A free drafter, used narrowly

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

MonkeyCode enters only as the drafter for guide.md. The operator describes it as an open-source project and supplied two availability notes: free model access, and a free server option. This article does not treat a token total, a model list, a machine size, or a server lifetime as verified. Those figures change.

Read the license and the current limits in the project on the day you start. Do not copy an allowance from an older post into a sprint plan.

Prompt boundary

Provide the OpenAPI operation, the header name, and a write target of guide.md only.
Require the pointer sentence to freeze.md.
Forbid outcome verbs: always, guaranteed, identical response, safe to retry.
Do not pass freeze.md as a file the model may edit.

Server boundary

On the free server, run the checker and bring the exit code back to review. The server runs a draft and a script. It does not approve a promise.

If the model emits a contract verb, delete the sentence. That draft failed. Do not soften the verb and ship it.

Limitations

The regex misses paraphrase. "The second call gives you what the first call gave you" has none of the listed verbs and is still a promise. A human still reads guide.md.
The script does not parse OpenAPI. A promise inside a description ships if you generate straight into the spec. Keep those fields empty, or set them to the pointer sentence.
unspecified can become a stall. Put a review date on any row that stays unspecified past one release.
Do not reuse this verb list for auth scopes, cursor stability, or webhook signatures. Each of those needs its own freeze file.
No cost, latency, or quality figure is claimed. This article did not run a model comparison.

Who should skip it

Skip the split when no API owner will sign freeze.md. An unsigned freeze file is a second guide. Skip it when the handler is still a spike and you cannot publish unspecified; withhold the page.

Skip it when the site generator can emit only one source file and cannot include a second. You need two sources even if the built URL is one page. Skip it for incident notes and status-page commitments. A verb list tuned for payment replay will look strict and still miss the promise that matters there.

The next docs PR

Regenerate the guide. Leave the freeze file untouched unless the owner is on the review. A checker result of ok means the page can ship as orientation. It cannot ship as a retry contract while promise rows remain unspecified.

If a free drafting setup is already on your bench, point it at guide.md only and keep the freeze file out of the write set. A green checker is a boundary result, not evidence that production replays match the page.

Draft the Header Row, Sign the Replay Window

Morgan Sun — Thu, 08 Oct 2026 07:03:52 +0000

A composite support thread, not a measured incident, shows the failure in one place. A generated reference page listed Idempotency-Key on POST /charges. The row included a type, a max length, and a UUID example. A client retried a timed-out POST with the same key and a different amount, and a second charge landed.

The page never stated scope, retention, or the mismatch status. Every cell the schema could fill was filled. The cells that were absent were the ones support had to invent on a call.

That pattern is common in generated reference docs. Generators copy columns well. They do not own product promises.

Two layers in one header

Idempotency documentation is two artifacts that happen to share a heading. One is a field table. The other is a behavior contract.

The field table answers parser questions: name, location, required flag, pattern, example. The behavior contract answers client questions: what the key is scoped to, how long a replay is honored, and what a mismatched body returns.

Collapse those layers into one fluent paragraph and an optional string starts to read like an SLA. Keep them in separate files until a human attests the second file.

What generation may fill

A generator, including a model, may fill a row only when the repo already contains the fact. The test is mechanical: can you point at a file that already says it?

Allowed inputs:

The OpenAPI parameter object: name, in, required, schema, and example if present.
A passing contract test whose name is cited beside the status it asserts.
A fixture path recorded next to the example value.
A glossary sentence copied as a quote, not extended with new scope language.

Every drafted sentence should carry a source marker, for example . If the marker cannot be produced from the snapshot, the sentence stays out of the draft file. Silence is a valid output.

What a human must own

An API owner, with support policy in the room, must write and attest the following before the page is public. The list is short on purpose. Each item is a commitment, not a description.

Scope: account, route, or a stated composite such as account plus route.
Replay window: a duration with a unit, or the explicit phrase not published.
Mismatch rule: same key, different body, which status, and whether the first response is replayed.
Stored material: raw key, hash only, response snapshot, or request body.
Failover claim, if any: what happens when the store that remembers the key is unreachable.

These lines are promises. A blank that says not published is safer than a smooth sentence nobody signed.

Decision table

Use the table as a publish gate, not as inspiration for extra columns. Fill means a model may propose wording from a cited source. Attest means a named human accepts the line.

Element	Typical source	Mode	Publish rule
Header name and location	OpenAPI	Fill	Block if the parameter is absent
Required flag	OpenAPI `required`	Fill	Must match the gateway config you extracted
Pattern and max length	OpenAPI schema	Fill	Quote the schema; do not tighten it in prose
Example value	Schema example or fixture path	Fill	No invented story around the example
Scope	Product note	Attest	Empty or not published until signed
Replay window	Product note	Attest	No duration without an attestation id
Mismatch status	Contract test	Fill only with test name	Otherwise Attest
Stored fields	Security review	Attest	Never inferred from sample logs
Client retry list	Error catalog version	Quote	Human confirms the quoted list

Adding a column because a model usually includes it is a spec change. Spec changes do not ride along in a docs pull request.

Six-step workflow

Run the split on one operation before you scale it. The point is a blocked publish, not a longer page.

Snapshot. Write the parameter block and matching test names to build/idempotency/sources.json. Do not hand-edit that file.
Separate. Create fields.md for sourced rows and promises.md for blanks. Start promises.md with not published, not with a guess.
Fill. Point a model at sources.json and fields.md only. Do not put promises.md in the same context.
Scan. Run the checker below on the combined source before render.
Attest. The API owner replaces each not published they are willing to stand behind, and adds an attestation marker.
Release. Publish only when the checker exits 0 and each remaining not published is an intentional public statement, not an oversight.

Steps 1, 4, and 6 belong in CI. Step 5 does not. If the same account can both fill and waive attestation, the gate is decorative.

Commands for the snapshot

The commands below are illustrative. They are not a log from a live repository, and the paths assume a conventional layout.

mkdir -p build/idempotency
jq '.paths["/charges"].post.parameters[]
    | select(.name=="Idempotency-Key")' openapi.json \
  > build/idempotency/parameter.json
rg -n "Idempotency-Key" tests/contract -g '*.json' \
  > build/idempotency/test-hits.txt

A minimal sources.json shape, also illustrative:

{
  "operation": "POST /charges",
  "parameter_file": "build/idempotency/parameter.json",
  "tests": ["tests/contract/charges_retry.json"],
  "extracted_at": "set-by-ci"
}

The timestamp is a CI value. Do not ask a model to invent it.

Checker script

The script is a proposal. It has not been run on a production tree for this article. It flags duration and scope phrasing that lacks an attestation marker.

#!/usr/bin/env python3
"""Flag idempotency promise lines that lack an attestation marker.

Proposal only. Not a measured scan.
"""
import re
import sys
from pathlib import Path

PROMISE = re.compile(
    r"\b(replay window|retained for|stored for|scoped to|per account|"
    r"per merchant|request body is kept|\d+\s*"
    r"(seconds?|minutes?|hours?|days?))\b",
    re.I,
)
MARKER = "promise-attested:"

def main() -> int:
    path = Path(sys.argv[1])
    hits = []
    for i, line in enumerate(path.read_text(encoding="utf-8").splitlines(), 1):
        if PROMISE.search(line) and MARKER not in line:
            hits.append(f"{path}:{i}: unsigned promise language")
    if hits:
        print("\n".join(hits))
        return 1
    print("no unsigned promise lines")
    return 0

if __name__ == "__main__":
    raise SystemExit(main())

Run it on the source file CI will publish from, not on the rendered HTML. A green process exit is an input to review, not a substitute for it.

python3 check_idempotency_promises.py docs/charges.md

An attested line in source:

Replay window: 24 hours. <!-- promise-attested: api-owner, note idem-04 -->

The marker is intentionally plain. Renderers may strip HTML comments. The file CI reads must still contain the marker, or the gate checks the wrong artifact.

What a green run does not prove

A pass means the listed phrases were either absent or marked. It does not mean the duration is true. It does not catch a paraphrase such as duplicates collapse through the next settlement cutover. Pair the script with the table, and still read every page that mentions replay or retention.

False confidence is the main risk. A short regex feels like policy. It is only a tripwire.

A free pass for the field rows

The expensive part of this split is not the attestation meeting. It is producing field rows for many operations without letting blanks get completed in the same breath.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Operator-supplied availability, not a benchmark: MonkeyCode's free model access can run that field-row pass on schema excerpts, and a free server option can run the checker as a small job next to CI. This article states no token quota, model name, hardware size, time limit, or permanence. Confirm current terms before you depend on either option.

If those options are unavailable, the same files work with any model you can constrain to sources.json. The useful constraint is context, not brand: keep promises.md out of the prompt. Send the parameter JSON, the test-hit list, and an instruction to emit not published for scope and window.

Reject any completion that adds minutes, accounts, or storage claims. That rejection is the workflow. The model is not the author of the promise.

If you try the split, start with one operation and the checker in CI. Confirm the free model access and free server option still match your constraints before you make either a required check.

Limitations

The approach has a narrow job. It stops fluent invention of replay policy. It does not certify the API.

The regex misses paraphrase. Plan a human read for any page that discusses retries.
OpenAPI can disagree with the gateway. A filled required cell is only as current as the snapshot.
A test that expects HTTP 200 on retry does not define a window. Do not let the test name smuggle a duration into prose.
Free model access and a free server option can be withdrawn or limited. They are not an uptime commitment for your docs pipeline.
Storage lines can be a privacy issue. Security review owns them even when the table says Attest.
Client samples do not belong in the promise file. A syntax edit must not be able to rewrite a retention sentence.

Who should skip this

Skip it when you have neither an OpenAPI document nor contract tests. There is no honest source to fill from, and a model will treat the gap as a writing prompt.

Skip it when the team wants a suggested window because nobody has decided. Decide in the product note. Then let docs quote the note.

Skip it on public payment or booking APIs where the replay sentence is a customer commitment, unless counsel already owns that sentence. A docs job should not become the author of record for a charge guarantee.

Skip it when one person can generate the page and override the checker to meet a release time. Unsigned rows have to fail the build, or the split is a style guide nobody runs.

Keep the promise unsigned until a person signs

Field rows are a rendering task. Replay windows are a product promise. Let generation handle the first. Put a name, a note id, and a date on the second.

Publish not published when the decision is not ready. That sentence is accurate. A guessed retention period is not.

Generated Pages Need Owners: A Lane Manifest for API Docs

Morgan Sun — Thu, 24 Sep 2026 19:52:59 +0000

A release candidate goes out with regenerated endpoint pages. The new prose is easier to scan than the old notes. One added sentence tells clients they can cache a successful response for a day. Another says a 404 means the order was deleted, not that it was missing.

Treat that scene as a composite. It is not a measured incident from this account. The OpenAPI document in the scene says neither sentence. The page is clear anyway. The owner of the two promises is not.

A recent public thread on DEV separated clean code from clear code, and treated comments as a clarity tool rather than a violation (topic signal). The documentation version of that split is stricter. A generated walkthrough can be clear and still unauthorized. This piece is a workflow for that boundary, with a proposed checker. It is not a report of production rates.

The job split

This workflow asks one question before anyone prompts a model. Which sentences may be drafted, and which sentences must be owned by a named person?

Three lanes are enough for an endpoint page. More lanes tend to become a second specification. Keep the split mechanical so a script can enforce it.

Narrative

The narrative lane explains fields and call order using only what the spec already states. A model may draft it. A human reviews it for invented steps. Regeneration is allowed when the spec changes.

Specimen

The specimen lane holds request and response examples. This article does not reopen how fixtures relate to SDK snippets. The boundary is enough. Examples do not live in the narrative file, and a prose pass does not invent them.

Promise

The promise lane is compatibility language. Caching, retry rules, deletion meaning, deprecation windows, and any sentence with "always", "never", "guaranteed", or a time bound belong here. A model must not draft this lane. A named human writes it, dates it, and locks it against regeneration.

A decision matrix

Use the matrix in review. Do not paste it into a prompt and call that a control.

Field restatement, such as id is a string. Lane: narrative. Keep it only when the spec agrees.
Sequence help, such as "create the order, then fetch it." Lane: narrative. Reject it when the second call is not in the source.
Status gloss absent from the spec, such as "404 means deleted." Lane: promise. Drop it, or have a human write a sourced sentence.
Time bound, such as "cache for 24 hours." Lane: promise. Drop it unless a human owner accepts the number and names the source.
Sample JSON. Lane: specimen. Do not let a narrative pass invent bodies.

The matrix is a review aid. It is not a measured failure rate. No frequency was collected for this draft.

The ownership manifest

The paths below are a proposed layout. They are not files from a published repository. Dates in the sample are review markers, not evidence that a team signed them.

version: 1
page: endpoints/orders.get
lanes:
  narrative:
    path: docs/orders.get.narrative.md
    regenerate: true
    inputs:
      - openapi/orders.yaml
  specimen:
    path: docs/orders.get.examples.md
    fixture: fixtures/orders/get-200.json
  promise:
    path: docs/orders.get.promise.md
    owner: docs-review
    signed_on: 2026-09-25
    forbid_regen: true
    source: openapi/orders.yaml#/paths/~1orders~1{id}/get

The promise file stays short. Length is not ownership. Policy changes happen by editing this file and moving the date, not by regenerating the page.

<!-- regen:forbidden -->
Owner: docs-review
Signed-on: 2026-09-25
Source: openapi/orders.yaml#/paths/~1orders~1{id}/get

This operation publishes no cache lifetime.
A 404 means no matching order was found. It does not mean the order was deleted.

If product later decides that a 404 does mean deletion, a human edits those two sentences and changes Signed-on. A regeneration job does not get a vote.

The checker

The script is a proposed example. It was not executed for this draft. It needs PyYAML. It does not call a model. It fails when the promise lane is unlocked, or when narrative prose contains a small set of promise-like phrases.

#!/usr/bin/env python3
"""Lane checker for generated API pages. Proposed example; not run for this draft."""

import argparse
import pathlib
import re
import sys

try:
    import yaml
except ImportError:
    sys.exit("PyYAML is required: pip install pyyaml")

PROMISE_HINTS = (
    r"\bsafe to cache\b",
    r"\bguaranteed\b",
    r"\balways returns\b",
    r"\bwill not change\b",
    r"\bbackward[- ]compatible\b",
    r"\bretry until\b",
    r"\bmeans the resource was deleted\b",
    r"\bfor \d+ (?:hours|minutes|days)\b",
)
PROMISE_HEADERS = ("owner:", "signed-on:", "source:")


def main() -> int:
    parser = argparse.ArgumentParser()
    parser.add_argument("--manifest", required=True)
    args = parser.parse_args()
    manifest_path = pathlib.Path(args.manifest)
    data = yaml.safe_load(manifest_path.read_text(encoding="utf-8"))
    lanes = data.get("lanes") if isinstance(data, dict) else None
    if not isinstance(lanes, dict):
        print(f"{manifest_path}: missing lanes")
        return 1

    errors: list[str] = []
    narrative = lanes.get("narrative") or {}
    promise = lanes.get("promise") or {}

    if not promise.get("forbid_regen"):
        errors.append("promise.forbid_regen must be true")
    if not promise.get("owner") or not promise.get("signed_on") or not promise.get("source"):
        errors.append("promise owner, signed_on, and source are required")

    narrative_path = pathlib.Path(narrative.get("path", ""))
    promise_path = pathlib.Path(promise.get("path", ""))
    if not narrative_path.is_file():
        errors.append(f"missing narrative file: {narrative_path}")
    else:
        text = narrative_path.read_text(encoding="utf-8")
        for pattern in PROMISE_HINTS:
            for match in re.finditer(pattern, text, flags=re.IGNORECASE):
                errors.append(
                    f"{narrative_path}: promise-like phrase in narrative: {match.group(0)!r}"
                )

    if not promise_path.is_file():
        errors.append(f"missing promise file: {promise_path}")
    else:
        body = promise_path.read_text(encoding="utf-8")
        lowered = body.lower()
        if "<!-- regen:forbidden -->" not in lowered:
            errors.append(f"{promise_path}: missing regen lock comment")
        for header in PROMISE_HEADERS:
            if header not in lowered:
                errors.append(f"{promise_path}: missing header {header}")

    if errors:
        print("\n".join(errors))
        return 1
    print("lane check passed")
    return 0


if __name__ == "__main__":
    raise SystemExit(main())

Run it from the directory that contains docs/. These commands are part of the proposal. They are not a transcript.

python3 -m pip install pyyaml
python3 check_doc_lanes.py --manifest docs/orders.get.lanes.yaml

A useful negative case is a narrative line that says clients may cache the response for 24 hours. The script should exit non-zero and name that phrase. That is the intended behavior of the example, not a captured log. Run it locally before you wire it to CI.

What the model may see

Give the draft pass only the narrative inputs listed in the manifest. Do not attach the promise file as a style sample. Nearby policy sentences get copied.

Keep the prompt boundary dull and specific.

Paste the operation from the spec, not the whole repository.
Name the output as the narrative path only.
Forbid cache lifetimes, retry policy, deletion meaning, and compatibility claims.
Ask for no sample JSON in that file.

Then run the checker. A green run means the hint list did not match. It does not mean the prose is free of policy. Read the diff anyway. The read is the control the script cannot be.

Where a free draft host fits

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The operator states that MonkeyCode offers free model access and a free server option. This draft does not name models, token quotas, hardware, or a time window. Those details were not verified here. Read them from current project documentation before depending on them.

Use that access after the manifest exists, not before. A free model pass may draft the narrative file from the spec excerpt. A free server, if you use the hosted option, is only a place to run the checker so review is not tied to one laptop. Your repository CI remains the merge gate. The host does not own the promise file.

If a hosted draft lane is useful once that gate is in the repo, confirm the free option in the project docs and then try it. Do not add a host in order to invent cache rules or status meanings.

Limitations

Phrase misses

The hint list is short. "Clients can keep the body until the next deploy" will not match safe to cache. Paraphrase is the usual miss. Extend the list from your own incident notes, and still read the narrative file.

Sign-off is not behavior

A signed promise can be false. The date records who accepted the sentence. It does not prove the server behaves that way. Contract tests remain the behavioral check. This script only guards against policy smuggled into a regenerated walkthrough.

Who should skip it

Skip the approach in these cases.

No named person will accept the promise file. A shared label with no rotation is theater.
The page is a security, privacy, or legal commitment. That text needs its own review path.
You want the model to choose product policy. That repeats the opening failure.
The pipeline assumes a fixed free quota or a permanent runner. Availability can change. Keep the checker in the repo so local CI still runs.

Review order

Follow the same order on each change. Skipping to generation is how promise sentences reappear.

Update the spec, or record the human decision the spec does not yet encode.
Edit the promise file only if policy changed, and change Signed-on.
Regenerate narrative from spec inputs, not from the previous narrative.
Run the lane checker.
Read every sentence the hints did not catch. Budget that read. Do not replace it with a green script.

Clear generated prose is a drafting result. A compatibility sentence is a decision. The split holds only when those live in different files, with different rights to regenerate.

Null, Zero, Absent: A Three-State Gate for AI-Drafted API Docs

Morgan Sun — Wed, 23 Sep 2026 16:29:53 +0000

The typical regen failure is quiet. A /list-invoices page comes back from the model with a parameter table. The table looks complete. Then a client sends limit=0 because the table said 0 means “no cap.”

The OpenAPI file never said that. limit was optional. The server default was 20. Sending 0 returned an empty page. Sending a JSON null in a query string was a 400. Three states, one collapsed sentence.

This is not a tone problem. It is a contract problem. Models are good at filling tables. They are bad at preserving the difference between a missing field, an explicit null, and a numeric zero.

The collapse, in one response

Consider a list endpoint with this query shape:

parameters:
  - name: limit
    in: query
    required: false
    schema:
      type: integer
      minimum: 0
      maximum: 100
      default: 20
  - name: cursor
    in: query
    required: false
    schema:
      type: string
      nullable: true
  - name: include_deleted
    in: query
    required: false
    schema:
      type: boolean

A docs model often emits something like this:

limit (integer, default 0) — Number of invoices to return. Use 0 for all.
cursor — Opaque pagination token. Pass null to start from the beginning.
include_deleted — Optional. Defaults to false if omitted or null.

Every line is fluent. Every line is wrong in a different way.

limit defaulted to 20, not 0. Zero is a legal value with a different meaning.
cursor omitted means “first page.” cursor=null is not a documented start token.
include_deleted has no default in the schema. Treating omitted and null as false is an invented policy.

If you only grep for hallucinated URLs, this page ships. The damage shows up in client SDKs and in support tickets, not in the Markdown linter.

What the model may draft

Keep the model inside the schema. If a fact is in OpenAPI, the model may restate it. If a fact is not in OpenAPI, the model may not invent a substitute.

Draftable (schema-backed):

Parameter name, location (query / path / header / body), and JSON type
required vs optional, when the flag is explicit
minimum, maximum, enum, pattern, maxLength
Example payloads copied from a checked-in fixture, not from the prompt
Cross-links to other operations that already exist in the same spec

Not draftable (human-owned):

Default values that are missing from the schema
Equivalence claims: “omitted means X”, “null means Y”, “zero means Z”
Server-side fallback timing (client default vs server default vs proxy default)
Pagination opacity, cursor expiry, and what happens after a filter change
Idempotency, retry, and partial-success behavior
Units, timezones, rounding, and currency minor units

The split is mechanical. Name and type are structural. Meaning of absence is behavioral. Regenerating the first should never rewrite the second.

A decision table you can check in CI

Schema facts	Draft lane	Human lane	Fail the regen if
`type` + `required`	Parameter table row	—	Model adds a default not in schema
`default` present	Restate the default, quoted from schema	When the default is applied (client vs server)	Model changes the number or says “ignore if zero”
`nullable: true`	Mark the field nullable	What null does vs omit	Model equates null with omit or with zero
no `default`, not required	“Omitted is allowed”	What the server does on omit	Model invents “defaults to …”
`minimum: 0` on integer	Show the bound	Whether `0` is empty, unlimited, or invalid	Model says `0` means unlimited
`enum`	List members	Why a member exists	Model adds a member

The table is the artifact, not the prose around it. If a row cannot be filled from the spec, the cell stays empty until a human writes it in a signed file.

Fixture: three states, one field family

Save this as fixtures/list-invoices.openapi.yaml. It is small on purpose. The gate should fail on meaning, not on file size.

openapi: 3.0.3
info:
  title: Invoices
  version: 0.0.0
paths:
  /v1/invoices:
    get:
      operationId: listInvoices
      parameters:
        - name: limit
          in: query
          required: false
          schema:
            type: integer
            minimum: 0
            maximum: 100
            default: 20
        - name: cursor
          in: query
          required: false
          schema:
            type: string
            nullable: false
        - name: include_deleted
          in: query
          required: false
          schema:
            type: boolean
      responses:
        "200":
          description: A page of invoices

Human-owned notes live beside the spec, not inside the generated page. Example: docs/owned/listInvoices.states.md.

# listInvoices — three-state notes
# owner: api-docs
# signed: true

## limit
- omit: server applies 20
- 0: empty page, not “unlimited”
- null: not a legal query value (400)

## cursor
- omit: first page
- empty string: 400
- null: 400
- opaque token: server-defined; do not document internals

## include_deleted
- omit: unspecified; do not claim a default
- true / false: filter as named
- null: 400

The generated Markdown may cite these notes. It may not paraphrase them into a friendlier default.

The gate: classify, then scan

The following checker is a self-contained example. Run it against the fixture. It is a contract linter, not a production docs platform.

#!/usr/bin/env python3
"""Fail docs regen when a model collapses omit / null / zero."""
from __future__ import annotations

import json
import re
import sys
from pathlib import Path

try:
    import yaml
except ImportError:
    yaml = None

INVENTED_DEFAULT = re.compile(
    r"defaults?\s+to\s+[`'"]?(all|unlimited|none|null|0|false|true)",
    re.I,
)
EQUATES_NULL_OMIT = re.compile(r"null\s+(means|is)\s+(omitted|absent|missing)", re.I)
ZERO_MEANS_ALL = re.compile(r"\b0\b.{0,40}(unlimited|all records|no cap)", re.I)


def load_spec(path: Path) -> dict:
    text = path.read_text(encoding="utf-8")
    if path.suffix in {".yaml", ".yml"}:
        if yaml is None:
            raise SystemExit("pip install pyyaml")
        return yaml.safe_load(text)
    return json.loads(text)


def iter_params(spec: dict):
    for path, item in (spec.get("paths") or {}).items():
        for method, op in item.items():
            if not isinstance(op, dict):
                continue
            op_id = op.get("operationId") or f"{method.upper()} {path}"
            for param in op.get("parameters") or []:
                schema = param.get("schema") or {}
                yield {
                    "op": op_id,
                    "name": param.get("name"),
                    "required": bool(param.get("required")),
                    "type": schema.get("type"),
                    "default": schema.get("default", _Missing),
                    "nullable": schema.get("nullable", False),
                    "minimum": schema.get("minimum", _Missing),
                }


class _Missing:
    pass


def classify(param: dict) -> dict:
    flags = []
    if param["default"] is _Missing and not param["required"]:
        flags.append("no_default_do_not_invent")
    if param["default"] is not _Missing:
        flags.append("default_is_schema_owned")
    if param["nullable"]:
        flags.append("null_is_not_omit")
    else:
        flags.append("null_is_invalid_unless_spec_says")
    if param["type"] == "integer" and param["minimum"] == 0:
        flags.append("zero_is_a_value")
    return {**param, "flags": flags}


def scan_markdown(md: str, classified: list[dict]) -> list[str]:
    errors = []
    if INVENTED_DEFAULT.search(md):
        errors.append("invented_default_phrase")
    if EQUATES_NULL_OMIT.search(md):
        errors.append("null_equated_to_omit")
    if ZERO_MEANS_ALL.search(md):
        errors.append("zero_collapsed_to_unlimited")
    for row in classified:
        if "no_default_do_not_invent" in row["flags"]:
            pat = re.compile(
                rf"{re.escape(row['name'])}.{{0,80}}defaults?\s+to",
                re.I | re.S,
            )
            if pat.search(md):
                errors.append(f"invented_default:{row['op']}:{row['name']}")
        if "zero_is_a_value" in row["flags"] and row["default"] != 0:
            pat = re.compile(
                rf"{re.escape(row['name'])}.{{0,80}}default\s+[`']?0",
                re.I | re.S,
            )
            if pat.search(md):
                errors.append(f"wrong_default_zero:{row['op']}:{row['name']}")
    return errors


def main(argv: list[str]) -> int:
    if len(argv) != 3:
        print("usage: three_state_gate.py <openapi> <generated.md>", file=sys.stderr)
        return 2
    spec = load_spec(Path(argv[1]))
    md = Path(argv[2]).read_text(encoding="utf-8")
    classified = [classify(p) for p in iter_params(spec)]
    errors = scan_markdown(md, classified)
    print(json.dumps({"params": classified, "errors": errors}, indent=2, default=str))
    return 1 if errors else 0


if __name__ == "__main__":
    raise SystemExit(main(sys.argv))

A failing page is useful. Save this as fixtures/list-invoices.bad.md:

## Query parameters

- `limit` (integer, default `0`) — use `0` for unlimited.
- `cursor` — pass `null` to start. Null means omitted.
- `include_deleted` defaults to false.

Run:

python3 three_state_gate.py \
  fixtures/list-invoices.openapi.yaml \
  fixtures/list-invoices.bad.md

Expected: non-zero exit. The JSON report should include wrong_default_zero, zero_collapsed_to_unlimited, null_equated_to_omit, and invented_default:listInvoices:include_deleted.

A clean page restates schema facts and points at the signed notes:

## Query parameters

- `limit` (integer, optional, schema default `20`, minimum `0`). Zero is a legal value; it is not unlimited. See owned notes.
- `cursor` (string, optional, not nullable). Omit for the first page. Do not send null.
- `include_deleted` (boolean, optional). No schema default. Do not equate omit with false.

The gate does not try to understand invoices. It only refuses collapsed states.

Regen workflow

Keep three inputs, one output, one reject path.

Extract. Read OpenAPI. Classify each parameter with the table above. Write classified.json.
Draft. Prompt the model with the classified rows and a ban list: no default unless default is present; no “null means omitted”; no “0 means all.”
Scan. Run the gate on the draft. Fail the job on any error code from the scanner.
Join. If the scan passes, concatenate the draft with the signed three-state notes. Do not let the model rewrite the notes file.
Publish. Only the joined document is canonical. The draft file is disposable.

Prompt text should be boring. A usable system prompt is a list of refusals, not a style guide.

You draft parameter tables from classified.json only.
If default is missing, write “no schema default.”
Never equate null with omit.
Never equate 0 with unlimited.
Never invent units, timezones, or retry rules.
If a human note file is provided, quote it; do not paraphrase defaults.

The model is a formatter sitting on a classifier. It is not the source of behavior.

Where a free model is enough

This job is classification plus constrained drafting. It does not need a long-running agent and it does not need the model to invent retry policy.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode’s free model access and free server option are enough to run the extract-draft-scan loop on a spec like the fixture: the model fills tables, the gate rejects collapsed states, and the signed notes stay off the regen path. That is the only product role here. The ownership split still works if you swap the model.

If you already treat OpenAPI as canonical, the useful experiment is the gate, not a new prose style.

Limitations

The scanner is phrase-based. A model can still smuggle a wrong default in a sentence the regex does not catch. Tighten it with AST checks on generated MDX, or require every default to appear as a fenced schema.default value.

OpenAPI 3.0 nullable is not the same as a 3.1 union type. The classifier above does not expand oneOf / anyOf. If your spec uses unions for “string or null,” extend classify() before you trust the gate.

Query parameters are the easy case. Body fields with additionalProperties, vendor extensions, and polymorphic discriminator objects need a different owned-notes layout. Do not reuse this script as a general API linter.

The gate does not prove the server matches the spec. If production applies a default of 50 while OpenAPI says 20, both the model and the human notes will be consistently wrong. Spec drift is a different pipeline.

Who should not use this

Skip the split if your docs are hand-written and rarely regenerated. A gate that assumes disposable drafts will only add ceremony.

Skip it if legal or safety text lives in the same Markdown file as parameter tables. Those files should not be model-writable at all.

Skip it if you cannot name an owner for the three-state notes. Unsigned “behavior” comments next to the spec will be overwritten, and the gate cannot tell a reviewed sentence from a leftover prompt.

The practical rule is narrow. Let the model draft the table. Keep omit, null, and zero in a signed file. Fail the regen when those three states collapse into one adjective.

DEV Community: Morgan Sun

The Model May Draft the Prose. The Repo Must Confirm the Facts.

Split the doc before you generate it

Artifact: a claim checker that fails the build

Why a grep and not a model

Wiring it into a real loop

Limitations, stated plainly

Who should not use this

Docs Rot Unevenly: Rank Sections by Drift Before You Regenerate Them

The unit is the section, not the file

Score drift with git, not with vibes

Three lanes, three levels of trust

The prompt contract that keeps the lane honest

Where the free tier actually fits

A test plan you can run this week

Where this breaks

Who should skip this

Anchor Coverage: A CI Gate for Docs a Model May Draft but Never Own

Reviewing prose does not scale, checking anchors does

Three lanes: extract, draft, own

The artifact: anchor_gate.py

Generate claims.json, do not type it

The list a model must never write

Running the draft pass where it costs nothing to re-run

A four-case test plan for the gate itself

Limitations, and who should skip this

Start with one directory

Who Owns This Paragraph? An Ownership Manifest for AI-Drafted Docs

Generated docs rarely fail on prose

Classify blocks, not whole files

Mark blocks in the source

The gate

The decision table

Where a free tier actually helps

Limitations and who should not use this

Start with three blocks

Allowlist the Command Before the Generated Runbook Ships

What breaks when draft and binding share one file

What the model may draft

What a human must own

A checker you can run before publish

How to wire the split

Where a free draft lane fits

A short test plan

Limitations

Who should skip this

Close

Classify the Environment Before the Walkthrough Is Drafted

Draft the Migration Note. Refuse the Unsigned Number.

The Model May Draft the Example. It May Not Bind the Client.

Two sentence classes, one page

What the model may draft

What a human must own

Markers the checker can see

A claim-class checker

Exit codes

Decision table

Where the draft pass runs

Test plan

A failure, then the fix

Limitations

Who should not use this

Keep Replay Outcomes Out of the Generated Page

Two jobs, one URL

What the model may draft

What a human must own

Decision table

Worked files

Boundary checker

What the checker does not prove

Review order

A free drafter, used narrowly

Prompt boundary

Server boundary

Limitations

Who should skip it

The next docs PR

Draft the Header Row, Sign the Replay Window

Two layers in one header

What generation may fill

The artifact: `anchor_gate.py`

Generate `claims.json`, do not type it