Source-Bound Docs: Draft From Tests, Own the Promises

Model-drafted documentation stays safe only when every paragraph cites a file the repository already contains. A prompt is not a source of truth, and a fluent page is not a contract. The workflow below binds draftable sections to tests, OpenAPI files, or command help, then fails the build on unbound claims. Humans still own promises, compatibility statements, and security posture, because those sentences cannot be checked against a path.

Why prompt-only documentation drafts fail review

Generated pages usually stall in review because the prose cannot be traced to a repository artifact. The model invents flags, default values, error strings, and support promises that never appear in code. Reviewers then hunt for missing citations inside a fluent diff, which is slower than writing the page. A source map reverses that sequence by declaring, before generation, which files may feed which headings.

This is not a style guide and it is not a semantic proof that the prose matches the product. It is a gate that answers a narrower question: did this heading name a file the tree can still open? If the answer is no, the heading is not draftable, even when the surrounding page looks complete. That single question removes the most expensive class of documentation review: claims that cannot be falsified.

The source-binding record

Treat each documentation heading as a record with a path, a binding kind, and an owner role. Bound headings may be drafted by a model because a checker can load the cited file and confirm it exists. Unbound headings stay human-owned, even when neighboring sections are generated from tests or schemas. The contract is intentionally small: path existence and role, not semantic equivalence between prose and source.

Binding kinds a checker can enforce

Use a short vocabulary so CI stays deterministic and authors do not invent one-off labels. The four kinds below cover most product repositories and avoid pretending the checker understands English claims.

1. from_test — procedure steps that rest on an automated test file the suite actually runs.
2. from_schema — request and response fields that rest on OpenAPI or checked-in JSON Schema.
3. from_help — flags and subcommands that rest on a frozen --help fixture, not live memory.
4. human_promise — compatibility, support, security, pricing, and “we will never” sentences.

Decision table

Heading pattern	Binding kind	Model may draft?	Human must own
How-to procedure with a named test	from_test	Yes, after the test path exists	Any sentence that turns a test into a customer guarantee
API field table	from_schema	Yes, limited to names in the schema	Semantics that the schema does not state
CLI flag list	from_help	Yes, limited to flags in the fixture	When a flag will be removed, and for whom
Breaking-change note	human_promise	No	The entire heading
Security or threat-model page	human_promise	No	The entire heading
Support hours, SLA, or severity matrix	human_promise	No	The entire heading
Conceptual “why we built this”	human_promise	No	The entire heading

The table is the policy. The YAML map below is the machine-readable form of the same policy, checked on every pull request that touches docs/.

A reproducible source map

Store the map beside the docs tree so pull requests change map and prose together. The YAML below is a proposal for a small checker, not an executed production dataset.

# docs/source-map.yaml — proposal
version: 1
docs_root: docs
generated_marker: "generated_from: source-map"
entries:
  - heading: "docs/howto/retry-failed-jobs.md#retry-a-failed-job"
    kind: from_test
    source: tests/jobs/test_retry_failed_job.py
    owner: model-draft
  - heading: "docs/api/jobs.md#job-status-fields"
    kind: from_schema
    source: openapi/jobs.yaml
    owner: model-draft
  - heading: "docs/cli/jobs.md#job-flags"
    kind: from_help
    source: fixtures/help/jobs.txt
    owner: model-draft
  - heading: "docs/api/jobs.md#compatibility"
    kind: human_promise
    source: null
    owner: human
  - heading: "docs/security/overview.md#trust-boundaries"
    kind: human_promise
    source: null
    owner: human

Each heading value is a file path plus an anchor derived from the Markdown H2 text. Bound entries must point at a file that exists in the same commit. human_promise entries must set source to null and owner to human, which the checker treats as a write ban for draft jobs.

Numbered workflow

Step 1 — Inventory headings and refuse empty bindings

Walk the documentation tree and list every H2 path before a model is invoked. A heading without a map entry is treated as an error, not as a candidate for generation. That rule prevents a model from filling leftover sections that nobody has classified yet. Run the inventory in CI on documentation pull requests so an unmapped heading cannot merge silently.

# Proposal: list H2 anchors under docs/
python3 scripts/check_doc_bindings.py inventory --docs-root docs

Step 2 — Bind procedures to tests, not to ticket text

Ticket titles drift, so test files remain the only procedure source the checker can open. If a how-to section cannot name a test, leave it unbound and write it by hand. Do not lower the bar by pointing the map at a design document that CI does not run. A procedure that cannot be exercised is not a procedure the model should narrate.

Step 3 — Bind interface docs to schema files

API field tables should cite an OpenAPI path or a checked-in JSON Schema document. When the schema changes, the map still points at the same file, and the draft job reruns. Examples that assert product behavior beyond the schema belong under human_promise headings instead. That split keeps generated tables short and keeps product promises out of the draftable set.

Step 4 — Snapshot command help before drafting CLI pages

CLI documentation is a frequent source of invented flags because help text is cheap to ignore. Check a fixture of --help output into the repository and bind the CLI page to that fixture. The model may rephrase the fixture; it may not introduce a flag the fixture does not list. Refresh the fixture in the same pull request that changes the parser, not in a later documentation cleanup.

# Proposal: freeze help text next to the map
python3 -m yourcli jobs --help > fixtures/help/jobs.txt
git add fixtures/help/jobs.txt docs/source-map.yaml

Step 5 — Run the existence and token checkers in CI

Path existence is the cheap gate and should fail fast. A second, still mechanical, gate compares --flag tokens in from_help sections against the fixture. Neither gate proves the English is correct; both gates prove the draft stayed inside a cited artifact. Put both gates on pull requests that touch docs/, openapi/, fixtures/help/, or tests/.

Step 6 — Draft only bound sections on a disposable worker

Draft jobs need isolation more than they need a large model, because bound sections are short and sourced. Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode's free model access and free server option can host that isolated draft step without becoming the source of truth. The checker still runs in the repository's own CI, where path existence does not depend on a vendor worker.

Feed the worker the heading, the cited file, and a ban list of promise verbs. Do not feed it the rest of the product narrative, and do not ask it to “make the page complete.” Completeness is how unbound claims enter a how-to. The labeled prompt below is a proposal, not a recorded production session.

# Proposal prompt — unexecuted example
You may rewrite only the heading named in the map entry.
You may use only the attached source file as factual input.
Do not add flags, fields, errors, or defaults that the source file does not contain.
Do not write compatibility, support, security, pricing, or SLA sentences.
If the source file is silent, write: "NOT IN SOURCE" and stop.

Step 7 — Human-sign the promise headings

After the draft job returns, the remaining work is not copyediting of generated how-tos. It is authorship of every human_promise heading in the same pull request, with a named reviewer. If the how-to changed because a test changed, check whether a promise now overclaims the new behavior. Merge only when the map, the cited files, the generated sections, and the human-owned sections share one commit graph.

Checker (proposal)

The script below is a concrete starting point. It is labeled a proposal because it has not been executed against a private product corpus in this article. It fails the build when a heading is missing from the map, when a bound source path is absent, when a human_promise heading still carries a generated marker, or when a from_help section mentions a flag the fixture does not contain.

#!/usr/bin/env python3
"""docs/source-map.yaml checker — proposal, unexecuted in this article."""
from __future__ import annotations

import argparse
import re
import sys
from pathlib import Path

try:
    import yaml
except ImportError as exc:
    raise SystemExit("PyYAML is required: pip install pyyaml") from exc

HEADING_RE = re.compile(r"^(#{2})\s+(.+?)\s*$", re.M)
FLAG_RE = re.compile(r"--[a-z0-9][a-z0-9-]*")
ANCHOR_RE = re.compile(r"[^a-z0-9]+")
GENERATED_RE = re.compile(r"generated_from:\s*source-map")


def slug(text: str) -> str:
    value = text.strip().lower()
    value = ANCHOR_RE.sub("-", value).strip("-")
    return value


def heading_id(path: Path, title: str) -> str:
    return f"{path.as_posix()}#{slug(title)}"


def load_map(map_path: Path) -> dict:
    data = yaml.safe_load(map_path.read_text(encoding="utf-8"))
    if not isinstance(data, dict) or "entries" not in data:
        raise SystemExit(f"invalid map: {map_path}")
    return data


def inventory(docs_root: Path) -> list[str]:
    found: list[str] = []
    for path in sorted(docs_root.rglob("*.md")):
        text = path.read_text(encoding="utf-8")
        for match in HEADING_RE.finditer(text):
            found.append(heading_id(path, match.group(2)))
    return found


def flags_in(text: str) -> set[str]:
    return set(FLAG_RE.findall(text))


def check(map_path: Path) -> list[str]:
    data = load_map(map_path)
    docs_root = Path(data.get("docs_root", "docs"))
    errors: list[str] = []
    mapped = {entry["heading"]: entry for entry in data["entries"]}
    found = inventory(docs_root)

    for heading in found:
        if heading not in mapped:
            errors.append(f"unmapped heading: {heading}")

    for heading, entry in mapped.items():
        kind = entry.get("kind")
        source = entry.get("source")
        owner = entry.get("owner")
        md_path = Path(heading.split("#", 1)[0])
        if not md_path.exists():
            errors.append(f"missing doc file for {heading}")
            continue
        body = md_path.read_text(encoding="utf-8")
        generated = bool(GENERATED_RE.search(body))

        if kind in {"from_test", "from_schema", "from_help"}:
            if not source:
                errors.append(f"{heading}: bound kind requires source")
            elif not Path(source).exists():
                errors.append(f"{heading}: missing source {source}")
            if owner != "model-draft":
                errors.append(f"{heading}: bound kind expects owner=model-draft")
            if kind == "from_help" and source and Path(source).exists():
                extra = flags_in(body) - flags_in(Path(source).read_text(encoding="utf-8"))
                for flag in sorted(extra):
                    errors.append(f"{heading}: flag {flag} not in {source}")

        if kind == "human_promise":
            if source is not None:
                errors.append(f"{heading}: human_promise must set source: null")
            if owner != "human":
                errors.append(f"{heading}: human_promise expects owner=human")
            if generated:
                errors.append(f"{heading}: generated marker on human-owned heading")

    return errors


def main() -> int:
    parser = argparse.ArgumentParser()
    parser.add_argument("command", choices=["inventory", "check"])
    parser.add_argument("--docs-root", default="docs")
    parser.add_argument("--map", default="docs/source-map.yaml")
    args = parser.parse_args()
    if args.command == "inventory":
        for item in inventory(Path(args.docs_root)):
            print(item)
        return 0
    errors = check(Path(args.map))
    for error in errors:
        print(error, file=sys.stderr)
    return 1 if errors else 0


if __name__ == "__main__":
    sys.exit(main())

Wire the check into a documentation job that does not publish anything. The Makefile target is a proposal for local use; replace it with the repository's existing CI syntax.

# Proposal local gate
.PHONY: doc-bindings
doc-bindings:
    python3 scripts/check_doc_bindings.py check --map docs/source-map.yaml

pip install pyyaml
python3 scripts/check_doc_bindings.py check --map docs/source-map.yaml
# exit 0 only when every H2 is mapped and every bound path exists

What the checker does not prove

Path existence does not mean the paragraph is true, complete, or kind to the reader. A test can be outdated, a schema can omit a live field, and a help fixture can lag the parser by one commit. Token overlap on --flags is still a string check, so a wrong description of a real flag will pass. Those remain human review problems, which is why human_promise headings never enter the draftable set.

The map also does not replace product legal review. Compatibility statements, export-control notes, and incident-response pages need an accountable author even when a nearby how-to is generated from tests. If a team cannot name that author, the page should not ship, generated or not.

Who should not use this approach

Skip this workflow when the repository has no tests, no schema, and no frozen help text, because the map would bind headings to nothing checkable. Skip it for threat models, regulated claims, medical or safety procedures, and customer-facing incident reports. Skip it when the documentation is the product contract, such as a public SLA, because a draft worker has no standing to write that contract. In those cases, write the page, cite the owner, and keep models off the file.

Teams that already generate an entire site from a single prompt will find this workflow slower on the first week. The cost is the point: unmapped headings fail closed, and that failure is cheaper than a fluent page that invents a flag. If the goal is volume of pages rather than checkable procedures, a source map will feel like friction and should not be adopted as theater.

Keep the map in the same pull request as the prose, and reject drafts that cannot cite a file. A free model and a free server can fill bound headings; they cannot own a promise.