DEV Community

Avery Lin
Avery Lin

Posted on

Give Every Docs Heading a Draft Permit Before a Model Writes

Generated documentation stays honest only when each heading declares, in a file, what a model may write. Chat windows and whole-file prompts hide that decision and let unowned promises leak into reference pages. A heading-level permit file is a cheaper control plane than reviewing every sentence after the model returns. This article describes a small classifier, a checker script, and a workflow that keeps human judgment off the model path.

Treat the heading as the permission boundary

Files are too coarse as a permission unit because one markdown document mixes extractable facts with untested promises. Sentences are too fine as a permission unit because reviewers cannot stably address them across regenerations. Headings sit in the middle: they are stable, linkable, and already the unit readers use to navigate a reference. Each heading therefore receives exactly one permit before any model is invited to draft text.

The permit file does not score writing quality, and it does not rank competing language models. It answers only two operational questions that a continuous integration pipeline can enforce without another model. The first question is whether a model may emit any text under that heading. The second question names which frozen source files that emitted text is allowed to rest on.

A four-value permit vocabulary

Keep the vocabulary small enough that a linter can enforce it without calling a second model. Four values cover the cases that usually appear in API and product documentation. QUOTE and RESTATE are the only permits that should ever reach a drafting model. HUMAN_ONLY and FORBIDDEN headings remain documentation, but they are not generation targets.

Permit Model may emit Required inputs Human still owns
QUOTE Identifiers, status codes, field names, copied tables Frozen schema, OpenAPI, or test names Surrounding interpretation
RESTATE Paraphrase of extractable facts with a citation Frozen sources plus a source map Whether the paraphrase is complete
HUMAN_ONLY Nothing; the heading stays empty or handwritten None Compatibility, SLAs, roadmaps, recommendations
FORBIDDEN Nothing; CI fails if generated text appears None Legal, pricing, and security guarantees

Mixing generation targets with human-owned promises is how untested claims enter a changelog. The vocabulary exists to make that mix a build failure instead of a review comment. If a heading cannot choose one row in the table, split the heading before any draft runs.

Worked example: a payments reference tree

The following tree is a labeled proposal, not a production corpus from a live company. It shows how permits attach to headings rather than to entire files. Paths are relative to the repository root so the checker can resolve sources. Treat every path as an example identifier, not as evidence about a real payments product.

docs/
  reference/
    charges.md
    errors.md
  product/
    sla.md
    security.md
permits.yaml
openapi/charges.yaml
tests/charges_create_test.py
Enter fullscreen mode Exit fullscreen mode

A compact permit file can address headings with a path and an optional markdown anchor:

# permits.yaml — proposal for heading-level draft control
version: 1
defaults:
  unmatched_heading: HUMAN_ONLY
headings:
  - id: docs/reference/charges.md#create-a-charge
    permit: RESTATE
    sources:
      - openapi/charges.yaml
      - tests/charges_create_test.py
  - id: docs/reference/charges.md#idempotency-keys
    permit: QUOTE
    sources:
      - openapi/charges.yaml
  - id: docs/reference/charges.md#when-to-retry
    permit: HUMAN_ONLY
    sources: []
  - id: docs/product/sla.md
    permit: HUMAN_ONLY
    apply: file
  - id: docs/product/security.md
    permit: FORBIDDEN
    apply: file
Enter fullscreen mode Exit fullscreen mode

Unmatched headings default to HUMAN_ONLY so a new section cannot become a generation target by omission. That default is the opposite of a blank prompt, which treats every heading as fair game. File-level apply: file rows cover product pages that should never be split into generated fragments.

Numbered workflow

Follow the steps in order, because skipping the checker before the model step reintroduces the original failure mode.

  1. Freeze the sources that QUOTE and RESTATE headings may read, and commit schemas, fixtures, and tests first.
  2. Author or update permits.yaml in the same change as the heading, and never let a model invent permits.
  3. Run the checker in continuous integration so unmatched headings, missing sources, and FORBIDDEN paths fail the build.
  4. Invoke a model only for headings whose permit is QUOTE or RESTATE, and pass only the listed source files.
  5. Merge model output into a generated overlay directory, and never write it into handwritten HUMAN_ONLY files.
  6. Require a human reviewer on any HUMAN_ONLY heading, even when neighboring sections in the file were generated.

A Makefile keeps the mechanical commands next to the permit file:

.PHONY: permit-check permit-list
permit-check:
    python3 -m pip install -q pyyaml
    python3 check_permits.py
permit-list:
    python3 list_allowed.py
Enter fullscreen mode Exit fullscreen mode

Artifact: a checker you can run locally

The script below is an unexecuted example. It parses a simplified permit file, walks markdown headings, and exits non-zero when a heading would be unsafe to generate. It does not call a model and it does not download sources.

#!/usr/bin/env python3
"""Heading permit checker. Example only; not a production linter."""
from __future__ import annotations

import pathlib
import re
import sys
from typing import Any

try:
    import yaml
except ImportError:
    print("Install pyyaml before running this example.", file=sys.stderr)
    sys.exit(2)

HEADING_RE = re.compile(r"^(#{2,4}) +(.+?) *$")
GENERATED_BANNER = "<!-- generated:permit="


def slug(title: str) -> str:
    text = title.strip().lower()
    text = re.sub(r"[^a-z0-9]+", "-", text)
    return text.strip("-")


def load_permits(path: pathlib.Path) -> dict[str, Any]:
    data = yaml.safe_load(path.read_text())
    index: dict[str, dict[str, Any]] = {}
    for row in data.get("headings", []):
        index[row["id"]] = row
    return {
        "unmatched": data.get("defaults", {}).get("unmatched_heading", "HUMAN_ONLY"),
        "index": index,
    }


def headings_in(md_path: pathlib.Path, root: pathlib.Path) -> list[str]:
    rel = md_path.relative_to(root).as_posix()
    found = [rel]
    for line in md_path.read_text().splitlines():
        match = HEADING_RE.match(line)
        if match:
            found.append(f"{rel}#{slug(match.group(2))}")
    return found


def permit_for(hid: str, permits: dict[str, Any]) -> dict[str, Any]:
    index = permits["index"]
    if hid in index:
        return index[hid]
    file_id = hid.split("#", 1)[0]
    file_row = index.get(file_id)
    if file_row and file_row.get("apply") == "file":
        return file_row
    return {"id": hid, "permit": permits["unmatched"], "sources": []}


def check_tree(root: pathlib.Path, permit_path: pathlib.Path) -> list[str]:
    permits = load_permits(permit_path)
    errors: list[str] = []
    docs_root = root.joinpath("docs")
    if not docs_root.exists():
        return ["docs/ directory is missing"]
    for md in sorted(docs_root.rglob("*.md")):
        text = md.read_text()
        for hid in headings_in(md, root):
            row = permit_for(hid, permits)
            permit = row["permit"]
            if permit == "FORBIDDEN" and GENERATED_BANNER in text:
                errors.append(f"{hid}: FORBIDDEN heading contains generated banner")
            if permit in {"QUOTE", "RESTATE"}:
                sources = row.get("sources") or []
                if not sources:
                    errors.append(f"{hid}: {permit} requires at least one source")
                for src in sources:
                    src_path = src.split("#", 1)[0]
                    if not (root / src_path).exists():
                        errors.append(f"{hid}: missing source {src_path}")
            if permit == "HUMAN_ONLY" and GENERATED_BANNER in text and "#" not in hid:
                errors.append(f"{hid}: HUMAN_ONLY file contains generated banner")
    return errors


def main() -> int:
    root = pathlib.Path(".").resolve()
    errors = check_tree(root, root / "permits.yaml")
    if errors:
        print("permit check failed:")
        for item in errors:
            print(f"  - {item}")
        return 1
    print("permit check passed")
    return 0


if __name__ == "__main__":
    sys.exit(main())
Enter fullscreen mode Exit fullscreen mode

Run it from the repository root after permits.yaml exists:

python3 -m pip install pyyaml
python3 check_permits.py
echo $?
Enter fullscreen mode Exit fullscreen mode

A second script can list the headings a model is allowed to draft, which is the input set for any later generation step.

#!/usr/bin/env python3
"""Print QUOTE and RESTATE headings. Example only."""
from pathlib import Path

from check_permits import headings_in, load_permits, permit_for

root = Path(".").resolve()
permits = load_permits(root / "permits.yaml")
for md in sorted((root / "docs").rglob("*.md")):
    for hid in headings_in(md, root):
        row = permit_for(hid, permits)
        if row["permit"] in {"QUOTE", "RESTATE"}:
            sources = ", ".join(row.get("sources") or [])
            print(f"{row['permit']:8} {hid} <- {sources}")
Enter fullscreen mode Exit fullscreen mode
python3 list_allowed.py
Enter fullscreen mode Exit fullscreen mode

That list is the contract the model must honor during drafting. Anything absent from the list is out of scope, including adjacent headings in the same markdown file. Reviewers can diff the list across pull requests when headings move.

Draft only the permitted headings

When a heading is QUOTE, the draft should copy names and codes from the listed sources and should not invent synonyms. When a heading is RESTATE, the draft may paraphrase, but every paragraph should cite a source path in a machine-readable comment. A reviewer then checks completeness of the restatement, not authorship of field names that already exist in the schema.

A thin driver can write one heading at a time. The example below is a template, not a vendor SDK, and it assumes a local draft_heading function that you control.

from pathlib import Path


def draft_allowed(heading_id: str, permit: str, sources: list[str]) -> str:
    if permit not in {"QUOTE", "RESTATE"}:
        raise ValueError(f"{heading_id} is not a generation target")
    blobs = []
    for src in sources:
        blobs.append(Path(src.split("#", 1)[0]).read_text())
    # Call a local drafting function with blobs only.
    return draft_heading(heading_id, permit, blobs)
Enter fullscreen mode Exit fullscreen mode

Keep HUMAN_ONLY files out of that loop on purpose. If a model cannot see docs/product/sla.md, it cannot quietly rewrite an uptime promise while restating a status code table. Generated files should start with a banner that names the permit, so the checker can reject overlays that landed in the wrong tree.

<!-- generated:permit=RESTATE sources=openapi/charges.yaml -->
## Create a charge

`POST /charges` accepts `amount` and `currency` as required fields.
<!-- cite: openapi/charges.yaml -->
Enter fullscreen mode Exit fullscreen mode

Where a hosted draft step can sit

Some teams prefer not to run drafting models on developer laptops, especially when source files already live in CI. MonkeyCode's free model access and free server option can host that narrow drafting step without extra product claims. Disclosure: This article was prepared as part of MonkeyCode's product outreach. Send one permitted heading and its listed sources, receive a draft, then discard the session.

The product is optional in this workflow, and the permit file still helps if every draft stays handwritten. Do not send HUMAN_ONLY or FORBIDDEN paths to any hosted runner, free or otherwise. A useful experiment is to list allowed headings for one reference file and confirm they match reviewer intent.

Failure modes the checker is meant to catch

The following table is a test plan for fixtures, not a measured production incident log from a live corpus. Add each fixture as a small docs tree under testdata/ and assert the process exit code before enabling drafts. The checker performs mechanical work only, so fluent but unsupported paraphrases that cite the wrong field still need humans.

Fixture Expected checker result Why it matters
Missing source path fail RESTATE without a file is indistinguishable from a blank prompt
New heading, no permit row HUMAN_ONLY default Omission must not grant generation rights
Generated banner under FORBIDDEN fail Security pages must not be rewritten by a draft job
QUOTE heading with empty sources fail Copied identifiers still need a frozen origin
HUMAN_ONLY file with banner fail Promises should not arrive through the overlay

Store each fixture under testdata/ with its own permits.yaml and assert python3 check_permits.py from that directory. Mechanical failures should stay mechanical, which is why the checker never asks a model whether a heading looks safe enough. Completeness of a RESTATE paragraph remains a human gate after the exit code is zero.

Limitations

Heading permits do not prove that a paraphrase is true, and they do not replace a completeness review. They only prove that a draft was allowed to exist and that listed files were present at check time. They also assume headings remain stable across edits, which is not true for every writing culture. If writers retitle sections without updating permits.yaml, unmatched headings fall back to HUMAN_ONLY instead of generating.

The four-value vocabulary cannot express partial ownership inside a single paragraph without splitting the heading. If a section must mix a copied status code with a compatibility promise, split the heading before assigning permits. Refusing that split is how HUMAN_ONLY text gets laundered through a RESTATE draft and into production docs.

This approach is a poor fit for narrative blogs, incident reviews, and marketing pages whose value is judgment rather than extractable fact. It is also a poor fit for teams that will not freeze schemas or tests before drafting. Without frozen inputs, QUOTE and RESTATE collapse into ordinary prompting, and the permit file becomes decoration.

Who should not use this

Skip the permit file if the documentation set is only a handful of pages and every sentence is already handwritten. Skip the generation overlay if legal or security reviewers require a full human draft and will reject machine text. Skip hosted drafting if source files cannot leave your network, even when a free server happens to be available.

In those cases, keep the four-value vocabulary as a review checklist and do not add a model to the pipeline. The core conclusion does not depend on any vendor, hosted runner, or particular drafting stack. Headings that encode promises stay human, headings that restate frozen files may be drafted, and the rest is misclassification.

Top comments (0)