DEV Community

Avery Lin
Avery Lin

Posted on

Tag Every Docs Heading as Fixture, Review, or Policy Before Generation

Generated documentation stays trustworthy when every heading is classified before a model is allowed to draft a single sentence. Fixture-backed examples can be reconstructed from recorded HTTP traffic, while policy statements about support, uptime, and legal commitments cannot. This article presents a heading-lane classifier, a merge gate, and a pytest suite that fail builds on lane violations. The workflow is a labeled proposal with sample files, not a report of measured production outcomes.

Why heading lanes beat post-generation cleanup

Reviewers waste time when a draft mixes reconstructible examples with irreversible promises inside the same Markdown file. A model can restate a captured request and response with high fidelity, because those bytes already exist in a fixture. A model cannot own an SLA, a deprecation calendar, or a support escalation path, because those statements are organizational commitments. Classification at heading time prevents the merge of mixed files rather than asking humans to unmix them later.

The three lanes are mutually exclusive for any given heading, and the classifier rejects outlines that omit a lane. Fixture headings must point at a recorded HTTP transcript or contract-test dump. Review headings may paraphrase a schema fragment, but a human still signs the rendered table. Policy headings never enter the model context window, and their files are owned by a named maintainer with a review date.

The outline contract

Store the heading map beside the docs tree, not inside prompt text, so CI can hash it independently. Each record needs a stable id, a heading string, a lane, and a source or owner. Dates below are placeholders for your calendar; replace them before any real merge.

# docs/outline.yaml — proposal sample, not a live catalog
version: 1
sections:
  - id: create-session-example
    heading: "Create a session with an API key"
    lane: fixture
    source: tests/fixtures/http/create_session_201.http
    output: docs/generated/create-session-example.md
  - id: session-headers
    heading: "Session response headers"
    lane: review
    source: openapi.yaml#/components/headers/SessionHeaders
    output: docs/generated/session-headers.md
    reviewer: api-docs
  - id: session-sla
    heading: "Session availability and support hours"
    lane: policy
    output: docs/human/session-sla.md
    owner: support-lead
    reviewed: "2026-08-20"
Enter fullscreen mode Exit fullscreen mode

The output paths keep generated files and human files in separate directories so a naive cp cannot overwrite policy. Review-lane files still land under docs/generated/ because they remain disposable after a schema change. Policy files live under docs/human/ and are listed in CODEOWNERS with the same owner field.

Artifact: classify, generate, and refuse mixed writes

The following module is a compact proposal you can run locally. It does not call a network model; it only enforces the lane contract and emits a prompt payload for fixture headings. Treat it as unexecuted sample code until you add your own HTTP fixtures.

# tools/docs_lanes.py
from __future__ import annotations

import hashlib
import json
from pathlib import Path
from typing import Any

import yaml

LANES = {"fixture", "review", "policy"}
ROOT = Path(__file__).resolve().parents[1]


def load_outline(path: Path) -> dict[str, Any]:
    data = yaml.safe_load(path.read_text(encoding="utf-8"))
    if data.get("version") != 1:
        raise ValueError("outline version must be 1")
    return data


def validate_section(section: dict[str, Any]) -> list[str]:
    errors: list[str] = []
    lane = section.get("lane")
    if lane not in LANES:
        errors.append(f"{section.get('id')}: unknown lane {lane!r}")
        return errors
    heading = section.get("heading") or ""
    if len(heading.split()) < 3:
        errors.append(f"{section['id']}: heading too short to be unique")
    output = ROOT / section["output"]
    if lane == "policy":
        if not section.get("owner"):
            errors.append(f"{section['id']}: policy heading needs owner")
        if not section.get("reviewed"):
            errors.append(f"{section['id']}: policy heading needs reviewed date")
        if "docs/human/" not in section["output"]:
            errors.append(f"{section['id']}: policy output must live under docs/human/")
        if not output.exists():
            errors.append(f"{section['id']}: missing policy file {output}")
    else:
        source = section.get("source")
        if not source:
            errors.append(f"{section['id']}: {lane} heading needs source")
        elif lane == "fixture":
            fixture = ROOT / source
            if not fixture.exists():
                errors.append(f"{section['id']}: missing fixture {fixture}")
        if "docs/generated/" not in section["output"]:
            errors.append(f"{section['id']}: generated output must live under docs/generated/")
    return errors


def fixture_payload(section: dict[str, Any]) -> dict[str, str]:
    raw = (ROOT / section["source"]).read_text(encoding="utf-8")
    digest = hashlib.sha256(raw.encode("utf-8")).hexdigest()[:16]
    return {
        "id": section["id"],
        "heading": section["heading"],
        "lane": "fixture",
        "source_sha256_16": digest,
        "transcript": raw,
        "instructions": (
            "Restate the HTTP transcript as a fenced example. "
            "Do not add SLAs, timelines, or support promises. "
            "Cite the source filename in the first paragraph."
        ),
    }


def build_jobs(outline: dict[str, Any]) -> dict[str, Any]:
    errors: list[str] = []
    jobs: list[dict[str, str]] = []
    for section in outline["sections"]:
        errors.extend(validate_section(section))
        if section.get("lane") == "fixture" and not errors:
            jobs.append(fixture_payload(section))
    if errors:
        raise SystemExit("\n".join(errors))
    return {"jobs": jobs, "skipped_lanes": ["review", "policy"]}


if __name__ == "__main__":
    outline = load_outline(ROOT / "docs/outline.yaml")
    print(json.dumps(build_jobs(outline), indent=2))
Enter fullscreen mode Exit fullscreen mode

A matching HTTP fixture keeps the example lane honest. The file below is a synthetic transcript for the classifier to hash; it is not captured production traffic.

# tests/fixtures/http/create_session_201.http
POST /v1/sessions HTTP/1.1
Host: api.example.test
Authorization: Bearer TESTKEY
Content-Type: application/json

{"ttl_seconds": 3600}

HTTP/1.1 201 Created
Content-Type: application/json
X-Request-Id: req_test_001

{"session_id":"sess_test_001","expires_in":3600}
Enter fullscreen mode Exit fullscreen mode

Numbered workflow

  1. Record fixtures from contract tests. Dump request and response bytes after each passing test, using a host that never appears in customer documentation. Keep secrets out of the transcript; replace tokens with TESTKEY before the file is committed. Name files after status codes so heading ids stay stable across copy edits.

  2. Classify every planned heading. Add one outline record per H2 you intend to publish, and refuse leftover Markdown files that are not listed. Map examples to fixture, schema tables to review, and commitments to policy. If a heading needs both an example and a promise, split it into two records with two output paths.

  3. Emit model jobs only for the fixture lane. Run python tools/docs_lanes.py and pipe JSON to the drafting step you already use. The payload includes the transcript and a hard instruction to cite the filename. Review-lane tables can be rendered with a deterministic OpenAPI walk; they do not need a model at all.

  4. Write generated files through an allowlisted merge. Accept model output only when the first heading matches section["heading"] and the body contains the fixture basename. Reject files that mention uptime percentages, indemnities, or support hours, because those phrases belong in policy files.

  5. Gate the human directory on ownership metadata. Fail CI when a policy file changes without a matching owner review stamp in outline.yaml. Fail CI when a generated file path appears under docs/human/. Keep CODEOWNERS aligned with the owner field so review routing does not drift from the outline.

# tests/test_docs_lanes.py — proposal checks
from pathlib import Path

import yaml

ROOT = Path(__file__).resolve().parents[1]
FORBIDDEN = ("uptime", "SLA", "indemnity", "24/7", "we promise")


def test_policy_files_are_not_generated():
    outline = yaml.safe_load((ROOT / "docs/outline.yaml").read_text())
    for section in outline["sections"]:
        if section["lane"] == "policy":
            assert section["output"].startswith("docs/human/")


def test_generated_markdown_stays_in_fixture_vocabulary():
    gen = ROOT / "docs/generated"
    if not gen.exists():
        return
    for path in gen.glob("*.md"):
        text = path.read_text(encoding="utf-8")
        lowered = text.lower()
        for token in FORBIDDEN:
            assert token.lower() not in lowered, f"{path} contains {token}"
Enter fullscreen mode Exit fullscreen mode
python tools/docs_lanes.py > /tmp/fixture_jobs.json
python -m pytest tests/test_docs_lanes.py -q
git diff --check -- docs/human
Enter fullscreen mode Exit fullscreen mode

Where a free drafting environment fits

Fixture-lane prompts are small, repetitive, and easy to rerun after every contract-test change, which makes them a poor fit for an always-on paid endpoint. Teams that already draft in MonkeyCode can feed fixture_jobs.json into that workspace and keep policy files out of the prompt. Disclosure: This article was prepared as part of MonkeyCode's product outreach. Free model access and a free server option are sufficient to iterate on restatements of fixtures, without any claim about named models, quotas, hardware, duration, or scores.

The important constraint is still the outline, not the editor. If the classifier would reject a heading, the drafting environment must not see that heading. Review-lane tables should stay on a local OpenAPI renderer so the model never invents columns that the schema does not define.

Limitations

This split does not make examples correct; it only makes them traceable to a file that a test can hash. Fixtures that were recorded against a mock will document the mock, including status codes the production gateway never emits. Review-lane schema tables still need a human pass when OpenAPI descriptions are themselves stale. Policy dates in the outline are not legal review, and a reviewed stamp does not replace counsel.

The forbidden-token list is a coarse net. A careful model can write a support promise without using the words in FORBIDDEN, which is why policy headings must remain out of the context window entirely. The classifier also assumes one heading per output file; multi-heading blobs defeat both the merge gate and the vocabulary scan.

Who should not use this approach

Skip this pipeline if your documentation is a single narrative tutorial with no recorded traffic and no OpenAPI document. Skip it if legal or support teams require every paragraph, including examples, to pass counsel before publish. Skip it if your HTTP fixtures contain live credentials, customer payloads, or hosts that must not appear in a public repository. Skip it if you need the model to invent migration stories, pricing, or timelines that no fixture can prove.

Teams with a large existing Markdown tree should classify new headings first rather than batch-labeling years of mixed files. A forced migration creates false fixture labels on prose that was never reconstructed from traffic. The cheaper path is to freeze old files as policy, then grow the generated directory only from new contract tests.

What the model may draft, and what a human still owns

After classification, the model may draft only the restatement of a hashed transcript under a pre-registered heading. A human still owns the outline itself, every review-lane table sign-off, and every file under docs/human/. That boundary is the product: examples stay cheap to regenerate, while promises stay slow to change. If a heading cannot name a fixture, a schema pointer, or a policy owner, it does not belong in the next docs merge.

Top comments (0)