DEV Community

Morgan Sun
Morgan Sun

Posted on

Extract, Draft, Sign: Regen-Safe Lanes for AI Docs

A regenerated README often looks like progress. The install block is tighter. A stale caption disappears. Then a reviewer finds that the same commit also rewrote the security contact, dropped two SLO numbers, and turned a retention rule into a hedge.

The model did not sneak those edits. It treated the file as one string. Regeneration without lanes will keep doing that.

This article proposes a split you can check in CI: extract, draft, and sign. A model may write only the draft lane. Deterministic tools own extract. A named human owns sign. The checker below is sample code, not a production audit and not a legal control.

The failure mode is whole-file regen

Most doc pipelines still do this:

  1. Concatenate the old markdown.
  2. Ask a model to update it for the latest API.
  3. Overwrite the path.

That loop is cheap. It is also how a metaphor and a contractual sentence become the same token stream. If the process cannot name which paragraphs are allowed to change, every regen is a silent policy edit.

A useful split is not "AI versus human" as a vibe. It is three writers with three failure costs. Extract fails when a dump script is stale. Draft fails when the prose is wrong or vague. Sign fails when a promise moves without a person noticing.

Decision table: who is allowed to write

Doc fragment Source of truth Lane Model may draft? Human must sign?
CLI flags, env vars, HTTP routes --help, OpenAPI, code extract No Own the extractor
Samples that must compile tests / doctest extract No Own the test
Install narrative, "why this shape" none draft Yes Review
Architecture walkthrough none draft Yes Review
Security contact, threat-model claims policy sign No Yes
SLO numbers, error budgets SRE / contract sign No Yes
License, embargo, export notes legal sign No Yes
Changelog dates and SHAs git extract No Own git log
"We never store X" legal / security sign No Yes
Analogies, diagrams-as-prose none draft Yes Review

If wrong wording can page someone, leak PII, or change a customer promise, it is not draft. Put it in sign. If a fragment can be generated from a command, do not ask a model to remember it. Put it in extract.

A layout that makes the split visible

Keep lanes in different paths when you can. Mixed files need fences.

Proposed layout (example, not a standard):

docs/
  extracted/          # tools write this; CI rejects model authors
  drafted/            # model output; always reviewable
  signed/             # humans only
  lanes.yaml          # glob + heading rules
Enter fullscreen mode Exit fullscreen mode

docs/lanes.yaml:

version: 1
# Unmatched files fail closed in CI.
default_lane: deny
rules:
  - glob: "docs/extracted/**"
    lane: extract
    writers: [tool]
  - glob: "docs/drafted/**"
    lane: draft
    writers: [model, human]
  - glob: "docs/signed/**"
    lane: sign
    writers: [human]
  - glob: "README.md"
    lane: mixed
    heading_overrides:
      Install: draft
      Security: sign
      Support: sign
      API: extract
Enter fullscreen mode Exit fullscreen mode

Inside a mixed file, fence the blocks so a regen script can splice draft text without touching signed bytes:

<!-- lane:extract source:scripts/dump-help.sh -->
## CLI
<!-- /lane -->

<!-- lane:draft -->
## Why two config files
<!-- /lane -->

<!-- lane:sign owner:security expires:2026-12-31 -->
## Security
Report vulnerabilities to security@example.com.
Do not open a public issue for active exploits.
<!-- /lane -->
Enter fullscreen mode Exit fullscreen mode

HTML comments are a convention. Some site generators strip them. If yours does, use path isolation or a sibling lanes.json of heading IDs. Do not treat a comment as access control. Git will still accept a force-push.

Extract is a command, not a prompt

Keep extract boring. A model that "summarizes --help" will invent flags on a bad day.

#!/usr/bin/env bash
# scripts/dump-help.sh — extract lane. Unexecuted example.
set -euo pipefail
out=docs/extracted/cli.md
{
  echo "<!-- lane:extract source:scripts/dump-help.sh -->"
  echo "## CLI"
  echo
  echo '```

text'
  ./tool --help
  echo '

```'
  echo "<!-- /lane -->"
} > "$out"
Enter fullscreen mode Exit fullscreen mode

Changelog dates belong in the same lane:

git log --pretty=format:'- %ad %h %s' --date=short -n 20 > docs/extracted/changelog.md
Enter fullscreen mode Exit fullscreen mode

If the binary is missing in CI, fail the job. Do not fall back to a model "from memory." That fallback is how extract turns into draft without anyone renaming the lane.

Artifact: a regen-clobber checker

The following example is unexecuted sample code. It is meant to fail a build when a model-authored change edits a sign block, writes an extract path, or adds a markdown file with no rule.

#!/usr/bin/env python3
"""docs_lane_check.py — example checker, not a compliance tool."""
from __future__ import annotations

import argparse
import fnmatch
import hashlib
import json
import re
import sys
from pathlib import Path

import yaml  # PyYAML

LANE_RE = re.compile(
    r"<!--\s*lane:(extract|draft|sign)\b(.*?)-->(.*?)<!--\s*/lane\s*-->",
    re.S,
)


def sha(text: str) -> str:
    return hashlib.sha256(text.encode()).hexdigest()[:16]


def load_rules(path: Path) -> dict:
    data = yaml.safe_load(path.read_text())
    if data.get("version") != 1:
        raise SystemExit("unsupported lanes.yaml version")
    return data


def lane_for(relpath: str, rules: dict) -> tuple[str, dict]:
    for rule in rules["rules"]:
        if fnmatch.fnmatch(relpath, rule["glob"]):
            return rule["lane"], rule
    return rules.get("default_lane", "deny"), {}


def blocks(markdown: str) -> list[dict]:
    out = []
    for m in LANE_RE.finditer(markdown):
        lane, attrs, body = m.group(1), m.group(2), m.group(3)
        owner = None
        if "owner:" in attrs:
            owner = attrs.split("owner:")[1].split()[0]
        out.append({"lane": lane, "owner": owner, "sha": sha(body.strip())})
    return out


def missing_sign_fences(markdown: str, rule: dict) -> list[str]:
    violations = []
    overrides = rule.get("heading_overrides") or {}
    for heading, required in overrides.items():
        if required != "sign":
            continue
        pattern = (
            rf"<!--\s*lane:sign\b[^>]*-->.*?^#+\s+{re.escape(str(heading))}\b"
        )
        if not re.search(pattern, markdown, re.M | re.S):
            violations.append(
                f"heading {heading!r} must sit inside a lane:sign fence"
            )
    return violations


def main() -> int:
    p = argparse.ArgumentParser()
    p.add_argument("--rules", default="docs/lanes.yaml")
    p.add_argument("--role", choices=["model", "tool", "human"], required=True)
    p.add_argument("--base", help="JSON object of path -> previous file text")
    p.add_argument("paths", nargs="+")
    args = p.parse_args()

    rules = load_rules(Path(args.rules))
    base = json.loads(Path(args.base).read_text()) if args.base else {}
    errors: list[str] = []

    for path in args.paths:
        lane, rule = lane_for(path, rules)
        text = Path(path).read_text()
        if lane == "deny":
            errors.append(f"{path}: no lane rule (fail closed)")
            continue
        if lane == "sign" and args.role == "model":
            errors.append(f"{path}: model must not write sign lane")
        if lane == "extract" and args.role == "model":
            errors.append(f"{path}: model must not write extract lane")
        if lane == "mixed":
            errors.extend(f"{path}: {v}" for v in missing_sign_fences(text, rule))
            if args.role == "model" and path in base:
                old_sign = {
                    b["sha"] for b in blocks(base[path]) if b["lane"] == "sign"
                }
                new_sign = {b["sha"] for b in blocks(text) if b["lane"] == "sign"}
                if old_sign != new_sign:
                    errors.append(
                        f"{path}: sign-block hash changed under role=model"
                    )

    for item in errors:
        print(item, file=sys.stderr)
    return 1 if errors else 0


if __name__ == "__main__":
    sys.exit(main())
Enter fullscreen mode Exit fullscreen mode

A regen script should rewrite only docs/drafted/** and fenced lane:draft bodies. Extract jobs write only docs/extracted/**. Humans edit docs/signed/**. CI should call the checker with the role that actually produced the diff.

# sketch only — wire to your own CI
- name: lane-check-model
  run: >
    python docs_lane_check.py
    --role model
    --rules docs/lanes.yaml
    --base base-docs.json
    $(git diff --name-only origin/main -- '*.md')
Enter fullscreen mode Exit fullscreen mode

Build base-docs.json from the merge base, not from the working tree after the model ran. Otherwise the hash comparison is a no-op.

Fixture plan

Run these as labeled fixtures. A green local run is not evidence that the policy is binding.

  1. Happy draft: change only a lane:draft body with --role model. Expect exit 0.
  2. Clobber sign: alter the security email inside lane:sign with --role model. Expect exit 1.
  3. Extract trespass: a model-authored commit touching docs/extracted/cli.md. Expect exit 1.
  4. Fail closed: add docs/notes.md with no glob. Expect exit 1.
  5. Fence missing: README.md has ## Security but no lane:sign fence. Expect exit 1.
  6. Tool may extract: --role tool writing docs/extracted/**. Expect exit 0.
  7. Human may sign: --role human changing a sign block. Expect exit 0.

Where a free model belongs in this pipeline

The draft lane is the only place a general-purpose model should write. Extract should be a script. Sign should be a person whose name appears in owner:.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

MonkeyCode is an open-source project with free model access and a free server option. Those are the only product facts used here; current limits belong on the project's own docs at the time you run the job. In this workflow the mapping is narrow: send docs/drafted/** prompts through the free model path, and run docs_lane_check.py plus extractors on the free server so the checker is not a laptop-only habit. Do not send docs/signed/** to any model, free or paid. The server does not own the guarantee. The owner: field does.

If you try that split, keep fail-closed rules. Free hosting does not make the sign lane optional.

Limitations

  • Comment fences are brittle. A formatter or CMS can drop them. Prefer separate paths over mixed files.
  • Hashing a sign body does not prove the text is true. It only proves the model did not change those bytes in that diff.
  • Extractors drift when flags change and the dump script is not in CI.
  • lanes.yaml is not an authorization system.
  • This article does not publish model names, token quotas, hardware specs, or uptime. Those change.
  • The checker is sample code. No benchmark numbers are claimed.

Who should not use this

  • Teams with no named human for the sign lane. A bot cannot sign.
  • Regulated filings, customer contracts, and security advisories that need counsel, not a glob.
  • Repos that generate large piles of markdown with no path convention. Fix layout first.
  • Anyone hoping a model will keep SLOs, prices, or "we never store X" statements in sync. Those are sign-lane edits.

Whole-file regen will keep looking faster. The cost shows up later, in a sentence nobody meant to publish. Split the lanes before the next rewrite, and keep the model in the only lane it can fail safely: draft.

Top comments (0)