DEV Community

Avery Lin
Avery Lin

Posted on

Scope the Doc Before the Prompt: What a Model May Draft and What a Human Must Own

The cheapest place to control a model-drafted documentation change is the prompt boundary, not the pull request. Before any request leaves your machine, split the page into sections a human must own, sections a model may draft, and sections neither should touch. Then gate that split with a script small enough to read in one sitting and fast enough to run in CI.

What follows is a two-command workflow built on a single-file Python script with no third-party dependencies. It produces a manifest describing every block of a Markdown page, assigns each block a scope and a reason code, and fails a merge when a human-owned scope has no named owner. The drafting step, where a remote model does the tedious prose work, comes last and touches the smallest possible surface.

Why the input side deserves the first gate

Review-only workflows share one structural weakness: the model has already seen and rewritten everything by the time a human looks. Once a commitment sentence has been regenerated, the reviewer has to reconstruct the original number from memory, a ticket, or a diff that no longer shows the source. That reconstruction is where quiet errors enter documentation, and it scales badly as page count grows.

Input-side scoping changes the failure mode rather than the review checklist. A section that states a quota, a guarantee, or an executable command never enters the prompt at all, so it cannot be silently reworded. The reviewer's job shrinks from "verify every sentence" to "confirm the split was right and the prose reads well".

This is not a substitute for human ownership of commitments; it is the mechanism that makes such ownership enforceable. The rest of the post is the mechanism, the fixtures that keep it honest, and the cases where it is the wrong tool.

The three scopes

Scope Contents Who writes it Merge rule
A, human-owned fenced code, security claims, numbers with units, quota, pricing, deprecation, contract language one named owner owner plus evidence link recorded in owners.json
B, model-draftable narrative prose without quantities or guarantees: overviews, mental models, glossary entries, "why it works" sections model drafts, one human reads style and accuracy read, no per-sentence sign-off
C, generated API reference from a spec, CLI help dumps, schema tables the generator never hand-edited, regenerate instead

The split is deliberately conservative. Fenced code lands in scope A even when it is illustrative, because someone must have executed it at least once before a reader copies it. Scope B is where models are genuinely useful and genuinely low-risk: prose that explains, connects, and restates without promising anything.

Step 1: Partition the page with a stdlib script

Save the following as docs_scope.py. It reads one Markdown file, keeps fenced blocks intact, tracks the nearest heading as a section slug, and prints a JSON manifest. The annotation import keeps it runnable on older Python 3 versions.

#!/usr/bin/env python3
"""docs_scope.py - split a Markdown doc into drafting scopes, then gate it.

  python docs_scope.py split docs/retries.md > retries.scope.json
  python docs_scope.py gate retries.scope.json owners.json
"""
from __future__ import annotations

import hashlib
import json
import re
import sys
from pathlib import Path

FENCE = re.compile(r"^```

(?P<lang>[A-Za-z0-9_+.-]*)\s*$")
HEADING = re.compile(r"^(?P<hashes>#{1,6})\s+(?P<text>.+?)\s*$")
SECURITY = re.compile(
    r"\b(encrypt\w*|TLS|SOC ?2|HIPAA|GDPR|compliant|audit log|CVE-\d{4}-\d+)\b", re.I
)
COMMITMENT = re.compile(
    r"\b(guarantee\w*|SLA|uptime|refund\w*|quota|deprecat\w*|pricing?|contract|deadline)\b",
    re.I,
)
QUANTITY = re.compile(
    r"\b\d+(?:\.\d+)?\s?(?:ms|s|sec|seconds|min|minutes|hours|days|%|MB|GB|TB|rps|USD)\b",
    re.I,
)


def slug(text: str) -> str:
    return re.sub(r"[^a-z0-9]+", "-", text.lower()).strip("-") or "root"


def blocks(text: str):
    buf, lang, out = [], None, []
    for raw in text.splitlines():
        fence = FENCE.match(raw.strip())
        if lang is None and fence:
            if buf:
                out.append(("\n".join(buf), None))
                buf = []
            lang = fence.group("lang").lower() or "text"
            buf.append(raw)
            continue
        if lang is not None:
            buf.append(raw)
            if raw.strip() == "

```":
                out.append(("\n".join(buf), lang))
                buf, lang = [], None
            continue
        if raw.strip() == "":
            if buf:
                out.append(("\n".join(buf), None))
                buf = []
        else:
            buf.append(raw)
    if buf:
        out.append(("\n".join(buf), lang))
    return out


def classify(body: str, lang):
    if lang is not None:
        return "A", "code_sample"
    head = body.lstrip().split("-->")[0]
    if body.lstrip().startswith("<!--") and "generated:" in head:
        return "C", "generated_marker"
    if SECURITY.search(body):
        return "A", "security_claim"
    if COMMITMENT.search(body) or QUANTITY.search(body):
        return "A", "commitment_number"
    return "B", "narrative"


def split(path: Path) -> dict:
    text = path.read_text(encoding="utf-8")
    section, counts, rows = "root", {"A": 0, "B": 0, "C": 0}, []
    for body, lang in blocks(text):
        lines = body.strip().splitlines()
        if lang is None and len(lines) == 1:
            m = HEADING.match(lines[0])
            if m:
                section = slug(m.group("text"))
                continue
        klass, reason = classify(body, lang)
        counts[klass] += 1
        rows.append(
            {
                "section": section,
                "class": klass,
                "reason": reason,
                "sha256": hashlib.sha256(body.encode()).hexdigest()[:12],
                "first_line": lines[0][:80] if lines else "",
            }
        )
    return {"file": str(path), "counts": counts, "blocks": rows}


def gate(manifest: dict, owners: dict) -> int:
    owned = {r["section"] for r in manifest["blocks"] if r["class"] == "A"}
    missing = sorted(owned - set(owners))
    for section in missing:
        print("UNOWNED scope A section: " + section)
    print("counts={} unowned={}".format(manifest["counts"], len(missing)))
    return 1 if missing else 0


def main(argv) -> int:
    if len(argv) == 2 and argv[0] == "split":
        print(json.dumps(split(Path(argv[1])), indent=2))
        return 0
    if len(argv) == 3 and argv[0] == "gate":
        manifest = json.loads(Path(argv[1]).read_text(encoding="utf-8"))
        owners = json.loads(Path(argv[2]).read_text(encoding="utf-8"))
        return gate(manifest, owners)
    print(__doc__)
    return 2


if __name__ == "__main__":
    raise SystemExit(main(sys.argv[1:]))
Enter fullscreen mode Exit fullscreen mode

Step 2: Read the manifest before you send anything

Run the splitter and print only the counters, which is the number you actually manage by.

python docs_scope.py split docs/retries.md > retries.scope.json
python -c "import json;m=json.load(open('retries.scope.json'));print(m['counts'])"
Enter fullscreen mode Exit fullscreen mode

The block list is the audit trail: each entry carries a section slug, a class, a reason code, and a short content hash. Reason codes are stable identifiers, so you can count them over time and watch which of your pages keeps producing commitment_number matches. Illustrative output from a small fixture page looks like this:

{
  "file": "docs/retries.md",
  "counts": {"A": 5, "B": 9, "C": 1},
  "blocks": [
    {"section": "retry-budget", "class": "A", "reason": "commitment_number", "sha256": "4f1c9a2e77b0", "first_line": "Each client retries up to 5 times within 30 s."},
    {"section": "why-bounded-retries", "class": "B", "reason": "narrative", "sha256": "8be02d51c4aa", "first_line": "Bounded retries keep tail latency predictable."}
  ]
}
Enter fullscreen mode Exit fullscreen mode

Those counters are illustrative fixture output, not a benchmark. The useful signal is the ratio: a page where scope A dominates is a page that should not be handed to a model at all.

Step 3: Gate scope A against owners.json

owners.json is a flat map from section slug to owner and evidence pointer. Keep it in the repository next to the docs, review it like code, and never populate it from a model.

{
  "retry-budget": {"owner": "@avery", "evidence": "docs/evidence/retry-budget-run.md"},
  "limits-and-quotas": {"owner": "@ops-lead", "evidence": "issue-482"}
}
Enter fullscreen mode Exit fullscreen mode

Then run the gate as a CI step and let the exit code do the talking.

python docs_scope.py gate retries.scope.json owners.json || exit 1
Enter fullscreen mode Exit fullscreen mode

An unowned scope A section prints its slug and returns status 1, which is the whole enforcement story. Note what the gate does not do: it does not verify that the evidence is good. It only guarantees that the question "who owns this commitment" has an answer before the page ships.

Step 4: Draft only scope B, and do it remotely

Scope B is the only part that should reach a model, and it arrives as isolated prose with no code fences and no numbers. Send one section per request with a fixed instruction: match the surrounding voice, keep sentences short, do not introduce examples that contain quantities.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode offers free model access and a free server option, according to the operator; treat both as availability statements rather than guarantees of capacity, model identity, or durability. For this workflow the fit is narrow and that is the point: the job is rewriting a handful of scope B sections, one request at a time, and the split has already removed the content that would make a shared remote box a bad idea.

The sequence that keeps the boundary honest:

  1. Run the splitter on the target page and commit the manifest alongside the draft branch.
  2. Export only scope B blocks, strip internal links and customer names, and send one section per request.
  3. Apply returned prose as a patch that touches only lines inside those sections, then re-run the splitter so the counts still match.
  4. Run the gate. If a model-added sentence introduced a quantity, the section flips to scope A and the gate asks for an owner, which is exactly the conversation you want to have.

Step 4 is why the manifest belongs in version control: a class change between two commits is a review event, not a formatting detail.

Fixture test plan

Keep four fixtures and assert on reason codes, not on exact hashes, so the tests survive rewording.

Fixture Contains Expected
commands.md one bash fence, one prose paragraph code_sample present, counts A at least 1
security.md "We encrypt data at rest with TLS" security_claim
quota.md "1000 rps per tenant" commitment_number
generated.md a <!-- generated: openapi --> marker generated_marker, counts C at least 1

A single shell assertion covers the first case and reads clearly in CI logs.

python docs_scope.py split tests/fixtures/commands.md \
  | python -c "import json,sys;m=json.load(sys.stdin);assert m['counts']['A']>=1, m['counts']"
Enter fullscreen mode Exit fullscreen mode

Limitations and who should not use this

The classifier is heuristic. Regexes over Markdown miss indented code blocks, fences nested inside lists, HTML blocks, tables that encode numbers in cells, and non-English commitment wording. If your documentation is regulated, the manifest is an input to legal review rather than a replacement for it.

If your docs are a single marketing page or a README that changes twice a year, the two extra commands cost more attention than they save. If your reference docs are fully generated from a spec, you already have scope C and mostly need scope A discipline. And if the free server is on a shared remote machine, do not send unredacted internal links, customer names, or incident details through it; the split reduces that exposure but does not eliminate it.

What to keep is small: a scope table, one stdlib script, an owners file reviewed by humans, and a gate that runs before merge. The model gets the prose it is good at, and people keep the sentences that can hurt them. If you run the splitter over your own handbook, the most useful thing to send back is your false-positive list, meaning the prose blocks that got flagged as commitments and why.

Top comments (0)