DEV Community

Avery Lin
Avery Lin

Posted on

Decide What a Model May Draft Before It Writes: A Section Ownership Gate for Docs

Decide What a Model May Draft Before It Writes: A Section Ownership Gate for Docs

A model-drafted page fails in one of two ways: it invents a fact, or it quietly overstates a promise your team cannot keep. The first failure is cheap to catch, because somebody can check a fact against a source file. The second one survives review for months, because the sentence sounds exactly like the ones your best technical writer produces.

So the useful question before drafting is not "is this paragraph good?" but "was a model allowed to write this paragraph at all?". That answer depends on consequences rather than on style: a changelog entry can be wrong and fixed on Monday, while a compatibility guarantee is a promise other people plan their quarter around.

The gate below encodes that decision in three places: a marker inside the document, a trailer on the commit, and a checker that fails a pull request when the two disagree. It is roughly eighty lines of Python plus one CI job, and it works with any drafting model you already route to.

1. Sort sections into three ownership tiers

Assign tiers by blast radius, not by how difficult the prose is to write. The table below is the decision rule I would apply to a typical product documentation set.

Tier Marker Typical sections What the model may do
Draftable owner=model changelog entries, CLI flag reference, config tables derived from schema write full prose from supplied context
Draftable with evidence owner=model evidence=src/... troubleshooting steps, migration snippets, error-message explanations write only claims traceable to the listed files or logs
Human-owned owner=human security posture, deprecation promises, pricing and limits, incident postmortems, cross-version guarantees propose a patch as a comment, never commit one

The second tier is where most real pages live, and it is the tier that models silently upgrade to the first. Asking for a citation path forces the drafting step to name its source, which makes a fabricated claim visible before a reviewer spends attention on tone.

The third tier is short on purpose. If you cannot name the person who owns a human-tier section, the section is not owned at all, and the marker is decoration.

2. Keep the marker in the file, not in a wiki

A boundary stored in a separate spreadsheet drifts away from the text within two sprints. Put it in the document as an HTML comment, directly above the heading it governs, so it never renders and never leaves the diff.

# Rate limits

<!-- section: rl-current owner=model evidence=src/limiter/limits.ts -->
## Current limits

The default is 60 requests per minute per API key. Bursts of 100 are accepted ...

<!-- section: rl-guarantees owner=human -->
## What we guarantee across versions

We do not raise the documented default within a major version. ...
Enter fullscreen mode Exit fullscreen mode

Three parsing rules keep this boring: a marker on its own line opens a section, a section closes at the next marker, and the last section runs to end of file. Markers never nest, which means a section ID is unique per file and easy to reference in a review comment.

Because the marker lives in version control, moving a paragraph across the boundary becomes an ordinary one-line diff. That is the property you want: the boundary change gets reviewed like any other change.

3. Gate the diff with a checker

The following is a reference implementation; run it against your own fixtures before you make it a required check. It performs three mechanical assertions: no model-mode commit adds lines inside a human-owned section, no model-mode commit deletes a human-owned marker, and every changed Markdown file is still parseable.

#!/usr/bin/env python3
"""Fail when a model-mode commit writes into a human-owned doc section."""
import re
import subprocess
import sys

MARKER = re.compile(r'<!--\s*section:\s*(?P<sid>[A-Za-z0-9_.-]+)\s+owner=(?P<owner>model|human)\b')
TRAILER = re.compile(r'^Draft-Mode:\s*model\s*$', re.MULTILINE)


def git(*args):
    return subprocess.run(('git',) + args, capture_output=True, text=True, check=True).stdout


def parse_sections(text):
    sections, current = [], None
    for number, line in enumerate(text.splitlines(), start=1):
        found = MARKER.search(line)
        if found is None:
            continue
        if current:
            current['last'] = number - 1
            sections.append(current)
        current = {'sid': found['sid'], 'owner': found['owner'], 'first': number}
    if current:
        current['last'] = len(text.splitlines())
        sections.append(current)
    return sections


def added_lines(sha, path):
    diff = git('show', '--unified=0', '--format=', sha, '--', path)
    numbers = set()
    for hunk in re.finditer(r'^@@ -\S+ \+(\d+)(?:,(\d+))? @@', diff, re.MULTILINE):
        first, count = int(hunk.group(1)), int(hunk.group(2) or 1)
        numbers.update(range(first, first + count))
    return numbers


def model_commits(base):
    for sha in git('rev-list', f'{base}..HEAD').split():
        if TRAILER.search(git('log', '-1', '--format=%B', sha)):
            yield sha


def main(base):
    failures = []
    for sha in model_commits(base):
        files = git('diff-tree', '--no-commit-id', '--name-only', '-r', sha).split()
        for path in (f for f in files if f.endswith('.md')):
            text = git('show', f'{sha}:{path}')
            sections = parse_sections(text)
            touched = added_lines(sha, path)
            for section in sections:
                span = set(range(section['first'], section['last'] + 1))
                if section['owner'] == 'human' and touched & span:
                    failures.append(f'{path}: {section["sid"]} written by model commit {sha[:9]}')
            if len(sections) < len(parse_sections(git('show', f'{sha}^:{path}'))):
                failures.append(f'{path}: model commit {sha[:9]} removed a section marker')
    for line in failures:
        print(line)
    return 1 if failures else 0


if __name__ == '__main__':
    sys.exit(main(sys.argv[1] if len(sys.argv) > 1 else 'origin/main'))
Enter fullscreen mode Exit fullscreen mode

The commit trailer is the only convention the checker depends on, and it is deliberately cheap to write:

docs(limits): refresh current defaults from limiter source

Draft-Mode: model
Evidence: src/limiter/limits.ts
Enter fullscreen mode Exit fullscreen mode

Wire it into CI as one job, and keep the job's failure message short enough to paste into a review thread:

name: docs-ownership
on: [pull_request]
jobs:
  gate:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with: { fetch-depth: 0 }
      - run: python3 scripts/check_ownership.py origin/${{ github.base_ref }}
Enter fullscreen mode Exit fullscreen mode

4. Make the model refuse instead of guessing

Filtering the input is stronger than instructing the model politely, because a policy it never sees cannot be argued with. Send only the draftable sections, and name the sections the model is not receiving.

You are editing docs/limits.md.
Editable sections: rl-current.
All other sections in this file are human-owned and are not included.

Rules:
1. Change only lines inside editable sections.
2. Do not add, remove, or rename section markers.
3. If the task requires a human-owned section, stop and reply exactly:
   REFUSED: <section-id>

Evidence files you may cite: src/limiter/limits.ts
Enter fullscreen mode Exit fullscreen mode

Route the REFUSED token into a queue rather than into a retry loop, and attach it to the owner of that section. A refusal is a successful outcome here: it converts a silent overreach into a work item with a name on it.

If the model produces a REFUSED line inside a draft body instead of as the whole reply, treat that as a failed run and discard the output. Mixed signal output is the one case where the gate and the prompt disagree.

5. Run the drafting pass where the budget is zero

The drafting step is the bursty part of this workflow: a release week needs thirty section refreshes, a quiet week needs none. MonkeyCode fits that shape because the project offers free model access and a free server option, which means a docs team can run both the drafting pass and the checker without a procurement ticket. Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The free server option matters for the gate as much as for the drafting, since a checker that only runs on a laptop is not a gate. The operator also cites a free token allowance for model usage; confirm the current number on the project page before you plan capacity around it, because free-tier terms are the most volatile part of any such offer.

Limitations worth stating plainly: on shared free infrastructure, check the data-handling terms before you send internal documentation, and never send incident postmortems or customer-specific content. Long evidence chains are also a weak spot for smaller models, so cap a draftable section at a few hundred words and one or two source files per run.

6. Failure modes and who should skip this

  • Spoofable trailer. Anyone can write Draft-Mode: model or omit it. Mitigate by requiring a trailer on any PR touching docs/**, with Draft-Mode: human as the explicit alternative.
  • The gate proves authorship, not truth. A model-written section that stays inside its tier can still be wrong; this gate is a boundary, not a correctness check.
  • Mixed pull requests. A PR with both model commits and human edits will pass only because the human commits are not inspected. Enforce one mode per PR, or accept that mixed PRs get less protection.
  • Small documentation sets. Under roughly fifteen pages, the marker overhead probably costs more than the errors it prevents.
  • Teams that already rewrite everything. If reviewers currently replace most model paragraphs, measure edit-after-merge first; you have a drafting-quality problem, not a boundary problem.
  • Legal, safety, and regulated text. Do not use model drafting for these at all, and do not mark them owner=model when nothing is being drafted.

7. Pilot in a week and watch four numbers

  1. Tag three pages: one mostly owner=model, one mixed, and one fully owner=human.
  2. Draft the model-tier sections in a single commit with the trailer, and open the PR as usual.
  3. Run the checker locally against origin/main and confirm it fails only when you deliberately touch a human section.
  4. Enable the CI job in report-only mode for three real pull requests, then make it required.
  5. Change one human-tier section in a model-mode commit on purpose, and confirm the failure message names the section rather than the file.

Track four numbers over the pilot: refusals per week, lines edited in model sections after merge, first-run pass rate of the CI job, and human-tier sections that owners downgrade to model tier. A rising refusal count with a falling edit count is the signal you want, because it means the boundary is absorbing work that used to reach review.

If you want to try the loop without a spend approval, start with three pages and one human-owned section, then let the refusals tell you where the boundary actually belongs.

Top comments (0)