Block Modal Verbs in Model-Drafted Docs With a Claim-Class Gate

Model-drafted documentation fails when it quietly converts a function signature into a support promise. The practical control is not a better prompt; it is a claim-class gate that rejects modal language before merge. Constative restatements of public symbols may be generated from extractable code, tests, and type annotations only. Commissive, directive, and evaluative sentences stay in human-owned files that the generator cannot overwrite.

The leak is grammatical, not missing context

Most documentation pipelines already parse OpenAPI files, docstrings, and tests into an intermediate representation of symbols. The remaining failure mode is grammatical: generated prose introduces obligation, duration, and advice that no artifact stated. Words such as must, should, will support, and recommended turn a restatement into a contract the repository cannot prove. A review culture that only skims tone will miss those conversions because the surrounding sentences still look accurate.

Treat each sentence as a typed record before it is allowed to land in docs/generated/. The type is not a style label for writers; it is a merge constraint enforced by tests. If a sentence cannot be classified, the default is human ownership rather than model authorship. That default is the entire method, and prompts are only a convenience around it.

Four claim classes, one writable path

Define four classes before any model is allowed to write a Markdown paragraph for a public API.

1. CONSTATIVE — indicative description of a symbol that a reader can recover from code or tests.
2. COMMISSIVE — promises about support windows, compatibility, latency, or incident response.
3. DIRECTIVE — instructions that tell operators what they must do in production environments.
4. EVALUATIVE — recommendations, rankings, and best-practice language that encode product judgment.

Only CONSTATIVE sentences and executable examples belong on the model-writable documentation path in this workflow. The other three classes require a named human owner, a review record, and a file the generator cannot replace. Mixing the classes in one file recreates the original leak, because reviewers cannot see which sentences were generated.

A compact decision table keeps the split mechanical during review.

Surface	Allowed author	Evidence required	Forbidden tokens
docs/generated/*.md	model, then linter	public symbol, type, or test name	must, should, will support, SLA, recommend
examples/*.py	model, then pytest	import of public API plus assertion	network calls, sleep, live credentials
docs/promises.md	human only	owner, date, review link	none; this file is the promise surface
docs/runbooks/*.md	human only	incident owner and environment	generated boilerplate copied from chat

A reproducible gate, not a style guide

The artifact below is a small classifier plus tests. It is intentionally boring: regular expressions over sentence splits, a denylist, and a file-path policy. Label the patterns as heuristics, not linguistics research, and keep the denylist in version control beside the docs.

# claim_gate.py
from __future__ import annotations

import re
from dataclasses import dataclass
from pathlib import Path

SENTENCE_SPLIT = re.compile(r"(?<=[.!?])\s+")

COMMISSIVE = re.compile(
    r"\b(will support|supported until|sla|uptime|guarantee|we promise|compat(?:ible|ibility) window)\b",
    re.I,
)
DIRECTIVE = re.compile(
    r"\b(must|required to|do not|never run|always set|operators shall)\b",
    re.I,
)
EVALUATIVE = re.compile(
    r"\b(should|recommend(?:ed)?|best practice|prefer(?:ably)?|ideal(?:ly)?)\b",
    re.I,
)
CONSTATIVE_HINT = re.compile(
    r"\b(returns|raises|accepts|defaults to|is defined as|signature|example)\b",
    re.I,
)

GENERATED_ROOT = Path("docs/generated")
HUMAN_ONLY = {Path("docs/promises.md")}


@dataclass(frozen=True)
class Finding:
    path: str
    sentence: str
    claim_class: str


def sentences_of(text: str) -> list[str]:
    parts = SENTENCE_SPLIT.split(text.strip())
    return [p.strip() for p in parts if p.strip()]


def classify(sentence: str) -> str:
    if COMMISSIVE.search(sentence):
        return "COMMISSIVE"
    if DIRECTIVE.search(sentence):
        return "DIRECTIVE"
    if EVALUATIVE.search(sentence):
        return "EVALUATIVE"
    if CONSTATIVE_HINT.search(sentence):
        return "CONSTATIVE"
    return "UNKNOWN"


def scan_generated(root: Path = GENERATED_ROOT) -> list[Finding]:
    findings: list[Finding] = []
    if not root.exists():
        return findings
    for path in sorted(root.rglob("*.md")):
        for sentence in sentences_of(path.read_text(encoding="utf-8")):
            claim = classify(sentence)
            if claim != "CONSTATIVE":
                findings.append(Finding(str(path), sentence, claim))
    return findings


def assert_human_only_not_generated() -> None:
    for path in HUMAN_ONLY:
        if path.exists() and GENERATED_ROOT in path.parents:
            raise SystemExit(f"human-owned file nested under generated root: {path}")

Pair the classifier with tests that encode the policy in fixtures, not in reviewer memory. The tests below should fail closed: unknown sentences are rejected on the generated path.

# test_claim_gate.py
from pathlib import Path

from claim_gate import classify, scan_generated


def test_restatement_is_constative() -> None:
    sentence = "`checkout` returns a PaymentResult and raises TimeoutError."
    assert classify(sentence) == "CONSTATIVE"


def test_support_window_is_commissive() -> None:
    sentence = "We will support API v1 until 2028."
    assert classify(sentence) == "COMMISSIVE"


def test_generated_tree_rejects_modals(tmp_path, monkeypatch) -> None:
    generated = tmp_path / "docs" / "generated"
    generated.mkdir(parents=True)
    (generated / "checkout.md").write_text(
        "`checkout` returns a PaymentResult.\nYou should retry on timeout.\n",
        encoding="utf-8",
    )
    import claim_gate as cg

    monkeypatch.setattr(cg, "GENERATED_ROOT", generated)
    findings = scan_generated(generated)
    classes = {item.claim_class for item in findings}
    assert "EVALUATIVE" in classes

Run the gate locally with the same commands continuous integration will use. Keep the commands short so the policy is visible in logs.

python -m pytest test_claim_gate.py -q
python -c "from claim_gate import scan_generated; f=scan_generated(); raise SystemExit(1 if f else 0)"

Numbered workflow for a single API page

Use the following sequence when a public function changes and documentation must follow that change. The sequence is a compile pipeline, not a chat transcript stored beside the repository.

1. Freeze the writable surfaces. List public symbols from ast or your language equivalent, and list tests that import those symbols. Do not pass issue tracker commentary into the draft context at this stage.
2. Draft only CONSTATIVE pages and examples. Ask the model for indicative restatements and for an example program that imports the public API. Refuse output that contains the denylisted tokens even if the surrounding description is otherwise correct.
3. Compile examples. Run the example file under pytest or the language test runner with network disabled. An example that cannot execute is not documentation in this workflow.
4. Classify every generated sentence. Fail the build on COMMISSIVE, DIRECTIVE, EVALUATIVE, or UNKNOWN classes under docs/generated/.
5. Handwrite promises in a separate file. Support windows, upgrade pressure, and production directives go in docs/promises.md with an owner field. The generator must not open that path for write.
6. Diff the two trees in review. Reviewers check that generated files cite symbols, and that promise files cite owners. Do not review them as one narrative blob.

A minimal example file shows the intended model output: code that a reader can run, not advice about how long you will maintain it.

# examples/test_checkout_example.py
from payments import PaymentResult, TimeoutError, checkout


def test_checkout_example_restates_types() -> None:
    result = checkout(amount_cents=199, currency="USD")
    assert isinstance(result, PaymentResult)

If checkout cannot be constructed without a network, the human-owned runbook should say so. The generated page should only restate the signature, the raised errors, and the example name. That split looks incomplete on first reading, which is a feature: incompleteness is cheaper than a false guarantee.

Where a free model and a free server actually participate

Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode's free model access is useful for drafting CONSTATIVE restatements and example programs after the symbol list exists. MonkeyCode's free server option is useful for running the classifier, pytest, and the exit-code gate without mixing that run into a laptop that also holds production credentials. Neither substitution changes the policy: the model still may not write modal verbs, and the server still may not publish docs/promises.md.

Keep product choice secondary to the file split. A local pytest run of claim_gate.py is enough to adopt the method. If a shared runner helps your team keep the denylist honest, a free server can host that runner; it cannot own a support window.

Limitations, failure modes, and who should skip this

The classifier is a denylist, not a parser of meaning, and it will both over-block and under-block. Over-blocking happens when a CONSTATIVE sentence uses must inside a quoted error string from the compiler. Under-blocking happens when a model writes "the endpoint remains available for existing customers" without any denylisted token. UNKNOWN rejection reduces the second case at the cost of more human edits.

Do not use this workflow for legal terms, pricing pages, security advisories, or anything that requires counsel. Do not use it when the repository has no stable public surface, because the CONSTATIVE class then has nothing recoverable to cite. Teams that publish only narrative tutorials will find the gate hostile, and they should keep a human editor instead of a sentence classifier. The method also assumes English modal verbs; other languages need a separate denylist and tests.

The approach will annoy writers who want a single flowing page. That annoyance is the point of the split. A generated paragraph that reads smoothly while smuggling should into an API reference is the defect this gate exists to catch. If your process already requires a named owner for every non-restatement, you do not need a new tool, only a test that encodes the rule you already believe.

What the model may draft, and what a human must own

The model may draft indicative restatements of public symbols, type names, default values, and raised errors. The model may draft example programs that import those symbols and fail in continuous integration when the symbols move. The human must own support duration, compatibility promises, production directives, incident posture, and any sentence that tells a customer what the vendor will do next. If a sentence still feels like a promise after the denylist is applied, move it to docs/promises.md and leave the generated page slightly shorter.

A documentation set that looks unfinished at the judgment line is safer than a complete page nobody can defend. Run the classifier on every generated file, keep modal verbs out of that tree, and treat unknown sentences as human work. The rest of the toolchain, including any free model or free server you use for drafts and checks, is optional machinery around that ownership line.