DEV Community

Avery Lin
Avery Lin

Posted on

Permit ASSERT and INSTRUCT; Keep COMMIT and EVALUATE Human

Generated documentation stays reviewable when every sentence is classified by speech act before a model drafts it. Models may restate proven facts, and they may emit instructions that a recorded command already runs successfully. Humans must still own warnings, product commitments, and evaluative claims that no fixture can prove. This article implements that split as a permit table, a tagged Markdown convention, and a small Python gate.

Why a citation check still lets promises through

A schema pointer can prove that a field exists, yet it cannot prove that support will continue. A passing test can justify an instruction, yet it cannot justify a compatibility promise to callers. Citation gates therefore let COMMIT and EVALUATE sentences through whenever those sentences happen to mention a file. Speech-act permits close that gap by asking what the sentence is doing, not only which path it names.

This distinction shows up in release notes, getting-started guides, and high-level API overviews that mix tones. Those pages combine restatements of code with advice, risk language, and explicit support boundaries for readers. A generator that treats the whole page as one blob will invent surrounding tone around otherwise real facts. Reviewers then spend review time deleting commitments the model never had authority to make in the first place.

The working rule is narrow and mechanical rather than stylistic. If the sentence reports a present fact, it may be generated after evidence is attached. If the sentence tells the reader to run a command, a fixture must already exit zero in CI. If the sentence warns, promises, or ranks options, a human writes it in a separate tree.

Five speech acts and what each one is allowed to do

Treat every documentation sentence as exactly one of the following speech acts, never as a blend. Mixed sentences fail the gate until an editor splits them into separate lines with separate tags.

  1. ASSERT records a present-tense claim about code, schema, or observed output that a pointer can locate.
  2. INSTRUCT gives an imperative step a reader can run, bound to a fixture script and an expected exit code.
  3. WARN states a risk, failure mode, or security implication that still needs a human operator to accept.
  4. COMMIT promises support, compatibility, an SLA boundary, or future behavior that policy files must own.
  5. EVALUATE recommends, ranks, or says "should" when no test forces that ranking on the reader.

ASSERT is allowed only when a source pointer resolves inside a documented tree and a verbatim span matches. INSTRUCT is allowed only when the command identifier maps to a script the repository actually executes in automation. WARN, COMMIT, and EVALUATE remain human-owned even when a model could guess them from nearby facts. The permit file, not the prompt, is the policy the merge gate enforces.

Decision table the repository can version

The table below is the artifact this workflow enforces. Store it at docs/_permits/speech-acts.yml so policy changes travel through ordinary review.

# docs/_permits/speech-acts.yml
# Proposed policy file. Unexecuted until you commit it and run the gate.
version: 1
acts:
  ASSERT:
    drafter: model
    requires: [source_pointer, verbatim_span]
    human_review: spot
  INSTRUCT:
    drafter: model
    requires: [command_id, fixture_path, expected_exit]
    human_review: spot
  WARN:
    drafter: human
    requires: [owner, ops_note]
    human_review: required
  COMMIT:
    drafter: human
    requires: [owner, policy_file]
    human_review: required
  EVALUATE:
    drafter: human
    requires: [owner, rationale]
    human_review: required
forbidden_in_generated:
  - WARN
  - COMMIT
  - EVALUATE
split_on_mixed_sentence: true
lexical_hints:
  COMMIT: ["we will", "we promise", "supported until", "SLA", "guaranteed"]
  EVALUATE: ["recommended", "prefer", "best practice", "you should"]
  WARN: ["do not", "dangerous", "security", "data loss"]
Enter fullscreen mode Exit fullscreen mode

The table stays small so a pull request can tighten policy without rewriting parser code. Generated files may contain ASSERT and INSTRUCT only, each with the keys listed above. Human files may contain any act, but COMMIT still needs a policy-file pointer rather than a schema fragment.

Tagged Markdown the assembler can refuse

Each sentence in a generated fragment starts with an HTML comment the renderer ignores. The comment encodes the act, a stable id, and the evidence the permit requires for that act.

<!-- act:ASSERT id:billing.currency pointer:openapi.yaml#/components/schemas/Charge/properties/currency span:currency -->
The Charge object includes a `currency` field.

<!-- act:INSTRUCT id:billing.curl.charge command:create-charge fixture:fixtures/commands/create-charge.sh exit:0 -->
Run `fixtures/commands/create-charge.sh` against the local sandbox to create a charge.
Enter fullscreen mode Exit fullscreen mode

Human-owned pages use the same comment shape so leakage is a parse error instead of a taste debate. A COMMIT sentence must point at a policy document, not at an OpenAPI path that merely names a field.

<!-- act:COMMIT id:billing.currency.support owner:api-steward policy:docs/policy/supported-currencies.md -->
Sandbox and production accept only the currency codes listed in the supported-currencies policy.
Enter fullscreen mode Exit fullscreen mode

Keep generated fragments under docs/generated/ and human fragments under docs/human/. An order file lists how those fragments concatenate into a reader-facing page. The assembler must not rewrite sentence text; it only concatenates and then re-runs the gate.

Numbered workflow for one documentation surface

Run the following sequence on a single overview or getting-started page before you expand coverage. Stopping after one surface keeps the permit table honest, because failures stay local and reviewable.

Step 1: Extract facts and commands, and do not extract prose

Collect schema titles, enum values, and executable scripts into docs/_facts/. Do not ask a model to summarize the product during this extraction step. Store each fact as JSON with a pointer, a verbatim span, and the only speech act that fact can support.

{
  "id": "billing.currency",
  "pointer": "openapi.yaml#/components/schemas/Charge/properties/currency",
  "span": "currency",
  "speech_act": "ASSERT"
}
Enter fullscreen mode Exit fullscreen mode

Command fixtures live beside the facts and must be runnable without extra narrative. An instruction without a fixture identifier is not eligible for generation, even if the command looks obvious to a reviewer.

# Proposed local extraction. Label outputs as unverified until CI runs them.
python3 tools/extract_facts.py --schema openapi.yaml --out docs/_facts/assert.jsonl
python3 tools/index_fixtures.py --dir fixtures/commands --out docs/_facts/instruct.jsonl
Enter fullscreen mode Exit fullscreen mode

Step 2: Draft only the acts the permit table allows

Send the facts file to a drafting model with a hard constraint that output may contain ASSERT and INSTRUCT sentences only. Reject any response that includes commitment or ranking language unless those tokens already appear inside a verbatim span. The model restates spans and turns fixture commands into numbered steps. It does not invent recovery advice, support windows, or comparative claims.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode's free model access fits this drafting step because the prompt is narrow and the facts file is already extracted. MonkeyCode's free server option can run the permit gate as a remote check so a laptop is not the only place the classifier executes. Neither option writes WARN, COMMIT, or EVALUATE pages, and this article does not claim quotas, model names, or hardware details.

A drafting prompt should restate the permit table rather than appeal to general helpfulness. Keep the output as tagged Markdown, one sentence after each comment, with no wrapping paragraphs that hide a second speech act.

You may emit ASSERT and INSTRUCT sentences only.
Each sentence must follow a speech-act HTML comment with required keys.
Do not emit WARN, COMMIT, or EVALUATE.
Do not use recommended, guaranteed, SLA, we will, or you should.
If a fact cannot support ASSERT or INSTRUCT, skip it.
Enter fullscreen mode Exit fullscreen mode

Step 3: Run the speech-act gate before assembly

The gate parses HTML comments, validates required keys, and fails the build on mixed or forbidden acts. The script in the next section is proposed sample code. Treat it as unexecuted until you run it against your own tree and keep the failing fixtures.

python3 tools/speech_act_gate.py \
  --permit docs/_permits/speech-acts.yml \
  --generated-glob 'docs/generated/**/*.md' \
  --human-glob 'docs/human/**/*.md'
Enter fullscreen mode Exit fullscreen mode

Step 4: Assemble pages without blending ownership

Generated ASSERT and INSTRUCT fragments land only under docs/generated/. Human WARN, COMMIT, and EVALUATE fragments land only under docs/human/. A thin assembler reads docs/_permits/order.yml and concatenates in that declared order. If a forbidden act appears inside the generated tree, the assembler exits non-zero and prints the sentence id.

# docs/_permits/order.yml
page: docs/out/charges.md
parts:
  - docs/generated/charges.assert.md
  - docs/generated/charges.instruct.md
  - docs/human/charges.warn.md
  - docs/human/charges.commit.md
Enter fullscreen mode Exit fullscreen mode

Step 5: Review by act instead of by page length

Spot-check each ASSERT against the pointer span by searching the source file for that exact token. Re-run each INSTRUCT fixture and record the exit code next to the command id. Read every WARN, COMMIT, and EVALUATE line as contract text, because those lines are the product promise surface. This review is shorter than editing a fully generated overview, because the model never drafted the contract language.

# Re-run instruction fixtures as ordinary tests, not as documentation theater.
find fixtures/commands -name '*.sh' -print0 | xargs -0 -n1 /bin/sh
Enter fullscreen mode Exit fullscreen mode

Proposed gate implementation

The following Python module is a local linter, not a hosted service. It does not classify untagged prose with a model. It only enforces tags that writers and generators already attached.

# tools/speech_act_gate.py
# Proposed sample. Run tests before using it as a merge requirement.
from __future__ import annotations

import pathlib
import re
import sys
from typing import Dict, List

import yaml

COMMENT = re.compile(
    r"<!--\s*act:(?P<act>[A-Z]+)\s+(?P<body>.*?)\s*-->\s*(?P<text>.*)",
    re.DOTALL,
)
KEY = re.compile(r"(\w+):(\S+)")


def load_permit(path: pathlib.Path) -> dict:
    data = yaml.safe_load(path.read_text(encoding="utf-8"))
    if not data or "acts" not in data:
        raise SystemExit("permit file missing acts")
    return data


def parse_file(path: pathlib.Path) -> List[dict]:
    rows: List[dict] = []
    for raw in path.read_text(encoding="utf-8").splitlines():
        line = raw.strip()
        if not line:
            continue
        match = COMMENT.match(line)
        if not match:
            raise SystemExit(f"{path}: untagged sentence: {line[:80]}")
        fields: Dict[str, str] = dict(KEY.findall(match.group("body")))
        fields["act"] = match.group("act")
        fields["text"] = match.group("text").strip()
        fields["path"] = str(path)
        rows.append(fields)
    return rows


def validate(row: dict, permit: dict, generated: bool) -> List[str]:
    errors: List[str] = []
    act = row["act"]
    spec = permit["acts"].get(act)
    if spec is None:
        return [f"{row['path']}: unknown act {act}"]
    if generated and act in permit.get("forbidden_in_generated", []):
        errors.append(f"{row['id']}: {act} is forbidden in generated docs")
    for key in spec.get("requires", []):
        alias = {
            "source_pointer": "pointer",
            "verbatim_span": "span",
            "command_id": "command",
            "fixture_path": "fixture",
            "expected_exit": "exit",
            "ops_note": "note",
            "policy_file": "policy",
        }.get(key, key)
        if alias not in row and key not in row:
            errors.append(f"{row.get('id', '?')}: missing {key}")
    if act == "ASSERT" and row.get("span") and row["span"] not in row.get("text", ""):
        errors.append(f"{row['id']}: ASSERT text must contain span {row['span']}")
    hints = permit.get("lexical_hints", {})
    for other_act, tokens in hints.items():
        if other_act == act:
            continue
        lowered = row.get("text", "").lower()
        for token in tokens:
            if token.lower() in lowered:
                errors.append(
                    f"{row.get('id', '?')}: token {token!r} belongs to {other_act}"
                )
    return errors


def main(argv: List[str]) -> int:
    permit_path = pathlib.Path(argv[argv.index("--permit") + 1])
    permit = load_permit(permit_path)
    generated = []
    human = []
    # glob expansion left to the caller for portability
    for i, flag in enumerate(argv):
        if flag == "--generated" and i + 1 < len(argv):
            generated.append(pathlib.Path(argv[i + 1]))
        if flag == "--human" and i + 1 < len(argv):
            human.append(pathlib.Path(argv[i + 1]))
    errors: List[str] = []
    for path in generated:
        for row in parse_file(path):
            errors.extend(validate(row, permit, generated=True))
    for path in human:
        for row in parse_file(path):
            errors.extend(validate(row, permit, generated=False))
    for err in errors:
        print(err, file=sys.stderr)
    return 1 if errors else 0


if __name__ == "__main__":
    raise SystemExit(main(sys.argv[1:]))
Enter fullscreen mode Exit fullscreen mode

Pair the gate with a failing unit test so the COMMIT leak is a regression, not a style comment. The test below is also proposed sample code and should be executed before you trust the exit code in CI.

# tools/test_speech_act_gate.py
from pathlib import Path

from speech_act_gate import load_permit, parse_file, validate


def test_commit_in_generated_tree_fails(tmp_path: Path) -> None:
    permit = load_permit(Path("docs/_permits/speech-acts.yml"))
    sample = tmp_path / "leaked.md"
    sample.write_text(
        "<!-- act:COMMIT id:x owner:api policy:docs/policy/x.md --> "
        "We will support this field forever.\n",
        encoding="utf-8",
    )
    row = parse_file(sample)[0]
    errors = validate(row, permit, generated=True)
    assert any("forbidden" in item for item in errors)


def test_assert_requires_span_in_sentence(tmp_path: Path) -> None:
    permit = load_permit(Path("docs/_permits/speech-acts.yml"))
    sample = tmp_path / "assert.md"
    sample.write_text(
        "<!-- act:ASSERT id:y pointer:openapi.yaml#/x span:currency --> "
        "The object includes an amount field.\n",
        encoding="utf-8",
    )
    row = parse_file(sample)[0]
    errors = validate(row, permit, generated=True)
    assert any("must contain span" in item for item in errors)
Enter fullscreen mode Exit fullscreen mode
python3 -m pytest tools/test_speech_act_gate.py -q
Enter fullscreen mode Exit fullscreen mode

What this workflow does not prove

The gate does not prove that a pointer is semantically true, only that required keys exist and that ASSERT text still contains the span. It does not classify untagged paragraphs, so authors cannot paste a model essay and hope the linter infers speech acts. HTML comments can be stripped by aggressive Markdown pipelines, which means the rendered site is not the source of truth. Lexical hints are English-centric and will miss commitments written in other phrasing. Mixed sentences still need a human split; the parser refuses them rather than repairing them.

Free model access does not change those limits. A remote free server running the same script still only checks tags, required keys, and forbidden acts. If your facts file is wrong, permitted ASSERT sentences will be wrong in a well-tagged way. If your policy files are missing, COMMIT sentences should fail closed instead of being drafted.

Who should not use this approach

Skip this workflow if the page is primarily evaluative, such as a comparison essay or a thought-leadership post with no schema underneath. Skip it if you lack executable fixtures, because INSTRUCT then becomes theater and should stay human-written. Skip it if legal or compliance review must own every public sentence through a separate process this gate cannot see. Skip it if you need the model to invent a tutorial voice that includes judgment, recovery, and reassurance in the same paragraph.

Teams that already extract facts before drafting can apply the permit table to one overview page and keep COMMIT sentences in the human tree. After that page stays stable through a release, widen the same acts to a second surface without relaxing forbidden generated acts.

Top comments (0)