Most documentation reviews spend their attention on sentences that are wrong. For model-drafted pages, the expensive failure is usually a sentence that is gone: a must not, a deprecation notice, a prerequisite that quietly disappeared during a rewrite. The gate below makes removal of a constraint an explicit, signed act, so a model can keep drafting connective prose while a human keeps owning every restriction that leaves the page.
1. The failure mode is loss, not invention
A model asked to "improve this page" reads the request as a writing task, and writing tasks reward concision. Constraint sentences are the least concise part of any page, because a good page repeats its warnings in the overview, inside the procedure, and again in troubleshooting. One pass with a shortening instruction will collapse those repetitions into a single mention, and a reviewer reading the diff top to bottom sees a cleaner page rather than a weaker one.
Source-checking does not catch this. A checker that confirms "every claim in the draft has a source" passes happily when the draft contains fewer claims than the page it replaced, because it only looks forward. You need the opposite direction: start from the old page, enumerate the constraints it carried, and refuse to let any of them vanish without a named human saying they should. That check is mechanical, cheap, and independent of which model produced the draft.
2. Split draft rights by sentence type, not by section
Section-level ownership is easy to describe and hard to enforce, because a rewrite crosses section boundaries and the interesting sentences are scattered. Sentence classification is enforceable, since a modal verb and a fingerprint are things a script can actually see. The split below is the one the gate encodes; treat it as a starting policy and tune the right column to your product.
| Element on the page | Model may | Human must |
|---|---|---|
| Connective prose between steps | draft freely | approve tone and accuracy |
| Numbered procedure steps | propose, with a change summary | own ordering and prerequisites |
Constraint sentences (must, never, deprecated) |
copy verbatim or flag | own every edit, deletion, or merge |
| Version numbers, limits, dates, prices | insert TODO(verify)
|
verify against a primary source |
| Error strings and their fixes | extract from code | own the mapping to a cause |
| Security and data-loss warnings | leave untouched | own absolutely |
The practical consequence is that a drafting prompt should say what the model may touch. "Rewrite the introduction and expand the examples; copy every sentence containing must, never, or deprecated verbatim" is a prompt a human can actually audit later, because the output can be checked against the input.
3. Extract constraint sentences with a marker list
The script below uses only the Python standard library, which matters if you want it to run anywhere a draft is produced. It splits the page into sentences, keeps the ones matching a modal-marker regex, and compares the old set against the draft set. A constraint is present when the normalized sentence survives exactly, reworded when a close match exists, and missing when nothing in the draft is similar enough.
#!/usr/bin/env python3
"""normative_gate.py - block silent removal of constraints from model-drafted docs.
Stdlib only. Usage:
python3 normative_gate.py docs/before/retries.md docs/after/retries.md \
--notes docs/removal-notes.json --packet review/retries.md
"""
from __future__ import annotations
import argparse
import hashlib
import json
import re
import sys
from difflib import SequenceMatcher
from pathlib import Path
MODALS = re.compile(
r"\b(must|must not|never|required|requires|shall|should|should not|"
r"do not|deprecated|unsupported|breaking|warning|caution)\b",
re.IGNORECASE,
)
SPLIT = re.compile(r"(?<=[.!?:])\s+(?=[A-Z0-9`*_])")
STRIP = re.compile(r"[^a-z0-9`]+")
def sentences(text: str) -> list[str]:
out: list[str] = []
for block in re.split(r"\n\s*\n", text):
block = re.sub(r"\s+", " ", block).strip()
if block:
out.extend(s for s in SPLIT.split(block) if s.strip())
return out
def normalize(sentence: str) -> str:
return re.sub(r"\s+", " ", STRIP.sub(" ", sentence.lower())).strip()
def fingerprint(sentence: str) -> str:
return hashlib.sha256(normalize(sentence).encode()).hexdigest()[:12]
def constraints(text: str) -> list[str]:
return [s for s in sentences(text) if MODALS.search(s)]
def classify(old: list[str], new: list[str], threshold: float) -> dict:
pool = [(normalize(n), n) for n in new]
buckets: dict[str, list[tuple[str, str, float]]] = {
"present": [], "reworded": [], "missing": []
}
for sentence in old:
target = normalize(sentence)
best, best_text = 0.0, ""
for cand_norm, cand in pool:
if cand_norm == target:
best, best_text = 1.0, cand
break
score = SequenceMatcher(None, target, cand_norm).ratio()
if score > best:
best, best_text = score, cand
name = "present" if best >= 0.999 else "reworded" if best >= threshold else "missing"
buckets[name].append((sentence, best_text, round(best, 3)))
return buckets
def load_notes(path: Path) -> dict:
if not path.exists():
return {}
data = json.loads(path.read_text(encoding="utf-8"))
if isinstance(data, list):
return {entry["fingerprint"]: entry for entry in data}
return data
def main() -> int:
parser = argparse.ArgumentParser()
parser.add_argument("before")
parser.add_argument("after")
parser.add_argument("--notes", default="docs/removal-notes.json")
parser.add_argument("--packet", default="")
parser.add_argument("--threshold", type=float, default=0.75)
args = parser.parse_args()
before = Path(args.before).read_text(encoding="utf-8")
after = Path(args.after).read_text(encoding="utf-8")
result = classify(constraints(before), constraints(after), args.threshold)
notes = load_notes(Path(args.notes))
lines = ["# Review packet", "", f"- before: `{args.before}`", f"- after: `{args.after}`", ""]
unaccounted: list[str] = []
used: set[str] = set()
for bucket in ("missing", "reworded"):
for sentence, closest, score in result[bucket]:
fp = fingerprint(sentence)
used.add(fp)
note = notes.get(fp)
state = "accounted" if note and note.get("owner") else "UNACCOUNTED"
lines += [f"## {state} - {bucket} (score {score})", "", f"> {sentence}", ""]
if closest:
lines += [f"Closest draft sentence: {closest}", ""]
if note:
lines += [f"- owner: {note.get('owner')}", f"- reason: {note.get('reason')}"]
else:
unaccounted.append(fp)
lines += [f"- proposed fingerprint: {fp}", "- owner: ", "- reason: "]
lines.append("")
stale = sorted(set(notes) - used)
for fp in stale:
lines += [f"## STALE note - {fp}", "", "The sentence is back on the page; drop this note.", ""]
packet = "\n".join(lines)
if args.packet:
Path(args.packet).parent.mkdir(parents=True, exist_ok=True)
Path(args.packet).write_text(packet, encoding="utf-8")
print(packet)
print(
f"present={len(result['present'])} reworded={len(result['reworded'])} "
f"missing={len(result['missing'])} unaccounted={len(unaccounted)} stale={len(stale)}"
)
return 1 if unaccounted else 0
if __name__ == "__main__":
sys.exit(main())
The similarity threshold is the only tunable number, and 0.75 is a deliberately loose default: it classifies a reworded constraint as reworded rather than missing, so a human sees the old and new sentences side by side instead of a single opaque deletion.
4. Removal notes are the human signature
A missing or reworded constraint is not automatically an error, because sometimes a restriction really is obsolete. The point is that the decision must be attributable. Each accepted change gets an entry keyed to the fingerprint of the original sentence:
{
"6b1f0c9a2d34": {
"owner": "avery",
"reason": "Retry limit removed in v3; see CHANGELOG entry 2026-09.",
"date": "2026-10-19"
}
}
Two details make this durable rather than ceremonial. First, the key is derived from the sentence text, so a note cannot drift onto a different constraint; if the sentence is edited, the fingerprint changes and the note becomes STALE, which the packet surfaces instead of silently ignoring. Second, the gate exits non-zero whenever any account is missing, so a CI job can gate publication on the same signal a reviewer sees.
5. Wire it into the drafting loop
- Freeze the published page before any generation:
git show HEAD:docs/retries.md > docs/before/retries.md. - Draft the new page with a prompt that names the protected sentences and forbids edits to them.
- Save the draft as
docs/after/retries.mdand run the gate. - Read the packet, not the whole page: restore what should stay, or write a note with an owner and a reason.
- Re-run until
unaccounted=0, then review the prose normally. - Commit the notes file with the page, so the reasoning ships alongside the text.
The drafting step is the part that benefits from a generous free tier, since you will regenerate drafts several times while tuning the prompt. Running this loop on MonkeyCode's free model access for the draft, and the gate on the free server option, keeps the whole cycle in one environment rather than shuttling files between a paid runtime and a laptop. Disclosure: This article was prepared as part of MonkeyCode's product outreach. Those two availability claims come from the operator, not from a benchmark I ran; confirm current terms before you build a process that depends on them.
6. Test the gate before trusting it
A gate that always passes is worse than no gate, so validate it against fixtures with known answers:
| Case | Fixture change | Expected exit | Expected packet |
|---|---|---|---|
| 1 | introduction rewritten, no constraint touched | 0 | empty |
| 2 | one must not sentence deleted |
1 | one UNACCOUNTED, missing
|
| 3 |
Must not retry writes reworded to Do not retry writes
|
1 | one UNACCOUNTED, reworded
|
| 4 | case 2 plus a note with owner and reason | 0 | one accounted |
| 5 | note added, then the original sentence restored | 0 | one STALE note |
for c in 1 2 3 4 5; do
python3 normative_gate.py fixtures/c$c.before.md fixtures/c$c.after.md \
--notes fixtures/c$c.notes.json > /dev/null
echo "case $c exit=$?"
done
If case 2 passes, your marker list is too narrow. If case 5 stays silent, stale notes will accumulate and quietly pre-approve future deletions, which is the failure this gate exists to prevent.
7. Limitations and who should skip it
This gate checks presence, not truth: a constraint copied verbatim into a draft that now contradicts the code passes, and only a human comparing the page to the implementation will catch that. Sentence splitting on Markdown is approximate, since tables, nested lists, and code blocks all break naive splitters, and the marker list will produce false positives on ordinary uses of words like required or should. The fingerprints are stable only while normalization rules stay fixed, so changing the regex invalidates existing notes and should be done in a single commit with regenerated notes.
Skip this approach if your docs are generated entirely from code and never hand-edited, because there is no prior human constraint set to protect. Skip it too if your team cannot commit to writing a one-line reason for each removal, since an allowlist that everyone edits casually becomes a rubber stamp. It fits best where a small number of pages carry real operational risk and a model is expected to keep rewriting them.
Start with the two fixtures that matter most, cases 2 and 3, on the single most dangerous page you own. If the gate fires there, you have found a deletion you would otherwise have shipped; if you run it on the free server option, that first run costs you nothing but the time to write the markers down.
Top comments (0)