Draft the Holes, Not the Facts: A Slot Protocol for Agent-Written Docs

Suppose a documentation PR arrives with every sentence already written. The prose is clean, the headings are right, and nothing in the diff looks like a placeholder. Now try to review it.

You cannot tell which sentences came from the repository and which came from the model's sense of what usually goes in a paragraph like that. Reviewers in this situation do what everyone does: they skim, they approve, and one wrong number ships.

The failure is not that the model writes badly. It is that we asked it to produce sentences when the thing we were missing was facts. Sentences are the part it already has.

Invert the task: make the holes the deliverable

Hole-first drafting flips the order of work. In the draft phase, the model's job is not to fill the page; it is to find every place where the page needs a fact it cannot source, and mark it. Filling those marks is a separate phase with a separate owner.

That gives you three properties you do not get from a better prompt:

• The set of missing facts is enumerable before review starts, so review has a scope instead of a vibe.
• The prose frame and the fact fills are separate artifacts, so you can freeze one and audit the other.
• "The model invented a number" stops being a tone problem and becomes a failing check with a file and line number.

The slot grammar

The whole protocol rests on two shapes, both greppable and both human-readable in a raw diff:

open slot    [[?name]]
filled slot  [[!name=value @ citation]]

Rules that make it enforceable rather than decorative:

1. Every causal or numeric claim in a fact-bearing doc must be a slot, not free prose.
2. A filled slot without value @ citation is invalid.
3. A citation is a repo path with an optional #anchor, or an http(s) URL. If the path does not resolve, the fill is invalid.
4. A fill PR may change slot contents and nothing else. The prose frame is hashed at draft time and must match at fill time.

The artifact: slotcheck.py

One file, standard library only, two modes that map to the two PR types. Save it as tools/slotcheck.py.

#!/usr/bin/env python3
"""slotcheck.py - hole-first drafting gate.

Usage:
  python slotcheck.py draft docs/*.md   # model PR: prose frame + open slots
  python slotcheck.py fill  docs/*.md   # human PR: slots resolved, frame frozen
"""
from __future__ import annotations

import hashlib
import json
import pathlib
import re
import sys

SLOT = re.compile(r"\[\[(\?|!)([a-z0-9_]+)(?:=(.+?)\s*@\s*(\S+))?\]\]")
FREEZE = pathlib.Path("docs/.slots/frozen.json")
MIN_SLOTS = 3  # per doc; tune per doc class, do not set to 0 to "clean up CI"


def frame(text: str) -> str:
    """Collapse every slot to its open form: the part the model owns."""
    return SLOT.sub(lambda m: f"[[?{m.group(2)}]]", text)


def digest(text: str) -> str:
    return hashlib.sha256(frame(text).encode("utf-8")).hexdigest()[:16]


def citation_ok(cite: str) -> bool:
    if cite.startswith(("http://", "https://")):
        return "/" in cite.split("//", 1)[1]  # shape check only, see limitations
    return pathlib.Path(cite.split("#", 1)[0]).exists()


def scan(path: pathlib.Path):
    open_slots, filled, errors = [], [], []
    text = path.read_text(encoding="utf-8")
    for lineno, line in enumerate(text.splitlines(), 1):
        for kind, name, value, cite in SLOT.findall(line):
            if kind == "?":
                open_slots.append(f"{path}:{lineno}:{name}")
                continue
            if not value or not cite:
                errors.append(f"{path}:{lineno}: {name} filled without value@cite")
            elif not citation_ok(cite):
                errors.append(f"{path}:{lineno}: {name} cites a dead path -> {cite}")
            else:
                filled.append(f"{path}:{lineno}:{name}")
    return text, open_slots, filled, errors


def main() -> int:
    if len(sys.argv) < 3:
        print(__doc__)
        return 2
    mode, targets = sys.argv[1], [pathlib.Path(p) for p in sys.argv[2:]]
    if mode not in {"draft", "fill"}:
        print(f"unknown mode: {mode}")
        return 2

    frozen = json.loads(FREEZE.read_text()) if FREEZE.exists() else {}
    errors: list[str] = []

    for path in targets:
        text, open_slots, filled, scan_errors = scan(path)
        errors += scan_errors
        key = str(path)

        if mode == "draft":
            total = len(open_slots) + len(filled)
            if total < MIN_SLOTS:
                errors.append(f"{path}: draft has {total} slot(s), need >= {MIN_SLOTS}")
            frozen[key] = digest(text)
        else:
            if open_slots:
                errors.append(f"{path}: {len(open_slots)} unresolved slot(s): {open_slots}")
            if key not in frozen:
                errors.append(f"{path}: no frozen draft; run `draft` mode first")
            elif digest(text) != frozen[key]:
                errors.append(f"{path}: prose changed outside slots (frame hash drift)")

    if mode == "draft":
        FREEZE.parent.mkdir(parents=True, exist_ok=True)
        FREEZE.write_text(json.dumps(frozen, indent=2, sort_keys=True) + "\n")

    for line in errors:
        print(f"FAIL {line}")
    print(f"{mode}: {len(targets)} file(s), {len(errors)} problem(s)")
    return 1 if errors else 0


if __name__ == "__main__":
    sys.exit(main())

The interesting function is frame. It rewrites every filled slot back to [[?name]] before hashing, which means the hash is invariant across draft and fill. A human can change 250 ms to 300 ms without breaking the gate. A human cannot quietly rewrite the surrounding paragraph, and neither can a second model pass.

What it looks like on a real doc

Draft, as the model produces it:

## Retry budget

The upload worker retries a failed part [[?retry_budget]] before it surfaces an error.
Backoff starts at [[?backoff_base]] and doubles on each attempt, capped at [[?backoff_cap]].

Fill, as a human produces it after reading deploy/worker.yaml:

## Retry budget

The upload worker retries a failed part [[!retry_budget=3 attempts @ deploy/worker.yaml#L41]] before it surfaces an error.
Backoff starts at [[!backoff_base=250 ms @ deploy/worker.yaml#L52]] and doubles on each attempt, capped at [[!backoff_cap=8 s @ deploy/worker.yaml#L53]].

Example output from the two runs (illustrative, not a benchmark):

$ python slotcheck.py draft docs/upload-worker.md
draft: 1 file(s), 0 problem(s)

$ python slotcheck.py fill docs/upload-worker.md
FAIL docs/upload-worker.md: 2 unresolved slot(s): ['docs/upload-worker.md:9:backoff_base', 'docs/upload-worker.md:9:backoff_cap']
fill: 1 file(s), 1 problem(s)

Wire the two modes into CI

Two jobs, two identities. The draft job runs on the bot's PR; the fill job runs on the human's PR. Keep the checks dumb and the messages line-numbered.

jobs:
  draft-gate:
    if: github.event.pull_request.user.login == 'docs-bot'
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: python tools/slotcheck.py draft docs/*.md
      - run: git diff --exit-code docs/.slots/frozen.json || echo '::notice::freeze updated, commit it with the draft'

  fill-gate:
    if: github.event.pull_request.user.login != 'docs-bot'
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: python tools/slotcheck.py fill docs/*.md

Decision table: which sentences become slots

The protocol only pays off if you decide this once, per doc class, and write it down.

Question the sentence answers	Who may answer	Becomes a slot?
What does this function do today?	Model, citing the file	No — but prose must name the file
Why was this built this way?	Human	Yes
Any number: latency, limit, price, timeout	Human	Always
What a user or team experienced	Human	Always
Migration order and rollback steps	Human	Yes
Config key names, API signatures	Model, verified by a repo grep test	No
Planned rewrite or deprecation date	Human	Always
Restating a README that the repo already owns	Nobody — link it	No

The middle column is the contract. If a doc class has no human-owned rows, the model can draft it end to end and the slot gate is wasted ceremony.

A reproducible test plan

Run this before you trust the gate on your own repo. It takes about ten minutes.

1. Draft a doc with three open slots. Confirm draft mode exits 0 and writes docs/.slots/frozen.json.
2. Fill all three with valid repo paths. Confirm fill mode exits 0.
3. Change one filled value (250 ms to 300 ms), leaving the citation. Confirm fill mode still exits 0 — this is the case that must not be blocked.
4. Add a sentence outside any slot. Confirm fill mode fails with frame hash drift.
5. Point one citation at deploy/nope.yaml. Confirm fill mode fails with cites a dead path.
6. Delete a slot entirely and run fill. Confirm the frame hash catches it.
7. Regression case: run the whole thing on a doc with zero facts (a changelog index). If MIN_SLOTS forces slot padding there, your doc classes need splitting, not a lower threshold.

Limitations, and who should not use this

Be honest about the ceiling. A citation check proves the path exists; it does not prove the path supports the number you wrote next to it. A tired human can cite L52 while reading L51, and slotcheck.py will pass. This gate moves the failure from "invisible" to "one link away", which is better but not the same as verified.

The frame hash also blocks legitimate concurrent editing. Two people fixing different paragraphs in the same file will trip it, so this works best with small docs and short fill windows.

Do not adopt this if:

• Your docs have no stable artifact to cite — early design RFCs, positioning pages, anything that is itself the source of truth.
• Your doc set is small enough that one reviewer reads every line anyway; the ceremony outweighs the catch rate.
• You need sub-hour turnaround on doc fixes, since the protocol adds a phase boundary by design.
• You cannot enforce two PR identities. Without a separate bot identity, draft and fill collapse into one, and the whole point evaporates.

Where free model access fits the loop

The draft phase is high-volume and low-trust: you want many passes at "where are the holes in this page", and you want the ability to throw all of them away. That is exactly the workload where paid per-token calls get rationed by habit rather than by need, and rationed drafting means fewer passes and more unmarked invented facts.

MonkeyCode is relevant here for one specific reason: it offers free model access and a free server option, which lets you run the hole-finding pass repeatedly and host the slot checker plus its freeze file without opening a spend conversation first. Check the project's own site for the current free allocation and server terms — quotas and hardware change, and this article deliberately does not restate numbers that may be stale by the time you read it.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Nothing above depends on the product being free. The slot grammar, the script, and the two CI jobs run identically against any model endpoint, which is the point: the protocol is the durable part, and the drafting budget is a temporary detail.

The review question changes

With the protocol in place, a doc reviewer stops asking "is any of this wrong?" and starts asking "are the holes filled with things I can open?" That is a question a person can actually answer at 5 p.m. on a Friday.

The model keeps the part it is good at — structure, transitions, complete sentences around a gap — and you keep the part it cannot have: knowing what is true in your system this week. If you want to try the draft half without a procurement detour, MonkeyCode's free models and free server are one way to get the loop running today.