DEV Community

Avery Lin
Avery Lin

Posted on

Compile Docs Through Typed Slots: Models Fill EXTRACT, Humans Own AUTHOR

Generated documentation stays trustworthy only when every model sentence occupies a typed slot with a hashed source. EXTRACT slots restate files and schemas; RENDER slots format fixtures; AUTHOR slots remain empty until a human writes them. A compile step that rejects untyped prose prevents obligation language from arriving as if it were an observed fact. This article specifies a slot compiler, a decision table, and a small Python checker you can run in CI.

Free-form drafts mix observation with obligation

Most documentation generators emit a continuous narrative from a repository snapshot plus a loosely worded prompt. That narrative often interleaves endpoint lists, which can be checked, with support windows, which cannot. Readers cannot see which sentences were derived from OpenAPI and which sentences were invented as policy. Review then becomes a full reread instead of a slot-by-slot audit against hashed inputs.

A compiler framing fixes that mixing problem without asking the model to judge policy language. The template is the grammar, and the model is only a filler for already-typed holes. Anything that cannot be filled from an extract or a fixture stays blank, and CI fails the build. The human authoring pass is therefore bounded to AUTHOR identifiers rather than to an undifferentiated wall of markdown.

Three slot kinds, one compile rule

Define every replaceable region in the documentation template as one of three explicit kinds.

  1. EXTRACT — Restate a named file, schema path, or test identifier, and hash that source before generation starts.
  2. RENDER — Format a fixture or recorded response without adding fields, hedges, or compatibility commentary of any kind.
  3. AUTHOR — Keep the literal placeholder untouched until a named human commit replaces the identifier with owned prose.

The compile rule stays short and mechanical so CI can apply it without editorial judgment. A model may write only into EXTRACT and RENDER regions, and only from the JSON bundle built from those hashes. If the model writes into AUTHOR, or emits tokens outside any slot, the compiler rejects the entire draft. If an EXTRACT bundle is stale relative to the current file hash, the compiler rejects the draft as well.

A seven-step compile workflow

The following sequence is a proposal you can implement against one OpenAPI document and one markdown template.

  1. Inventory the source files that documentation may observe, and record a sha256 digest for every allowed path.
  2. Write the markdown template with explicit SLOT tags rather than implied headings that a model might freely extend.
  3. Run an extractor that copies only allowlisted JSON paths into facts.json, emitting no free-text commentary at all.
  4. Send facts.json and the slotted template to a model instructed to fill EXTRACT and RENDER regions only.
  5. Run slot_compiler.py on the filled markdown and fail the job on untyped prose, AUTHOR writes, or hash drift.
  6. Open a human-owned change that replaces each AUTHOR identifier with policy, audience guidance, or an explicit blank.
  7. Publish the page only when the compiler reports a clean slot map and a complete AUTHOR ownership ledger.

Each step leaves an artifact on disk, so a later reviewer can replay the compile without trusting chat history. That replay property is the point of treating documentation as a build, not as a conversation. If replay is impossible, the page is no longer generated documentation; it is an untraceable edit.

Template shape the compiler can parse

A minimal template makes the three kinds visible to both the model and the compiler. Keep slot bodies small so a failed fill does not hide inside a long paragraph. Reviewers can grep for SLOT markers instead of rereading the whole page for hidden promises. The support-window section is left as a placeholder on purpose during the model pass.

# Payments HTTP API

## Endpoints
<!-- SLOT EXTRACT id=endpoints source=openapi.json sha256={{HASH:openapi.json}} -->
{{EXTRACT:endpoints}}
<!-- /SLOT -->

## Example: create payment
<!-- SLOT RENDER id=create_payment_201 fixture=fixtures/create_payment_201.json sha256={{HASH:fixtures/create_payment_201.json}} -->
{{RENDER:create_payment_201}}
<!-- /SLOT -->

## Support window
<!-- SLOT AUTHOR id=support_window owner=docs-oncall -->
{{AUTHOR:support_window}}
<!-- /SLOT -->
Enter fullscreen mode Exit fullscreen mode

The support-window section is the typical leak in otherwise careful generated API handbooks today. Models often draft support windows because surrounding headings sound like marketing copy rather than schema restatement. Leaving the AUTHOR placeholder visible turns that leak into a compile error instead of a subtle wording defect. A wording defect survives review; a compile error blocks the merge until a human owns the identifier.

Extractor: facts only, no prose

The extractor should print JSON that a model can copy, not paragraphs that a model can paraphrase later. Paraphrase is the usual path by which extra modality words enter an otherwise factual section. An allowlist of dotted paths is stricter than asking the model to use the spec as loose context. If a path is absent from the allowlist, the model never sees it and cannot document an internal field.

# extract_facts.py — proposal / worked example, not a measured production run
import hashlib, json, sys
from pathlib import Path

def sha256(path: Path) -> str:
    return hashlib.sha256(path.read_bytes()).hexdigest()

def pick(obj, dotted):
    cur = obj
    for part in dotted.split("."):
        if part.isdigit():
            cur = cur[int(part)]
        else:
            cur = cur[part]
    return cur

def main(openapi_path: str, allowlist_path: str, out_path: str) -> None:
    src = Path(openapi_path)
    allow = json.loads(Path(allowlist_path).read_text())
    spec = json.loads(src.read_text())  # convert YAML to JSON in a prior step
    facts = {
        "source": str(src),
        "sha256": sha256(src),
        "fields": {key: pick(spec, path) for key, path in allow["extract"].items()},
    }
    Path(out_path).write_text(json.dumps(facts, indent=2, sort_keys=True) + "\n")

if __name__ == "__main__":
    main(sys.argv[1], sys.argv[2], sys.argv[3])
Enter fullscreen mode Exit fullscreen mode
{
  "extract": {
    "create_payment_path": "paths./v1/payments.post",
    "payment_id_schema": "components.schemas.Payment.properties.id"
  }
}
Enter fullscreen mode Exit fullscreen mode
python extract_facts.py openapi.json allowlist.json facts.json
sha256sum facts.json openapi.json
Enter fullscreen mode Exit fullscreen mode

Run the extractor in the same job that later runs the compiler so the hashes share one tree. Do not cache facts.json across unrelated commits, because a silent schema edit would otherwise keep an old extract. When the source is YAML, convert it to JSON first and hash the converted bytes the extractor actually read. Document that conversion inside the template attributes so reviewers do not compare the wrong digest.

Compiler: reject untyped prose

The compiler walks the filled markdown, checks slot boundaries, and compares hashes against facts.json. Treat any text between slots as a defect, including polite introductions and transition sentences. Forbidden-token matching is a backstop for obligation language that slipped into EXTRACT or RENDER bodies. It is not a substitute for keeping AUTHOR regions untouched during the model pass.

# slot_compiler.py — proposal / worked example
import hashlib, json, re, sys
from pathlib import Path

SLOT_RE = re.compile(
    r"<!-- SLOT (?P<kind>EXTRACT|RENDER|AUTHOR) id=(?P<id>\S+) (?P<attrs>.*?) -->\n"
    r"(?P<body>.*?)\n"
    r"<!-- /SLOT -->",
    re.S,
)
FORBIDDEN = re.compile(
    r"\b(will support|must|SLA|until further notice|breaking change|guaranteed)\b",
    re.I,
)

def sha256_file(path: str) -> str:
    return hashlib.sha256(Path(path).read_bytes()).hexdigest()

def parse_attrs(raw: str) -> dict:
    return dict(re.findall(r"(\w+)=(\S+)", raw))

def outside(text, slots):
    parts, last = [], 0
    for match in slots:
        parts.append(text[last:match.start()])
        last = match.end()
    parts.append(text[last:])
    return parts

def check(template: str, filled: str, facts: dict) -> list[str]:
    errors = []
    t_slots = list(SLOT_RE.finditer(template))
    f_slots = list(SLOT_RE.finditer(filled))
    if len(t_slots) != len(f_slots):
        return ["slot count changed; model rewrote template structure"]
    if outside(template, t_slots) != outside(filled, f_slots):
        errors.append("untyped prose inserted outside SLOT markers")
    for tm, fm in zip(t_slots, f_slots):
        kind, sid = tm.group("kind"), tm.group("id")
        if (fm.group("kind"), fm.group("id")) != (kind, sid):
            errors.append(f"{sid}: slot header mutated")
            continue
        body = fm.group("body").strip()
        attrs = parse_attrs(tm.group("attrs"))
        if kind == "AUTHOR":
            placeholder = "{{AUTHOR:%s}}" % sid
            if body != placeholder and not body.startswith("<!-- human:"):
                errors.append(f"{sid}: AUTHOR filled by non-human pass")
        elif kind == "EXTRACT":
            src = attrs.get("source")
            if src and sha256_file(src) != facts.get("sha256"):
                errors.append(f"{sid}: EXTRACT hash drift against {src}")
            if FORBIDDEN.search(body):
                errors.append(f"{sid}: EXTRACT contains obligation language")
        elif kind == "RENDER":
            fixture = attrs.get("fixture")
            expected = attrs.get("sha256")
            if fixture and expected and sha256_file(fixture) != expected:
                errors.append(f"{sid}: RENDER fixture hash drift against {fixture}")
            if FORBIDDEN.search(body):
                errors.append(f"{sid}: RENDER contains obligation language")
    return errors

def main(template_path, filled_path, facts_path):
    errors = check(
        Path(template_path).read_text(),
        Path(filled_path).read_text(),
        json.loads(Path(facts_path).read_text()),
    )
    if errors:
        print("FAIL")
        print("\n".join(errors))
        sys.exit(1)
    print("PASS")

if __name__ == "__main__":
    main(sys.argv[1], sys.argv[2], sys.argv[3])
Enter fullscreen mode Exit fullscreen mode
python slot_compiler.py template.md filled.md facts.json
Enter fullscreen mode Exit fullscreen mode

Label these scripts as unexecuted examples that encode the rule rather than as a production incident study. They fail closed on structure, hash drift, and a small obligation lexicon rather than on taste. They will not prove that a restatement inside EXTRACT is semantically faithful to the schema. Semantic fidelity still needs a human spot-check or a snapshot test against golden JSON, which is a separate gate.

Human AUTHOR fills should replace the placeholder with a marker the compiler can recognize, then the owned prose. A conventional shape is <!-- human: docs-oncall --> followed by the policy paragraph in the same slot. Without that marker, a fluent model fill of support windows looks identical to a legitimate human edit and will not fail CI.

Decision table: what the model may draft

Claim in the filled draft Slot allowed Owner after compile Fail if
Path, method, status listed in OpenAPI EXTRACT model fill, hash-checked extra path or stale hash
Request field copied from schema EXTRACT model fill, hash-checked renamed field, added enum
Example body copied from a fixture RENDER model fill, fixture hash extra key, dropped error
Callers must retry with jitter AUTHOR human appears in EXTRACT or RENDER
Supported through 2027 AUTHOR human any date without human marker
This never breaks existing clients none human, usually refuse any slot until legal review
Introductory cheerleading between slots none rejected untyped prose

The last two rows matter more than the OpenAPI rows for most reviews. Dates, "never," and compatibility promises are not extractable facts in an interface document. If your process needs those sentences, they belong in a human change with an owner field. Until that owner exists, the compiler should keep the page unpublished rather than shipping fluent guesses.

Where a bounded model fill belongs

The fill step is the only place a model needs to run for this workflow. It should receive facts.json plus the slotted template, not a checkout of the entire repository. Repository-wide context is how AUTHOR material leaks into EXTRACT under the guise of being helpful. Narrow context is therefore a safety control, not merely a token-saving tactic.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

MonkeyCode's free model access and free server option can host that bounded fill when you want the job off a laptop. Keep the extractor and the compiler local; they are deterministic Python and should not depend on a model at all. A practical split keeps extract and compile on the agent that already has the repository checkout. Send only the facts bundle to the remote fill, then compile locally and discard drafts with a moved slot map.

Limitations

This compiler does not prove that an EXTRACT restatement is semantically faithful, only that it stayed inside markers. Models can still mis-copy a field description while remaining inside a well-formed slot. The forbidden-token list is incomplete by construction, because obligation language has many paraphrases and hedges. Hash drift catches stale sources, not wrong interpretations of a current and correctly hashed source.

Templates with large conceptual sections will fight this design from the first compile. If your handbook is mostly narrative, slot density will be low and the compiler will reject connective glue. The method assumes you can point at files, schemas, or fixtures for the majority of the page. When that assumption fails, stop generating and write the page as a human AUTHOR document with no model pass.

Who should not use this approach

Do not use this workflow for legal terms, security advisories, or pricing pages that must be entirely human-authored. Do not use it when no canonical schema or fixture exists, because EXTRACT and RENDER would be empty theater. Do not treat a passing compile as API review; structural cleanliness does not mean the interface is coherent. Teams that cannot fail CI on documentation should not adopt a compiler they will immediately bypass with force-merge.

What the model may draft, restated

The model may draft restatements of hashed extracts and formatting of hashed fixtures only. The model may not draft promises, calendars, audience advice, or untyped connective prose between markers. Humans own every AUTHOR identifier, including the decision to leave a slot blank and unpublished. If a sentence cannot name its slot kind, it does not belong in the generated pass.

Start from one endpoint, one fixture, and one AUTHOR identifier that you already know is policy. Attach the compiler to that page until it fails a deliberately bad draft that writes into AUTHOR. If a docs compile already runs in your pipeline, attach this slot gate to one OpenAPI file before expanding the allowlist.

Top comments (0)