DEV Community

Avery Lin
Avery Lin

Posted on

Human Fragments Stay Off the Prompt: A Slot-and-Link Workflow for Generated Docs

Generated documentation stays trustworthy when human-owned fragments never enter the model prompt and never share a writable file with machine drafts. Mixing caveats and generated procedures in one markdown file makes every regeneration a merge conflict against editorial judgment. The practical fix is a link step that stores human fragments separately and exposes only typed slots to the model. This article specifies that layout, a small Python linker, and a fill checker that rejects obligation language in model output.

The failure mode is mechanical rather than stylistic. A model that can see an SLA paragraph will paraphrase it, soften a number, or drop a negative constraint while rewriting nearby steps. Reviewers then compare two full files and miss the one clause that actually changed the product promise. Treating documentation like object files avoids that class of miss: humans compile fragments, models fill typed slots, and a linker emits the page that readers see.

What the model may draft versus what a human must own

Route work by claim type, not by heading title. Titles are cheap to invent and easy for a generator to mimic; claim types describe whether a sentence can be false without changing a user-visible contract. The split below is a working default for API and operator docs, not a legal taxonomy.

  1. Model-fillable slots. Numbered procedures, flag tables copied from --help, example skeletons, public identifier glossaries, and non-normative diagrams in text form.
  2. Human-owned fragments. Support boundaries, security non-goals, breaking-change policy, version support windows, billing rules, and any sentence that uses obligation language.
  3. Never a slot. Dates that create an SLA, incident language, export-control notes, and statements of the form “we will” or “guaranteed.”
  4. Link-time only. Table of contents, cross references, and “last assembled” stamps, which the linker can generate without a model.

Obligation language is the practical classifier. If a sentence would be quoted in a ticket against support, it does not belong in a slot brief or a model fill. If a sentence would be wrong only because a flag was renamed, it can live in a slot and be regenerated when the CLI surface changes.

Repository layout

Keep three trees and one schema. Published markdown is an output, not an authoring surface, which is the entire point of the link step.

docs/
  slots.yaml
  fragments/
    support-boundary.md
    breaking-changes.md
    security-non-goals.md
  briefs/
    install-steps.md
    flag-table.md
    glossary.md
  fills/
    install-steps.md
    flag-table.md
    glossary.md
  out/
    operator-guide.md
Enter fullscreen mode Exit fullscreen mode

slots.yaml names every hole the linker will fill, points at a brief the model is allowed to read, and lists the fragment identifiers that must appear around that hole. Briefs contain commands, fixture names, and structural hints. They must not paste the fragment files, or the prompt contamination problem returns under a new filename.

# docs/slots.yaml
document: operator-guide
order:
  - fragment: support-boundary
  - slot: install-steps
  - slot: flag-table
  - fragment: breaking-changes
  - slot: glossary
  - fragment: security-non-goals
slots:
  install-steps:
    brief: briefs/install-steps.md
    fill: fills/install-steps.md
    allow: [procedure, command]
    forbid: [obligation, date_promise]
  flag-table:
    brief: briefs/flag-table.md
    fill: fills/flag-table.md
    allow: [table, flag]
    forbid: [obligation, security_claim]
  glossary:
    brief: briefs/glossary.md
    fill: fills/glossary.md
    allow: [definition]
    forbid: [obligation]
fragments:
  support-boundary: fragments/support-boundary.md
  breaking-changes: fragments/breaking-changes.md
  security-non-goals: fragments/security-non-goals.md
Enter fullscreen mode Exit fullscreen mode

A brief should look like a build input. The example below is complete enough to draft from and still free of promises the model must not author.

# briefs/install-steps.md
Slot: install-steps
Allow: numbered procedure and shell commands only.
Forbid: SLAs, “supported”, “guaranteed”, calendar dates, security claims.
Source commands the fill must quote exactly:
  python -m pip install -e .
  python -m docs_link.cli --check
Stop after the check command. Do not add a support story.
Enter fullscreen mode Exit fullscreen mode

Numbered assembly workflow

The sequence is intentionally boring. Each step has a file it may write and a file it must not read.

  1. Edit fragments by hand. Change support or policy text only in docs/fragments. Do not paste those files into a chat transcript or a brief.
  2. Refresh briefs from code. When flags or install commands change, update docs/briefs from --help output or a fixture dump, not from memory.
  3. Fill slots in isolation. Send one brief per request. Write the model reply to docs/fills/<slot>.md and nowhere else.
  4. Scan fills before linking. Reject obligation language, calendar promises, and any heading that collides with a fragment identifier.
  5. Link to docs/out. Concatenate fragments and fills in slots.yaml order. Fail the build if a path is missing or a fill is stale relative to its brief.
  6. Publish the output tree only. Readers and the static site generator see docs/out. Reviewers still diff fragments and fills separately.

Step three is where a free drafting endpoint earns its keep. Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode's free model access and free server option can host that isolated fill step without putting fragment files on the same writable volume as the generator. The linker and the checker still run in CI on the repository, which remains the source of ownership.

Artifact: linker, obligation scan, and a regression test

The checker is deliberately lexical. It will not understand sarcasm, but it will catch the sentences that create tickets. Keep the pattern list in version control next to slots.yaml so a policy change is a diff, not a hallway decision.

# tools/docs_link.py
from __future__ import annotations

import re
import sys
from pathlib import Path

import yaml

ROOT = Path("docs")
OBLIGATION = re.compile(
    r"\b(we will|guaranteed|sla|always supported|never break|must succeed)\b",
    re.I,
)
DATE_PROMISE = re.compile(
    r"\b(until|through|before)\s+\d{4}-\d{2}-\d{2}\b", re.I
)


def load_schema() -> dict:
    return yaml.safe_load((ROOT / "slots.yaml").read_text())


def scan_fill(slot_name: str, text: str, forbid: list[str]) -> list[str]:
    errors = []
    if "obligation" in forbid and OBLIGATION.search(text):
        errors.append(f"{slot_name}: obligation language is not allowed in fills")
    if "date_promise" in forbid and DATE_PROMISE.search(text):
        errors.append(f"{slot_name}: dated promises are not allowed in fills")
    if re.search(r"^# ", text, re.M):
        errors.append(f"{slot_name}: fills may not introduce top-level titles")
    return errors


def link() -> int:
    schema = load_schema()
    errors: list[str] = []
    parts: list[str] = []
    for item in schema["order"]:
        if "fragment" in item:
            path = ROOT / schema["fragments"][item["fragment"]]
            if not path.exists():
                errors.append(f"missing fragment {path}")
                continue
            parts.append(path.read_text().rstrip() + "\n")
            continue
        slot = schema["slots"][item["slot"]]
        fill = ROOT / slot["fill"]
        brief = ROOT / slot["brief"]
        if not fill.exists() or not brief.exists():
            errors.append(f"missing fill or brief for {item['slot']}")
            continue
        if fill.stat().st_mtime < brief.stat().st_mtime:
            errors.append(f"stale fill for {item['slot']}; brief is newer")
        errors.extend(scan_fill(item["slot"], fill.read_text(), slot["forbid"]))
        parts.append(fill.read_text().rstrip() + "\n")
    if errors:
        sys.stderr.write("\n".join(errors) + "\n")
        return 1
    out = ROOT / "out" / f"{schema['document']}.md"
    out.parent.mkdir(parents=True, exist_ok=True)
    header = f"<!-- assembled from slots.yaml; do not edit {out.name} -->\n\n"
    out.write_text(header + "\n".join(parts))
    return 0


if __name__ == "__main__":
    raise SystemExit(link())
Enter fullscreen mode Exit fullscreen mode

Run the linker locally with the same command CI will use. A non-zero status means the output tree is not publishable, even if docs/out still contains yesterday’s page.

python -m pip install pyyaml pytest
python tools/docs_link.py
test -f docs/out/operator-guide.md
Enter fullscreen mode Exit fullscreen mode

The regression test pins one negative case: a fill that quietly introduces a support promise must fail before link. Positive fixtures stay small so the test remains a contract, not a novel.

# tools/test_docs_link.py
from pathlib import Path

import docs_link as dl


def test_obligation_in_fill_is_rejected(tmp_path, monkeypatch):
    monkeypatch.setattr(dl, "ROOT", tmp_path)
    (tmp_path / "fills").mkdir()
    text = "Install the package. We will always supported this path.\n"
    errors = dl.scan_fill("install-steps", text, ["obligation"])
    assert errors, "obligation language must fail the fill scan"


def test_procedure_without_promise_passes():
    text = "1. Run `python -m pip install -e .`\n2. Run `python tools/docs_link.py`.\n"
    assert dl.scan_fill("install-steps", text, ["obligation", "date_promise"]) == []
Enter fullscreen mode Exit fullscreen mode
pytest -q tools/test_docs_link.py
Enter fullscreen mode Exit fullscreen mode

Staleness is part of the contract. If a brief changes because --help gained a flag, an old fill must not link even when the obligation scan is clean. That rule is what makes regeneration local: only the stale slot is sent back to the model, and fragment files stay out of that request body.

What this does not prove

The linker proves assembly hygiene, not truth. A fill can still invent a flag that does not exist, and the obligation scan will not notice a missing command. Pair this workflow with an example runner if procedures are executable; the two gates answer different questions. Lexical forbid lists also drift. Teams that add marketing copy to fragments will train reviewers to ignore the split, which is worse than not splitting at all.

Free model fills remain drafts. Isolation reduces prompt leakage; it does not certify that a generated table matches production. Keep the fill step replaceable. The schema, fragments, and checker should still work if the drafting endpoint changes or goes offline for a day.

Who should not use this approach

Skip the link step for a single-page README that one maintainer edits in place. The file overhead is not justified when regeneration is rare and the only promise is “run the tests.” Skip it for filings, contracts, and security advisories that must be wholly human-authored; putting those texts in fragments/ is fine, but slots should not exist beside them. Skip it when nobody will fail CI. A linker that is routinely bypassed with generated output checked into docs/out by hand is a more confusing version of the mixed file you already have.

The core conclusion does not depend on a particular vendor. Human-owned claims need a file the model never writes, and model-drafted procedures need a hole the linker can replace without a semantic merge. If you already isolate fills on a free-model workspace, point that workspace at docs/briefs only and keep docs/fragments on the side of the repository that CI, not the generator, is allowed to read.

Top comments (0)