Keep Escalation Facts Out of Model-Drafted Troubleshooting Pages

Generated troubleshooting pages remain trustworthy only when escalation facts stay entirely outside the model draft boundary. A model may describe symptoms, quote captured commands, and arrange fixture output into readable prose for reviewers. A human must own severity, data handling, supported versions, and the explicit decision to page an on-call owner. This article specifies that split as a section manifest, a failing checker, and a narrow review path for operators.

Troubleshooting prose fails in a predictable way when a draft copies an incident note into a public page. The draft sounds specific, yet the severity label, retention window, and paging rule were never approved for publication. Readers then treat a generated sentence as an operational commitment, which is a documentation defect rather than a style issue. The workflow below separates draftable narration from owned assertions before any model call is allowed to begin.

Scope of the split

The split applies to internal runbooks that later become public troubleshooting pages, not to legal notices or status pages. It assumes the repository already stores redacted logs, command transcripts, and expected fixture output under version control. It does not assume a particular model vendor, quota, or hosting duration, because those details change and must be read from current terms. Operators who lack a named human owner for escalation facts should stop before generating any public prose.

What the model may draft

The model may draft only those sections that the boundary manifest explicitly marks as model_draftable for this page. Those sections cover symptom narrative, command sequence explanation, and a plain-language reading of fixture output that already exists in the repository. The model may also propose headings, cross-links, and a short note that points readers back toward the owned sections. It may not invent exit codes, version ranges, customer impact, or paging thresholds that are absent from the supplied packet.

A useful draft packet contains exactly three inputs, and it should contain nothing beyond those three inputs. The first input is a redacted symptom log whose timestamps have been removed or normalized before packaging. The second input is the exact command list the operator already ran, including flags that are safe to publish. The third input is the fixture output file that the checker will later hash, so the prose cannot drift from the captured result.

What a human must own

A human must own every sentence that would change operational behavior if a reader followed it literally. That set includes severity, paging criteria, data-retention statements, supported version ranges, and the named escalation owner. The human also owns the decision that a fixture is canonical rather than illustrative, because that choice changes what the page guarantees. Ownership is recorded as an identifier and a review date, not as a vague reviewed badge in the generated file.

Owned sections live in a separate file that the generator is forbidden to overwrite during a draft run. The draft file may quote an owned section by identifier, but the quote is rendered from the owned file at build time. If the owned file changes, the draft is stale until a human re-approves the boundary, even when the surrounding prose still reads well. This rule keeps support commitments attached to a named person instead of a model completion or a merge timestamp.

Decision table

The table below classifies five common runbook sections and states which side may write each one. A missing human owner is a hard failure, while a missing fixture hash is also a hard failure for draftable sections. The failure column is a review rule for the checker, not an estimate of how often drafts misbehave. Edit the rows before adoption so they match the support policy your readers already rely on.

Section	Class	Model may draft	Human must own	Failure if missing
Symptom narrative	model_draftable	Yes, from a redacted log	Confirm that no secrets remain	Checker rejects raw tokens
Command explanation	model_draftable	Yes, from listed commands	Confirm that flags are publishable	Checker rejects unknown flags
Fixture reading	model_draftable	Yes, from a hashed fixture	Mark the fixture canonical or sample	Checker rejects a hash mismatch
Severity and paging	human_owned	No	Severity, page rule, and owner	Checker rejects model markers
Retention and versions	human_owned	No	Retention statement and version range	Checker rejects forbidden draft phrases

The table is a proposal for this workflow, not a measured benchmark and not a claim about any vendor's accuracy. Teams should edit the forbidden-phrase list to match their own support policy before they trust the gate. A phrase that is harmless in one product can be a commitment in another, so the list is local data.

Numbered workflow

Follow the five steps in order, and do not skip the packet check merely to save a model call. Each step writes or reads a file that the next step can verify without opening a chat transcript. The sequence assumes a single runbook page, though the same manifest shape can list several pages. If a step fails, fix that file and rerun the checker before you continue to the model path.

1. Create the boundary file, list every section with a class and an owner, and keep owned sources outside the draft directory.
2. Redact logs, copy only publishable commands into the packet directory, and store a SHA-256 digest beside each fixture.
3. Run the checker in packet mode so it refuses the job when a forbidden key appears in the packet.
4. Send only model-draftable section briefs to the model path, then write the result under the draft directory with a provenance footer.
5. Run the checker in draft mode, render a local preview, and require the named owner to update the review date before merge.

Each step is intentionally small so a failed check points at one file rather than a whole conversation. The generator never receives the owned directory as writable context during the model call or the preview build. If a step needs a new operational fact, a person adds that fact to the owned file and rebuilds the packet. That rebuild is the only path by which a new fact becomes visible to later narration.

Proposed checker

The following Python is an unexecuted example intended for a repository CI job, and it does not call a remote API. It is not a production library, so install PyYAML yourself and review the marker format before you rely on it. The script reads a YAML boundary, hashes fixtures, and scans drafts for markers that indicate owned-section contamination. Adapt the forbidden list to your support policy, because the sample phrases are illustrations rather than a complete policy.

#!/usr/bin/env python3
"""Unexecuted example: reject drafts that touch human-owned runbook facts."""
from __future__ import annotations

import hashlib
import re
import sys
from pathlib import Path

import yaml  # proposal: PyYAML; pin it in your own lockfile

BOUNDARY_MARK = re.compile(
    r"<!--\s*boundary:\s*([a-z0-9-]+)\s+class:\s*(human_owned|model_draftable)"
)
FORBIDDEN = ("page immediately", "retention is", "supported since", "severity: p")


def read_text(path: Path) -> str:
    if not path.is_file():
        raise SystemExit(f"missing file: {path}")
    return path.read_text(encoding="utf-8")


def sha256(path: Path) -> str:
    return hashlib.sha256(path.read_bytes()).hexdigest()


def load_boundary(path: Path) -> dict:
    data = yaml.safe_load(read_text(path))
    if not isinstance(data, dict) or "sections" not in data:
        raise SystemExit("boundary file missing sections")
    return data


def marker_error(path: Path, section_id: str, klass: str) -> str | None:
    match = BOUNDARY_MARK.search(read_text(path))
    if not match:
        return f"{path}: missing boundary marker"
    found_id, found_class = match.group(1), match.group(2)
    if found_id != section_id or found_class != klass:
        return f"{path}: marker {found_id}/{found_class} != {section_id}/{klass}"
    return None


def check_packet(boundary: dict, root: Path) -> list[str]:
    errors: list[str] = []
    for section in boundary["sections"]:
        if section["class"] != "model_draftable":
            continue
        packet = root / section["packet"]
        err = marker_error(packet, section["id"], "model_draftable")
        if err:
            errors.append(err)
        text = read_text(packet).lower()
        for phrase in FORBIDDEN:
            if phrase in text:
                errors.append(f"{packet}: forbidden phrase {phrase!r}")
        fixture = root / section["fixture"]
        if sha256(fixture) != section["fixture_sha256"]:
            errors.append(f"{fixture}: hash mismatch")
    return errors


def check_draft(boundary: dict, root: Path) -> list[str]:
    errors: list[str] = []
    for section in boundary["sections"]:
        if section["class"] != "human_owned":
            continue
        if not section.get("owner") or not section.get("reviewed_on"):
            errors.append(f"{section['id']}: owner and reviewed_on required")
        source = root / section["source"]
        err = marker_error(source, section["id"], "human_owned")
        if err:
            errors.append(err)
        owned = read_text(source)
        if "MODEL_DRAFT" in owned or "provenance:" in owned:
            errors.append(f"{section['source']}: model marker in owned file")
        sibling = section.get("draft_sibling")
        if not sibling:
            errors.append(f"{section['id']}: draft_sibling required")
            continue
        draft = root / sibling
        if draft.exists() and owned.strip() and owned.strip() in read_text(draft):
            errors.append(f"{draft}: owned text copied into draft; render it at build time")
    return errors


def main() -> int:
    if len(sys.argv) != 4 or sys.argv[1] not in {"--packet", "--draft"}:
        print("usage: check_runbook_boundary.py --packet|--draft BOUNDARY ROOT")
        return 2
    mode, boundary_path, root = sys.argv[1], Path(sys.argv[2]), Path(sys.argv[3])
    boundary = load_boundary(boundary_path)
    if mode == "--packet":
        errors = check_packet(boundary, root)
    else:
        errors = check_draft(boundary, root)
    for err in errors:
        print(err)
    return 1 if errors else 0


if __name__ == "__main__":
    sys.exit(main())

Marker files

Place the boundary marker on the first lines of each packet and each owned file so the checker can confirm identity. The marker is a review aid, not a security control, because a writer can still add unsafe prose below it. Use the packet example only after redaction, and use the owned example only as a shell for human text. Do not copy the sample paging sentence into a real page, because it is a placeholder rather than a policy.

<!-- boundary: symptom-narrative class: model_draftable -->

The health command exits with status 1.
The captured stderr is stored in the fixture file.
No customer name appears in this packet.

<!-- boundary: escalation class: human_owned -->

Owner: oncall-lead
Paging rule: replace this sentence with human-authored policy.

A matching manifest can start as the short YAML file shown below, with dates and hashes treated as placeholders. Those placeholders are not observed measurements, and you should replace them after you compute digests locally. The owned section records a person and a review date, while the draftable section records a packet and a fixture hash. Keep the two records in one manifest so a reviewer can see the split without opening the generator prompt.

sections:
  - id: symptom-narrative
    class: model_draftable
    packet: docs/runbooks/packet/symptom.md
    fixture: docs/runbooks/packet/fixture.txt
    fixture_sha256: "replace-with-real-sha256"
    owner: none
  - id: escalation
    class: human_owned
    source: docs/runbooks/owned/escalation.md
    owner: oncall-lead
    reviewed_on: "2026-10-10"
    draft_sibling: docs/runbooks/draft/escalation.md

The commands below are a local shell sequence, not a vendor CLI, and they assume the script sits in the working directory. The first command blocks a contaminated packet before any model is allowed to receive those files. The second command blocks an owned file that contains draft markers or a draft that inlines owned text. The third command prints the digest you paste into the manifest after you recompute it on your own machine.

python3 check_runbook_boundary.py --packet docs/runbooks/boundary.yaml .
python3 check_runbook_boundary.py --draft docs/runbooks/boundary.yaml .
sha256sum docs/runbooks/packet/fixture.txt

Where a free model path and a free server fit

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

MonkeyCode's free model access is a reasonable place to draft only the model-draftable packet, provided you confirm current terms first. The free server option is a reasonable place to run the same checker and to serve the owner preview. Those two availability claims are operator-supplied for this article, and they are not quotas, hardware specs, or duration limits. They are also not a promise that the offer remains unchanged, so read the current terms before a release gate depends on them.

The model path should receive the packet directory and the section brief, not the owned directory as context. The server path should run the same checker binary you would run in CI, then expose the draft preview to the named owner. If either option is unavailable, the workflow still stands on a laptop, because the artifact does not require a hosted model to be valid. Using a free path does not reduce the human review step or justify an accuracy claim without a published measurement.

Limitations

The checker looks for phrases and markers, so a paraphrase can still smuggle a paging rule into a symptom section. Hash checks prove the fixture bytes did not change, but they do not prove the fixture was redacted correctly. A review date shows who accepted the owned file, yet it does not prove the owner still holds that role next week. Teams should add a periodic owner lookup outside this script rather than treating the review date as a permanent approval.

This example also ignores localization, screenshots, and diagrams, which can carry commitments the phrase list never sees. A translated page can reintroduce a forbidden commitment even when the source draft passed the checker cleanly. Screenshot text stays invisible to the phrase list unless you extract that text before the draft check runs. Those gaps are reasons to keep the gate narrow, not reasons to let the model own escalation facts.

Who should not use this approach

Skip this approach if you have no named owner for severity and paging, because the manifest would then record a fiction. Skip it for customer-specific incident reports, where even a carefully redacted symptom narrative may remain confidential. Skip it if the page is a contractual service commitment or a security advisory, because those documents need counsel rather than a phrase list. Skip it if you cannot keep the owned file out of the model context, since the split would then exist only on paper.

Closing

Treat the manifest as the contract between draft prose and operational facts, and regenerate narration only after that contract still passes. The checker is a proposal you can copy, pin, and extend with the forbidden phrases your support policy already uses. If you want a hosted place to draft only the allowed sections, confirm MonkeyCode's current free model access and free server option. Then keep the owned file on the human side of the boundary, regardless of which host runs the checker.