Compile Alert Duties Before a Model Drafts Runbook Symptoms

A runbook stays trustworthy only when escalation facts are frozen before any model writes symptom prose at all. Generated sentences can explain a firing page, but they cannot assign an owner, a severity, or a customer promise. This workflow compiles those duties from a reviewed alert catalog, then allows a model to draft only fields marked narrative. The check fails if drafted prose names an action, a contact, or a deadline that the catalog did not already authorize.

Why generated clarity is not an operational contract

Clear wording helps the person who is paged, yet clarity does not prove that a suggested step is safe to run. A model can restate a log event in plainer English and still invent a rollback that nobody on the team approved. The useful split is mechanical: compile duties from source, draft symptoms from those duties, and require a human signature on every risk change. Chat transcripts may inform a hint, but they never become the duty record that the published page is allowed to cite.

What the model may draft, and what a human must own

The table is the ownership artifact, and every later script exists to enforce one row of it. Narrative fields may be rewritten for readability, while duty fields may be copied but must not be paraphrased into a new promise. If a field can change who is woken up, what a customer is told, or which command runs, it stays in the duty lane. If a field only restates an event that the catalog already names, a model may draft it and an editor may still reject it.

Field	Lane	Who may change it	Failure if the lane is ignored
Alert id	Duty	Catalog author	The page points at the wrong signal
Log event	Duty	Catalog author	Symptom text cites a missing event
Symptom draft	Narrative	Model, then editor	Prose drifts from the event list
Safe checks	Duty	Human operator	A reader runs an unapproved command
Page target	Duty	Human owner	The alert routes to a stale rotation
Customer impact	Duty	Human owner	Copy overstates or hides harm
Escalation contact	Duty	Human owner	A model invents a name or channel

Step 1: Compile the alert catalog into a duty record

Store each alert in a small YAML catalog that a human already reviews before the file reaches the default branch. The compiler reads that catalog and writes a JSON duty record containing only the fixed key set listed below. It does not call a model, and it refuses to invent an owner when a required key is absent. Missing keys stop the build, because a partial record would look complete to a later drafting step.

# alerts/catalog.yaml — reviewed source, not model output
alerts:
  - id: queue_depth_high
    log_event: queue.depth.exceeded
    page_target: payments-oncall
    severity: page
    safe_checks:
      - name: read_depth
        command: "metrics get queue.depth --window 15m"
    customer_impact: "Checkout may delay; no automatic refund."
    escalation: payments-primary
    narrative_hint: "Depth stayed above the configured threshold."

# tools/compile_runbook_duties.py
# Requires PyYAML. Proposed, unexecuted example. Copies duty fields and never writes prose.
import json
from pathlib import Path

import yaml

DUTY_KEYS = (
    "id",
    "log_event",
    "page_target",
    "severity",
    "safe_checks",
    "customer_impact",
    "escalation",
)

def compile_duties(catalog_path: Path) -> dict:
    catalog = yaml.safe_load(catalog_path.read_text())
    records = []
    for alert in catalog["alerts"]:
        missing = [key for key in DUTY_KEYS if key not in alert]
        if missing:
            raise SystemExit(f"{alert.get('id', '?')}: missing {missing}")
        records.append({key: alert[key] for key in DUTY_KEYS})
    return {"schema": "runbook-duty-v1", "alerts": records}

if __name__ == "__main__":
    record = compile_duties(Path("alerts/catalog.yaml"))
    Path("build/runbook-duties.json").write_text(json.dumps(record, indent=2) + "\n")

Run the compiler before any drafting step, and treat the JSON as a build product rather than as prose. Commit that JSON only when the catalog diff is intentional, so reviewers can see duty changes without reading generated paragraphs. The two commands below check that the duty record exists and that a standard parser accepts the JSON. Neither command needs network access, which keeps the duty lane reproducible wherever a normal Python interpreter is already installed.

python tools/compile_runbook_duties.py
python -m json.tool build/runbook-duties.json >/dev/null

Step 2: Build a draft envelope that omits duty fields

The draft envelope is the only document a model should receive when the task is to write symptom prose. It includes the alert identifier, the log event name, and a short hint that a human already wrote. It excludes page targets, escalation names, commands, and customer-impact sentences so those strings cannot be copied by accident. That omission is the control, because a prompt instruction alone is something a model can ignore or half-follow.

The envelope builder uses an allow-list, not a deny-list, so a newly added duty key stays out by default. Hints are optional and must not contain commands, since a hint is copied into the model context. If a hint repeats an escalation name, the leak starts before the model writes a single sentence. Reject hints in review with the same banned-token list that the later linter applies to drafts.

# tools/build_draft_envelope.py
# Proposed, unexecuted example. Writable keys are an allow-list.
import json
from pathlib import Path

import yaml

def build_envelope(catalog_path: Path) -> dict:
    catalog = yaml.safe_load(catalog_path.read_text())
    packets = []
    for alert in catalog["alerts"]:
        packets.append({
            "id": alert["id"],
            "log_event": alert["log_event"],
            "hint": alert.get("narrative_hint", ""),
            "task": "Draft one symptom paragraph. Do not name owners, commands, or impact.",
        })
    return {"schema": "runbook-draft-envelope-v1", "packets": packets}

if __name__ == "__main__":
    envelope = build_envelope(Path("alerts/catalog.yaml"))
    Path("build/draft-envelope.json").write_text(json.dumps(envelope, indent=2) + "\n")

Step 3: Draft symptom prose, then stop at the envelope boundary

Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode's free model access fits this drafting step because the input is a small envelope rather than the whole repository. The free server option can run the compiler and the later linter if that job should not sit on a laptop. This article does not name a model, a quota, a machine size, or a retention period, because those facts were not supplied here.

Read the current product surface before you depend on availability, since a free tier can change outside this text. Send only the envelope file, and ask for one symptom paragraph that keeps each alert identifier unchanged. Require the log event string to appear exactly as compiled, with no added flags or endpoints. Reject the reply if it contains a command, a handle, or any sentence copied from customer impact.

python tools/build_draft_envelope.py
# Proposed handoff: upload build/draft-envelope.json only.
# Save the model reply as build/symptom-drafts.json using the shape in Step 4.
python tools/lint_symptom_drafts.py build/symptom-drafts.json build/runbook-duties.json

Step 4: Lint drafts against the duty record

The linter is the reproducible gate between a draft and a page that others might follow at night. It checks that every draft identifier exists in the duty record, so orphan prose cannot ship. It checks that the compiled log event appears verbatim, which ties the paragraph to a real signal. It checks that page targets, escalation names, commands, and impact sentences do not occur inside the draft.

A failing exit status means the prose lane leaked into the duty lane, so the page is not publishable yet. The script prints one line per failure and returns a non-zero status so a continuous-integration job can block the merge. Exact string matching will miss a paraphrase, which is why the human signature step remains mandatory after a green run. Do not weaken the linter to accept near matches, because near matches hide the leak you are trying to see.

# tools/lint_symptom_drafts.py
# Proposed, unexecuted example. Exit status is the review gate.
import json
import sys
from pathlib import Path

def lint(drafts_path: Path, duties_path: Path) -> list[str]:
    drafts = json.loads(drafts_path.read_text())["drafts"]
    duties = {
        row["id"]: row for row in json.loads(duties_path.read_text())["alerts"]
    }
    errors = []
    for draft in drafts:
        duty = duties.get(draft["id"])
        if duty is None:
            errors.append(f"{draft['id']}: not in duty record")
            continue
        text = draft["symptom"]
        if duty["log_event"] not in text:
            errors.append(f"{draft['id']}: missing log event")
        banned = [
            duty["page_target"],
            duty["escalation"],
            duty["customer_impact"],
        ]
        banned.extend(check["command"] for check in duty["safe_checks"])
        for token in banned:
            if token and token in text:
                errors.append(f"{draft['id']}: leaked duty token")
    return errors

if __name__ == "__main__":
    found = lint(Path(sys.argv[1]), Path(sys.argv[2]))
    for item in found:
        print(item)
    raise SystemExit(1 if found else 0)

How to read a red run

Use a tiny fixture before you trust this gate on a real catalog of production alerts. The bad draft below omits the log event and copies the escalation contact, so both rules should fire. The good draft names the compiled event and avoids every banned token that the duty record already lists. Save each fixture, run the linter, and compare the exit status before you wire the job into CI.

Draft fragment	Rule	Expected result
No queue.depth.exceeded	Event must appear verbatim	Fail
Contains payments-primary	Duty token is banned	Fail
Copies the impact sentence	Duty token is banned	Fail
Event present, no banned token	Narrative lane only	Pass

{
  "drafts": [
    {
      "id": "queue_depth_high",
      "symptom": "Page payments-primary if checkout slows."
    }
  ]
}

queue_depth_high: missing log event
queue_depth_high: leaked duty token

{
  "drafts": [
    {
      "id": "queue_depth_high",
      "symptom": "The event queue.depth.exceeded means depth stayed above the configured threshold."
    }
  ]
}

Step 5: Assemble the page only after a human signs

Assemble the page only after a named human signs the duty block in the same change as the prose. The renderer copies duty fields verbatim and appends the lint-clean symptom draft under a heading that marks it as explanation. It does not ask the model to merge the sections, because a merge is where invented escalation usually appears. Safe checks stay in a fenced list taken from the catalog, with no rewritten flags and no extra arguments.

1. Copy page target, severity, escalation, and customer impact from the duty record into a signature block that humans diff.
2. Paste the lint-clean symptom paragraph under a heading that says the text is explanatory and not an instruction.
3. Render each safe check as a fenced command taken from the catalog, without synonyms or added shell flags.
4. Require the signing human to be named in the commit, so a later reader can see who accepted the route.
5. Re-run the linter in CI so a later edit cannot smuggle a command back into the symptom section.

## queue_depth_high

**Severity:** page
**Page target:** payments-oncall
**Escalation:** payments-primary
**Customer impact:** Checkout may delay; no automatic refund.

### Symptom (narrative lane)
The event `queue.depth.exceeded` means depth stayed above the configured threshold.

### Checks (duty lane)
- read_depth: `metrics get queue.depth --window 15m`

Limitations of the string gate

The linter catches exact token leaks, not clever paraphrases of an escalation path or a customer promise. A draft can say to call the payments rotation without copying the catalog contact, and the string check will pass. Human review of the signature block is still required, and the catalog must already describe the live route. This workflow assumes one alert maps to one page target, and it does not model follow-the-sun rotations.

Temporary freezes, vendor status pages, and legal hold notices sit outside the schema shown in this example. If the YAML catalog is stale, the compiler will faithfully publish stale duties and the linter will still pass. The gate proves that lanes stayed separate, not that the underlying alert definitions are still fresh. Refresh ownership on a schedule you already trust, and do not treat a green linter as a paging drill.

Who should not use this approach

Skip this flow when the on-call source of truth cannot be exported, because a partial catalog will look complete. Skip it when remediation is customer-specific and cannot be listed as a closed set of commands. Skip it when counsel must approve every published sentence, since a narrative lane would create unreviewed copy. A team editing one static page will spend more time maintaining the compiler than maintaining the page.

Also skip the model step when the envelope would include secrets, customer names, or raw production logs. Free drafting access does not change the rule that sensitive operational payloads must stay out of the prompt. If you cannot redact the hint, stop at the duty record and write the symptom paragraph yourself. The compiler and the linter remain useful in that case, because they do not need a model at all.

Start from the catalog, not from the draft

If alerts already live in a reviewed file, compile the duty record before you ask any model for prose. A free drafting pass is relevant only after the envelope exists and banned tokens are enforced by code. Confirm current availability on the product surface you actually use, then keep duty fields out of the prompt. The page is ready when the linter is green and a human has signed the escalation block, not when the draft merely sounds clear.