Reject the Packet: Four Gates Between an Alert and a Mutating Command

An on-call helper that fills blank fields with a plausible story is not assisting; it is inventing the incident. I want every alert to travel as a packet with four gates, and I want a local validator to reject that packet before anyone types a restart. What happens if the freeze latch is unset? The bot should stop, not improvise. The rest of this article is the contract, the checks, and the commands I actually keep next to the pager.

This is a proposal and a working sketch, not a war story from a named company. I am not claiming minutes saved, or a production rollout you should copy blindly. If you already sealed first commands, modeled escalation as a graph, or wired a model to webhooks, treat this as a different control plane: empty fields are a page, not a prompt.

The failure I am trying to kill

Most agent write-ups still assume the model will “reason” about a pager dump and then suggest a fix. Have you watched that loop during a noisy night? The model collapses three alerts into one root cause, skips the read-only checks, and talks itself into a restart because restarts are linguistically cheap. That is not architecture. That is autocomplete with sudo nearby.

I want the opposite default. The helper may draft text, but it may not advance the incident until four gates contain evidence a script can parse. Humans still decide. The packet only proves that nobody skipped the boring parts.

Four gates, one packet

Here is the whole policy in one list. If any gate is empty, the validator exits non-zero and the mutating command list stays dark.

1. Alert identity. Fingerprint, service, severity, and whether this page is unique or a duplicate burst.
2. First commands. A fixed, read-only set, plus exit codes and a short hash of stdout.
3. Escalation evidence. What you observed, what you ruled out, and who you need if you cannot continue.
4. Freeze latch. frozen or thawed, with actor, ticket, and expiry. Mutations are illegal while frozen.

Notice what is missing on purpose. There is no free-text “root cause” field that can satisfy the schema by sounding confident. Root cause, if you write it at all, is a comment after the gates pass.

Artifact: the incident packet

Save this as incident.schema.json. It is intentionally small so a modest model can fill it without chewing the runbook itself.

{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "title": "IncidentPacket",
  "type": "object",
  "additionalProperties": false,
  "required": ["alert", "first_commands", "escalation", "freeze"],
  "properties": {
    "alert": {
      "type": "object",
      "additionalProperties": false,
      "required": ["fingerprint", "service", "severity", "page_worthy", "received_at"],
      "properties": {
        "fingerprint": { "type": "string", "minLength": 8 },
        "service": { "type": "string", "minLength": 1 },
        "severity": { "enum": ["info", "warn", "error", "critical"] },
        "page_worthy": { "type": "boolean" },
        "received_at": { "type": "string", "format": "date-time" },
        "duplicate_of": { "type": ["string", "null"] }
      }
    },
    "first_commands": {
      "type": "array",
      "minItems": 1,
      "items": {
        "type": "object",
        "additionalProperties": false,
        "required": ["id", "argv", "exit_code", "stdout_sha256", "readonly"],
        "properties": {
          "id": { "type": "string" },
          "argv": { "type": "array", "items": { "type": "string" }, "minItems": 1 },
          "exit_code": { "type": "integer", "minimum": 0 },
          "stdout_sha256": { "type": "string", "pattern": "^[a-f0-9]{64}$" },
          "readonly": { "const": true }
        }
      }
    },
    "escalation": {
      "type": "object",
      "additionalProperties": false,
      "required": ["status", "observed", "ruled_out", "need_from_human"],
      "properties": {
        "status": { "enum": ["stay", "ask", "page_owner", "page_secondary"] },
        "observed": { "type": "array", "items": { "type": "string" }, "minItems": 1 },
        "ruled_out": { "type": "array", "items": { "type": "string" } },
        "need_from_human": { "type": "string", "minLength": 8 }
      }
    },
    "freeze": {
      "type": "object",
      "additionalProperties": false,
      "required": ["state", "actor", "ticket", "expires_at"],
      "properties": {
        "state": { "enum": ["frozen", "thawed"] },
        "actor": { "type": "string", "minLength": 1 },
        "ticket": { "type": "string", "minLength": 3 },
        "expires_at": { "type": "string", "format": "date-time" },
        "reason": { "type": "string" }
      }
    }
  }
}

Why additionalProperties: false? Because that is how you stop a model from inventing root_cause_confidence and then treating the number as permission. If the field is not in the schema, it is not in the incident.

Artifact: reject empty gates in Python

The schema catches shape. The script below catches policy. Run it on the box that can see logs, not inside the chat transcript.

#!/usr/bin/env python3
"""Reject an incident packet that is missing evidence. Proposal / local helper."""
from __future__ import annotations

import hashlib
import json
import subprocess
import sys
from datetime import datetime, timezone
from pathlib import Path

ALLOWED_FIRST = {
    "pods_not_running": ["kubectl", "get", "pods", "-n", "payments",
                          "--field-selector=status.phase!=Running"],
    "unit_tail": ["journalctl", "-u", "api", "-n", "80", "--no-pager"],
    "prom_alerts": ["curl", "-sS", "http://127.0.0.1:9090/api/v1/alerts"],
}

MUTATING_HINTS = ("restart", "delete", "scale", "rollout", "drain", "reboot")


def sha256_bytes(data: bytes) -> str:
    return hashlib.sha256(data).hexdigest()


def parse_ts(value: str) -> datetime:
    return datetime.fromisoformat(value.replace("Z", "+00:00"))


def reject(msg: str) -> int:
    print(f"REJECT: {msg}", file=sys.stderr)
    return 2


def main(path: Path) -> int:
    packet = json.loads(path.read_text())
    alert = packet["alert"]
    if not alert["page_worthy"] and alert["severity"] == "critical":
        return reject("critical alert marked not page-worthy")
    if alert.get("duplicate_of") and alert["page_worthy"]:
        return reject("duplicate fingerprint should not page again")

    seen = []
    for step in packet["first_commands"]:
        if step["id"] not in ALLOWED_FIRST:
            return reject(f"command id not in allowlist: {step['id']}")
        if step["argv"] != ALLOWED_FIRST[step["id"]]:
            return reject(f"argv drift for {step['id']}")
        if not step["readonly"]:
            return reject("first commands must be readonly")
        joined = " ".join(step["argv"]).lower()
        if any(h in joined for h in MUTATING_HINTS):
            return reject(f"mutating hint in first command: {joined}")
        seen.append(step["id"])
    if set(seen) != set(ALLOWED_FIRST):
        return reject("first-command set is incomplete")

    esc = packet["escalation"]
    if esc["status"] != "stay" and len(esc["need_from_human"].split()) < 3:
        return reject("escalation needs a real question for a human")

    freeze = packet["freeze"]
    now = datetime.now(timezone.utc)
    if parse_ts(freeze["expires_at"]) <= now and freeze["state"] == "frozen":
        return reject("freeze expired; set a new latch, do not imply thaw")
    if freeze["state"] == "frozen":
        print("OK: packet valid; mutations remain blocked by freeze latch")
        return 0

    print("OK: packet valid; freeze is thawed, human may consider mutations")
    return 0


if __name__ == "__main__":
    sys.exit(main(Path(sys.argv[1])))

I keep the allowlist in the script, not in the model prompt. Can a chat window “remember” the freeze rule at 03:00? Sometimes. Can it silently drop the rule when the alert dump is huge? Yes, and that is why the reject path lives in Python.

First commands I actually want hashed

These are examples you should rewrite for your service. They are read-only on purpose. Run them, hash stdout, and paste the hashes into the packet. Do not let the model pick a different kubectl verb because it “looks equivalent.”

# Gate 2 candidates. Read-only. Hash the output, do not narrate it.
kubectl get pods -n payments --field-selector=status.phase!=Running
journalctl -u api -n 80 --no-pager
curl -sS http://127.0.0.1:9090/api/v1/alerts | jq -c '.data.alerts[] | {alertname,state,labels}'

A tiny helper for the hash field:

sha256sum <<'EOF'
$(kubectl get pods -n payments --field-selector=status.phase!=Running)
EOF

If the command is not in ALLOWED_FIRST, the packet is invalid even when the model writes a beautiful paragraph. That is the point. Beauty is not evidence.

Escalation is a question, not a phone tree

I already dislike phone trees, and I still do not want a hidden state machine in this article. The packet only asks: are you staying, asking, paging the owner, or paging secondary? Then it demands a question a human can answer in one glance.

Useful need_from_human lines look like this:

• “Owner: is payments-api allowed to shed load, or is freeze still on?”
• “Secondary: replica count is 3/3 but p99 is 2.4s; do we open the error budget?”
• “Stay: stdout hashes match last week’s known noise; not paging.”

Useless lines look like “please advise” and “might be Kubernetes.” If the validator sees a short shrug, it rejects. Why should a secondary wake up for a shrug?

Freeze is a latch, unfreeze is a write

A freeze window on a wiki is a reminder. A latch in the packet is a bit that mutations must read. I want three boring fields: who moved the bit, which ticket allowed it, and when it expires. When the expiry passes, the script does not auto-thaw. Auto-thaw is how a holiday freeze dies at midnight while the incident is still live.

Unfreeze is its own write. Something like:

{
  "state": "thawed",
  "actor": "pager-primary",
  "ticket": "CHG-1842",
  "expires_at": "2026-09-05T16:00:00+00:00",
  "reason": "error budget policy allows a single restart after hashes collected"
}

Until that object exists, restart, rollout, and friends stay off the table. Does your current bot treat “we should bounce it” as a plan? Then you do not have a latch. You have a suggestion.

Where a free model actually helps

The model’s job is to draft the packet from the alert text and the hashed command output, not to SSH. I would rather a small model fail the schema than a large one narrate a fictional outage. After the validator prints OK, a human still types the mutating command, or a tightly wrapped script does, with the packet id in the audit log.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode’s free model access and free server option are a convenient place to host this draft-and-validate loop when you do not want the helper sitting on the production API node. That is the whole product claim I am using here: a free model to fill JSON, and a free server to run the reject script beside it. I am not attaching model names, quotas, hardware, or benchmarks, because those would be guesses.

If you try that loop, keep secrets off that box. The packet should contain hashes and fingerprints, not tokens and customer payloads.

A walk-through with a fake but complete packet

Label this as an unexecuted example. Suppose payments-api pages on HighP99 at 2026-09-05T02:11:00Z. You run the three first commands, fill hashes, and you are still inside a freeze.

{
  "alert": {
    "fingerprint": "highp99-payments-api-a9c31d22",
    "service": "payments-api",
    "severity": "error",
    "page_worthy": true,
    "received_at": "2026-09-05T02:11:00+00:00",
    "duplicate_of": null
  },
  "first_commands": [
    {
      "id": "pods_not_running",
      "argv": ["kubectl", "get", "pods", "-n", "payments", "--field-selector=status.phase!=Running"],
      "exit_code": 0,
      "stdout_sha256": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
      "readonly": true
    }
  ],
  "escalation": {
    "status": "ask",
    "observed": ["p99 elevated", "all pods Running"],
    "ruled_out": ["crash loop"],
    "need_from_human": "Owner: freeze still on, or is CHG-1842 valid to thaw?"
  },
  "freeze": {
    "state": "frozen",
    "actor": "sre-calendar",
    "ticket": "FRZ-77",
    "expires_at": "2026-09-05T12:00:00+00:00",
    "reason": "release freeze"
  }
}

This sample should fail, and I want it to fail. Why? The first-command set is incomplete, and the hashes are fake. Incomplete is not close enough. Feed the validator a partial packet during drills until the reject messages feel more trustworthy than the chat.

Limitations, loudly

This contract does not page people, talk to Slack, or roll a cluster. It will not save you if the allowlist is wrong, or if kubectl is already aliased to something cute. Clock skew can make expires_at lie, so NTP still matters. A determined human with production credentials can ignore the reject path, and a model can still write fluent nonsense into observed if you never read it.

Free models will also mis-classify severity. That is expected. The schema is there to make the miss visible, not to make the model wise. If your alerts include customer data, do not paste them into any third-party model, free or not.

Who should not use this

Skip this if you do not own the pager, or if your “runbook” is still a slide deck nobody can execute. Skip it if you want the model to hold cluster-admin and “just fix it.” Skip it in life-safety, payments cutover, or any environment where a third-party model is already forbidden. And skip it if you cannot name a human who is allowed to thaw the latch. A latch without an actor is theater.

What I would do on the next quiet afternoon

Write the schema. Put three read-only commands in the allowlist. Run the validator against a packet that is missing the freeze object and confirm you get REJECT. Then run it against a complete frozen packet and confirm mutations stay blocked. Only after those two failures look boring should you let a model draft JSON. If you need a scratch box for that drill, MonkeyCode’s free server option is one place to park the validator next to a free model, with the disclosure above still in mind.