Incident Severity Does Not Belong on Free Inference

Production paging is a control plane. Free inference is a best-effort text service. Those two jobs do not share a fate, and the mismatch shows up only after a quiet night that should not have been quiet.

Severity, rotation, and silence belong in deterministic policy. A model may help a human write the timeline after the page has already fired. It must not choose whether the page fires.

The temptation is obvious. Alert text is messy. Dashboards disagree. On-call wants a sentence that says P1 or P3. A chat completion looks like a judge. It is not a judge. It is a narrator with variable latency, no fencing token, and no duty to stay within the enumerated severities the runbook actually supports.

Think of the pager as a fuse box. A fuse opens on current, not on a paragraph about current. Free inference is a whiteboard in a side room. Useful after the circuit is already open. Dangerous as the metal strip that is supposed to melt.

Red flags in the paging path

The first red flag is a hop from unstructured logs to a page that only exists if a model returns a label. That hop hides a second failure mode: the model can stall, refuse, or answer in prose. MTTA then includes token generation. The clock that matters to a customer does not.

The second red flag is letting the model invent a severity the policy file does not name. SEV1, SEV2, SEV3, and SEVNONE are a closed set. "Looks degraded but maybe regional" is not a member of that set. A narrator will happily mint a fifth state. A fuse cannot.

The third red flag is using free inference as a tie-breaker when two signals conflict. Conflict is a reason to page the humans who own the service, not a reason to ask a remote completion which signal "feels" louder. Ambiguity is itself a SEV2 in many shops. Pretending it is a language problem does not make it smaller.

The fourth red flag is feeding the model customer content, auth headers, or raw exception payloads so it can "understand impact." Impact should already be a boolean or a counter from the edge. If the estate cannot compute customer_facing_5xx without a paragraph, the estate is not ready for an LLM in the path. It is ready for better instrumentation.

None of this forbids language models nearby. It forbids them as the authority that maps telemetry to who wakes up.

A local oracle that does not chat

The artifact below is a closed policy. It reads structured signals only. Unknown keys fail closed. Tests pin the mapping. This is a proposal for a local gate, not a claim about any particular outage.

# severity_policy.py
from __future__ import annotations

from dataclasses import dataclass
from enum import Enum

class Severity(str, Enum):
    SEV1 = "SEV1"
    SEV2 = "SEV2"
    SEV3 = "SEV3"
    SEVNONE = "SEVNONE"

ALLOWED = frozenset(s.value for s in Severity)

@dataclass(frozen=True)
class Signals:
    heartbeat_lost_s: int
    edge_5xx_ratio: float
    customer_facing: bool
    burn_rate_1h: float
    manual_override: str | None = None

class PolicyError(ValueError):
    pass

def decide(signals: Signals) -> Severity:
    if signals.manual_override is not None:
        if signals.manual_override not in ALLOWED:
            raise PolicyError(f"unknown override {signals.manual_override!r}")
        return Severity(signals.manual_override)

    if signals.heartbeat_lost_s < 0 or signals.edge_5xx_ratio < 0 or signals.burn_rate_1h < 0:
        raise PolicyError("signals must be non-negative")

    if signals.customer_facing and (
        signals.heartbeat_lost_s >= 60 or signals.edge_5xx_ratio >= 0.05
    ):
        return Severity.SEV1

    if signals.burn_rate_1h >= 14.4 or signals.edge_5xx_ratio >= 0.02:
        return Severity.SEV2

    if signals.heartbeat_lost_s >= 20 or signals.edge_5xx_ratio >= 0.005:
        return Severity.SEV3

    return Severity.SEVNONE

The numbers are placeholders a team must replace with its own SLO math. The shape is the point. Heartbeat loss and edge error ratio are facts. Burn rate is a fact if the SLO library already computed it. None of those facts require a completion API.

# test_severity_policy.py
from severity_policy import PolicyError, Severity, Signals, decide
import pytest

def test_customer_facing_heartbeat_is_sev1():
    s = Signals(heartbeat_lost_s=60, edge_5xx_ratio=0.0, customer_facing=True, burn_rate_1h=1.0)
    assert decide(s) is Severity.SEV1

def test_high_burn_is_sev2_without_chat():
    s = Signals(heartbeat_lost_s=0, edge_5xx_ratio=0.0, customer_facing=False, burn_rate_1h=14.4)
    assert decide(s) is Severity.SEV2

def test_unknown_override_fails_closed():
    s = Signals(
        heartbeat_lost_s=0,
        edge_5xx_ratio=0.0,
        customer_facing=False,
        burn_rate_1h=0.0,
        manual_override="looks-ok-ish",
    )
    with pytest.raises(PolicyError):
        decide(s)

def test_negative_signal_is_not_a_prompt():
    s = Signals(heartbeat_lost_s=-1, edge_5xx_ratio=0.0, customer_facing=False, burn_rate_1h=0.0)
    with pytest.raises(PolicyError):
        decide(s)

Run the pins before wiring the function to any notifier.

python -m pytest test_severity_policy.py -q

A notifier should accept only Severity members. If a future refactor wraps decide() with a model "second opinion," the tests above will still pass while production quietly waits on a socket. Guard that with an explicit ban in the caller.

# page.py
FORBIDDEN_IMPORTS = ("openai", "anthropic", "litellm")

def assert_no_inference_in_paging_path(source: str) -> None:
    lowered = source.lower()
    for name in FORBIDDEN_IMPORTS:
        if name in lowered:
            raise RuntimeError(f"paging path imports {name}; severity must stay local")

That check is crude. It is also honest. Import graphs lie less often than prompt templates.

Where a free model is allowed to stand

After the incident is sealed, a narrator is useful. The page already happened. Severity is already in the ticket. The remaining job is a draft humans will edit: timeline, systems touched, open questions. That draft can miss a sentence without missing a pager.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode's free model access and free server option are relevant only for that sealed, after-the-fact draft. They are operator-supplied availability claims, not a paging backend, not an SLO, and not a substitute for the policy file above.

A safe split looks like this. The worker that pages never opens an inference client. A second worker, triggered only when incident.state == "sealed", may send a redacted timeline. Redaction happens before the prompt, with the same closed-set discipline.

# draft_timeline.py
ALLOWED_STATES = frozenset({"sealed"})

def build_draft_payload(incident: dict) -> dict:
    if incident.get("state") not in ALLOWED_STATES:
        raise RuntimeError("refuse to draft while paging is still live")
    return {
        "severity": incident["severity"],  # already decided
        "started_at": incident["started_at"],
        "ended_at": incident["ended_at"],
        "systems": incident.get("systems", []),
        "notes": incident.get("public_notes", ""),
    }

If a team wants to try that second worker in a free coding environment, that is a documentation chore. It is not on-call infrastructure. Keep the two codepaths in different deployables so a prompt change cannot ride along with a threshold change.

Better alternatives than a chat completion

Structured SLO burn from the metrics backend already answers "is this fast enough to wake someone." Customer-facing probes already answer "can a user complete the money path." Heartbeats already answer "is the process alive." Combining those with a small policy module is boring. Boring is the correct texture for a fuse.

When signals conflict, page at the higher severity and let a human downgrade. Downgrades are cheap compared with a missed SEV1. Manual override remains in the policy, as a closed enum, because humans also hallucinate, but they do it on a recorded bridge.

When the estate lacks those signals, the work is instrumentation, not prompt design. A model will not invent a trustworthy edge_5xx_ratio from a pile of stack traces. It will produce a confident paragraph. Confidence is not a counter.

Exit criteria

Leave the free-inference paging experiment when any of these become true. A page is delayed by model latency or rate limits. A completion returns a severity string the policy file cannot parse. On-call cannot explain a decision without pasting a prompt. Customer data entered a vendor log to "improve classification." A silent failure in the model path equals a silent production failure.

Exit is mechanical. Remove the client from the paging deployable. Restore decide() as the only mapper. Keep the draft worker if it still helps, behind the sealed gate. If the draft worker cannot prove that gate in a test, delete the worker too.

def test_draft_refuses_live_incident():
    from draft_timeline import build_draft_payload
    import pytest
    with pytest.raises(RuntimeError):
        build_draft_payload({"state": "firing", "severity": "SEV1"})

Limitations and who should not use this

Deterministic policy is only as good as the signals. A wrong threshold pages too often or too rarely. This article does not offer vendor SLOs, model names, token quotas, or hardware claims. It does not assert that any free server is available in a particular region or at a particular time. Teams still need an override path and a human commander.

Do not use a chat completion as the severity authority if paging is tied to a contractual SLA, if the on-call rotation is small, or if the product handles money, health, or safety. Do not use this field guide as permission to skip metrics. A fuse box with no current sensor is just a box.

The industry keeps wrapping narrators around APIs and calling the result an agent. Tool calling is a fine way to fetch a weather JSON. It is a poor way to decide who loses sleep. Keep the narrator in the side room. Keep the fuse in the wall.