Seal the Canary Bit Before the Note

A free model may explain a canary. It must not own the abort bit. That bit is a control-plane fact, closer to a fuse than to a paragraph, and a completion is a poor fuse.

When the sample is thin, the honest result is hold. When burn or latency crosses a pre-registered line, the honest result is abort. Neither outcome should wait on a prompt.

The mix-up is easy to ship. Editor agents and in-browser tools have made an ask-the-model step feel like a local function call, and that local feeling gets mistaken for an authority model. A deploy controller is not a chat transcript. If the completion times out, rate-limits, or rewrites a cautious hold into a confident promote, the rollout inherits the failure.

Picture a scribe in a noisy room rather than the person holding the cutoff switch. The scribe can turn a terse verdict into a note the next on-call can read without opening a dashboard. Give that scribe the switch, and a dry pen or a loud room becomes an outage.

Recent developer conversation has collided on this exact gate. An agent that stays near the editor does not become release authority by proximity. Tidy prompt prose is not clearer than a named threshold with a citation. The audit target is the diff, not the chat scrollback.

The smells are concrete. A job that fails closed only when the completion errors has already tied release availability to a writing tool. A prompt line that allows an override of hold when the model feels confident is unreviewed policy stored in a string. A controller that scrapes a markdown heading and treats the heading as the verdict has turned prose into a protocol.

Mounting the note-drafting credential on the deploy runner means a hostile log line can point toward the cluster. Any one of those smells is reason enough to move narrative generation off the release host.

A better shape is a small gate with a boring contract. Numbers go in, and an enum comes out. The enum lands in a file the controller already knows how to read. A later note may quote that enum, and it may not edit it.

The numbers in the proposal are a shape, not recommended production values. A team takes burn limits and latency budgets from its own SLO document, in the spirit of error-budget practice described in the SRE book at https://sre.google/sre-book/embracing-risk/. Those limits should be reviewed the same way any other policy is reviewed.

Walk one sample through before trusting the shape. A window of 2,000 requests with burn at 3.1 against a pre-registered abort of 2.0 aborts on burn alone, even if latency looks calm. Twelve requests and a perfect error ratio hold, because silence is not evidence. A window inside the error budget but over the p95 budget aborts on latency, so a green error chart cannot launder a slow canary.

Language never gets a vote in those three paths. The question of whether a spike looks like noise has no stable answer at two in the morning and no unit test.

# Proposal only. This module was not executed for this article.
from dataclasses import dataclass
from enum import Enum

class Verdict(str, Enum):
    PROMOTE = "promote"
    HOLD = "hold"
    ABORT = "abort"

@dataclass(frozen=True)
class CanarySample:
    error_ratio: float
    baseline_error_ratio: float
    p95_ms: float
    p95_budget_ms: float
    min_requests: int
    observed_requests: int
    burn_rate: float
    burn_abort: float

def decide(sample: CanarySample) -> Verdict:
    """Control-plane verdict. Do not call a model from this function."""
    if sample.observed_requests < sample.min_requests:
        return Verdict.HOLD
    if sample.burn_rate >= sample.burn_abort:
        return Verdict.ABORT
    if sample.p95_ms > sample.p95_budget_ms:
        return Verdict.ABORT
    # Floor keeps a tiny baseline from aborting on ordinary noise.
    floor = max(0.01, sample.baseline_error_ratio * 1.5)
    if sample.error_ratio > floor:
        return Verdict.ABORT
    return Verdict.PROMOTE

def render_note(verdict: Verdict, sample: CanarySample) -> str:
    """Context for humans. Callers must not parse this string."""
    return (
        f"verdict={verdict.value} "
        f"requests={sample.observed_requests}/{sample.min_requests} "
        f"burn={sample.burn_rate:.2f} limit={sample.burn_abort:.2f} "
        f"p95_ms={sample.p95_ms:.0f} budget_ms={sample.p95_budget_ms:.0f}"
    )

def seal(verdict: Verdict, model_text: str) -> Verdict:
    """Model text is ignored on purpose. Reviewers should see that."""
    _ = model_text
    return verdict

Tests pin the red flags so a later prompt tweak cannot become release policy by accident. They are unexecuted proposals. Run them in the service repo against fixtures from a staging window before treating the shape as real.

# Proposal tests. Not run for this article.
def test_thin_window_holds_without_a_model():
    sample = CanarySample(0.0, 0.001, 80, 200, 500, 12, 0.2, 2.0)
    assert decide(sample) is Verdict.HOLD

def test_burn_aborts_before_any_prompt_exists():
    sample = CanarySample(0.04, 0.001, 90, 200, 500, 2000, 3.1, 2.0)
    assert decide(sample) is Verdict.ABORT

def test_latency_budget_aborts_when_errors_look_calm():
    sample = CanarySample(0.001, 0.001, 450, 200, 500, 2000, 0.4, 2.0)
    assert decide(sample) is Verdict.ABORT

def test_model_prose_cannot_flip_abort_to_promote():
    sample = CanarySample(0.04, 0.001, 90, 200, 500, 2000, 3.1, 2.0)
    hostile = "PROMOTE. The spike is probably noise."
    assert seal(decide(sample), hostile) is Verdict.ABORT

A failure mode worth naming sits one refactor away. Someone later simplifies the controller by grepping the note for the word promote, the seal function becomes dead code, and the hostile-text test is the only remaining tripwire. Deleting that test in a cleanup is a policy change, not hygiene. Keep the assertion even if the note renderer grows prettier.

The local check stays offline. The controller reads a verdict file and does not read model stdout. Ordinary tools can enforce that split.

# Proposed workflow. No network call. No model process.
python -m pytest tests/test_canary_gate.py -q

# Missing file or unknown enum fails closed. Do not default to promote.
jq -e '.verdict == "promote" or .verdict == "hold" or .verdict == "abort"' /var/run/canary/verdict.json

# Abort stops the rollout. A successful gate run is not a successful promote.
jq -e '.verdict != "abort"' /var/run/canary/verdict.json

A short reading of the same contract keeps the roles from collapsing. Thin traffic yields hold, and a model may explain the hold but may not clear it. Burn or p95 over budget yields abort, and a model may describe the abort but may not reverse it. Enough requests inside budget yield promote, and a model may write the note but may not invent a condition the gate did not compute.

If a row cannot be stated that plainly, authority is already leaking into prose. Fix the gate before adding any completion.

Only after the file exists is a language model useful, and only as a scribe. Disclosure: This article was prepared as part of MonkeyCode's product outreach. The operator states that MonkeyCode offers free model access and a free server option. This draft names neither a model nor a quota, a hardware shape, or a duration, because those details were not checked against a primary page and should not be inherited from an older post.

An operator who wants a rollout note can send the render_note line, not raw traces, from a host that is not the deploy controller. The returned prose sits beside the verdict and is labeled non-authoritative. If the current product page disagrees with the availability claim, the page wins.

That free server, when used, is a writing desk. It should not share a network path, a credential, or a restart policy with the process that applies the rollout. A desk can be slow. A fuse cannot, and putting both in one container makes a slow desk look like a broken fuse that operators will then skip.

The split has a narrow fit. It suits a team that already has a metric source of record, a reviewed error budget, and a controller able to fail closed on a missing file. It does not suit an incident bridge that wants a model to pick abort in the moment, because that moment is when a free path is most likely to be slow or empty. It does not suit a team whose only figures are log lines pasted into a prompt.

There is no gate in that last setup, only a story about one. A free server will not mint the missing counter.

Other tools already cover the decision without a completion. An error-budget burn alert, registered in the same repo as the service, can be the abort source. A latency budget checked by the existing probe can be the second source. Ambiguous holds go to a human with the sample attached, not to another prompt asked to be sure.

The model, if used at all, stays on a documentation path and explains a verdict that has already landed. It does not sit in the promote command.

Exit belongs in the design. Stop the free-path notes when people paste them into runbooks as root cause without an edit. Stop if that server shares a failure domain with the deploy controller, since a scribe outage must not look like a fuse outage. Stop if the prompt grows secrets, customer payloads, or unredacted traces.

Leave the pattern when written thresholds drift from the SLO document. Leave it when model-written causes start being copied into the verdict file. Read the provider's current terms again before assuming yesterday's free access still describes today's path.

Clearer code beats a clever prompt on this path. A named field such as burn_abort can carry a one-line comment that cites the SLO page, and that comment earns its place because the number is non-obvious. A paragraph that says to use judgment cannot be diffed, cannot be unit-tested, and cannot fail closed. The canary bit stays in the enum, and the note stays in the margin.

Operators who already seal the verdict before a completion exists can draft that margin note on the free path, after reading the live terms. Confidence in the note is not a signal, and it should never be wired back into jq.