DEV Community: Casey Sun

Free Model Lanes Need an Expiry Date: A Trial Ledger With Kill Criteria

Casey Sun — Tue, 20 Oct 2026 09:25:27 +0000

A summarizer starts as a demo. Someone points it at a free model lane. Three weeks later it serves internal tickets. Nobody decided that. The trial simply never ended.

This post is the guard for the moment before that happens. It is a field guide to not using a free model lane, plus a small ledger that makes the decision explicit. The scenario above is illustrative, not a customer story.

The failure mode: trials without an end date

Most free-lane problems are bookkeeping problems. There is no record of who opted in, or when the trial was supposed to end.

Three symptoms show up together:

The free lane has no owner and no expiry timestamp.
Nobody can state tokens spent per useful output.
The paid baseline disappeared from the config months ago.

A trial without an end date is a production dependency wearing a demo label.

Three red flags that mean stop now

1. An unattended path

If a cron job, webhook handler, or queue worker calls the lane, stop. Unattended callers turn a trial into uptime risk with no human in the loop.

2. Data you cannot describe in one sentence

If you cannot say what fields leave the process boundary, stop. Summaries of ticket text and summaries of customer records look identical in code.

3. No baseline lane to compare against

A free lane is only a deal relative to something. Without one paid or self-hosted baseline, the ledger measures nothing.

Better alternatives, per flag

Unattended path: run the trial behind a manual trigger and a review queue first.
Unclear data: redact at the call site, or keep the workload on infrastructure you control.
No baseline: route 10% of traffic to a second lane and log both.

The artifact: a trial ledger with a hard expiry

The ledger does two jobs. It records every request with a task tag, and it refuses to run after a date or budget you set.

Both limits below are your trial limits. They are not vendor quotas.

#!/usr/bin/env python3
"""trial_ledger.py - log free-lane requests and enforce a trial expiry. Stdlib only."""

import argparse
import json
import sqlite3
import sys
import time
from pathlib import Path

DDL = '''
CREATE TABLE IF NOT EXISTS events (
  ts REAL NOT NULL,
  task TEXT NOT NULL,
  prompt_tokens INTEGER NOT NULL,
  completion_tokens INTEGER NOT NULL,
  accepted INTEGER NOT NULL,
  latency_ms INTEGER NOT NULL,
  lane TEXT NOT NULL
);
'''

def connect(path):
    con = sqlite3.connect(path)
    con.executescript(DDL)
    return con

def cmd_record(args):
    con = connect(args.db)
    con.execute(
        'INSERT INTO events VALUES (?,?,?,?,?,?,?)',
        (time.time(), args.task, args.prompt_tokens, args.completion_tokens,
         1 if args.accepted else 0, args.latency_ms, args.lane),
    )
    con.commit()
    print('recorded task=' + args.task + ' accepted=' + str(args.accepted))

def p95(values):
    if not values:
        return 0
    values = sorted(values)
    return values[max(0, int(round(0.95 * (len(values) - 1))))]

def cmd_report(args):
    con = connect(args.db)
    rows = con.execute(
        'SELECT task, COUNT(*), SUM(accepted), SUM(prompt_tokens + completion_tokens) '
        'FROM events WHERE lane = ? GROUP BY task ORDER BY task',
        (args.lane,),
    ).fetchall()
    print('task'.ljust(16) + 'reqs'.rjust(6) + 'acc'.rjust(6) + 'rate'.rjust(7) + 'tok/acc'.rjust(10))
    for task, reqs, acc, toks in rows:
        rate = acc / reqs if reqs else 0.0
        per = str(round(toks / acc)) if acc else 'n/a'
        print(task.ljust(16) + str(reqs).rjust(6) + str(acc).rjust(6)
              + str(round(rate, 2)).rjust(7) + per.rjust(10))
    lat = [r[0] for r in con.execute(
        'SELECT latency_ms FROM events WHERE lane = ?', (args.lane,))]
    print('lane=' + args.lane + ' p95_latency_ms=' + str(p95(lat)))

def cmd_check(args):
    cfg = json.loads(Path(args.config).read_text())
    con = connect(args.db)
    total = con.execute(
        'SELECT COALESCE(SUM(prompt_tokens + completion_tokens), 0) '
        'FROM events WHERE lane = ?', (cfg['lane'],),
    ).fetchone()[0]
    problems = []
    if time.time() > cfg['expires_at']:
        problems.append('trial expired at ' + str(cfg['expires_at']))
    if total >= cfg['max_tokens']:
        problems.append('trial budget spent: ' + str(total) + '/' + str(cfg['max_tokens']))
    for p in problems:
        print('BLOCK: ' + p, file=sys.stderr)
    if problems:
        return 1
    print('OK: ' + str(total) + '/' + str(cfg['max_tokens']) + ' trial tokens used')
    return 0

def main():
    ap = argparse.ArgumentParser()
    sub = ap.add_subparsers(dest='cmd', required=True)

    rec = sub.add_parser('record')
    rec.add_argument('--db', required=True)
    rec.add_argument('--lane', required=True)
    rec.add_argument('--task', required=True)
    rec.add_argument('--prompt-tokens', type=int, required=True)
    rec.add_argument('--completion-tokens', type=int, required=True)
    rec.add_argument('--latency-ms', type=int, required=True)
    rec.add_argument('--accepted', action='store_true')
    rec.set_defaults(func=cmd_record)

    rep = sub.add_parser('report')
    rep.add_argument('--db', required=True)
    rep.add_argument('--lane', required=True)
    rep.set_defaults(func=cmd_report)

    chk = sub.add_parser('check')
    chk.add_argument('--db', required=True)
    chk.add_argument('--config', required=True)
    chk.set_defaults(func=cmd_check)

    args = ap.parse_args()
    sys.exit(args.func(args) or 0)

if __name__ == '__main__':
    main()

The config file that kills the trial

{"lane": "free-a", "expires_at": 1793664000, "max_tokens": 2000000}

1793664000 is 2026-11-03T00:00:00Z. Compute your own rather than trusting this one:

date -u -d "2026-11-03" +%s

Run it

python trial_ledger.py record --db trial.db --lane free-a --task summarize \
  --prompt-tokens 1200 --completion-tokens 220 --latency-ms 840 --accepted

python trial_ledger.py report --db trial.db --lane free-a
python trial_ledger.py check  --db trial.db --config lane.json

Wire the last command into CI, or into the wrapper that calls the lane:

- name: free lane trial guard
  run: python tools/trial_ledger.py check --db trial.db --config lane.json

A non-zero exit is the point. The trial dies on schedule, not on vibes.

Reading the ledger: tokens per accepted outcome

The tok/acc column is the number that matters. It divides total tokens by outputs a human actually accepted.

A lane returning 60% junk looks cheap per token and expensive per outcome. That inversion is the whole argument.

Two caveats apply. Acceptance is human-labeled, so the labels age. And retries count as separate rows unless you tag them.

Decision table: stay, fix, or walk

Thresholds below are illustrations. Calibrate against your baseline lane.

Signal from the ledger	Reading	Action
tok/acc within 20% of baseline	Free lane is competitive	Stay, keep the expiry
tok/acc 2x baseline, latency fine	Quality problem, not capacity	Fix prompts, or switch models on the same lane
tok/acc high and p95 latency growing	Queueing or rate limiting	Walk; move the task to the baseline lane
Fewer than 30 accepted samples	No usable signal	Extend once, then decide with numbers
Acceptance rate collapsing mid-trial	Model or input drift	Walk now, re-open as a new trial

Exit criteria to commit before the trial starts

Write these down before the first request. A trial with exit criteria is an experiment; without them it is a migration.

A fixed end date, stored in the config, not in a chat thread.
A token budget for the trial, set by you.
A minimum sample count before any decision.
A named baseline lane and a comparison metric.
A rollback path that does not require an incident.
A reviewer who is not the person who wanted the trial.

Where a free lane legitimately fits

Short-lived trials are exactly the case for a free model lane and a free server option. Low-stakes tasks, human-reviewed output, and a hard stop date make a free tier a reasonable probe.

MonkeyCode is an open-source project that offers a free model lane and a free server option. The operator states a free token grant (10 million tokens at the time of writing) and a free server tier. Treat both as availability claims, not SLAs, and check the current terms in the repository before planning around them.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

In this workflow the free server hosts the ledger and the guard check. The free lane is the subject of the trial, not the foundation of it.

Where the free lane does not fit

Anything with a latency SLO.
Anything with a retention clause you cannot satisfy.
Anything that must run when no human is watching.
Anything that needs guaranteed capacity next quarter.

Who should not use this approach

Teams with no baseline lane. Measure something first.
Workloads below roughly 30 tasks per week. Statistical noise swamps the table.
Teams that cannot tag tasks at the call site. The ledger needs a stable key.
Regulated data flows where the boundary itself is the risk.

Limitations

The ledger measures cost per accepted outcome, not correctness. It cannot see retries unless you tag them. It trusts the accepted flag, which is a human judgment. And the expiry only holds if CI or the wrapper actually runs check. A guard nobody invokes is documentation.

If the guard logic is useful, run it against one small task on the free lane for a week. The ledger will tell you whether the lane deserves a second week.

The Anti-Use Checklist: Six Signals That a Free Model Lane Is Wrong for a Workload

Casey Sun — Mon, 19 Oct 2026 08:07:19 +0000

A team wired a support triage agent to a free model lane on a Friday. By Tuesday the queue had 340 unparsed replies. Nobody had measured the lane before routing production traffic to it.

The fix took an hour. The outage took three days of trust.

This is a field guide for the opposite decision. It describes when a free lane should be refused, what to measure first, and how to leave cleanly.

Why an anti-use pass comes first

Most teams evaluate a model lane by asking whether it works. That is the easy half. The expensive half is asking what breaks when it fails quietly.

A free lane fails differently from a paid one. Rate limits appear without warning. Output shape drifts after an upstream change. Latency spikes cluster at peak hours.

An anti-use pass costs about thirty minutes. It prevents weeks of rework. Run it before the first production request, not after the first incident.

Six stop signals

Each signal below has a measurable test. If two or more fire, the lane is the wrong tool for that workload.

1. The output is consumed by a signature or billing path

Anything cryptographically signed, financially settled, or legally attested must not depend on a free lane. Webhook verification, invoices, and audit records belong on a deterministic path.

Test: list every consumer of the output. Mark each as reversible or irreversible. Any irreversible consumer is a stop signal.

2. The workload needs a latency ceiling

Free capacity is shared capacity. Tail latency moves even when the median looks healthy.

Test: measure p95, not the mean. A ceiling of 2s at p95 with a free lane usually fails during peak hours.

3. The schema is strict and the parser is brittle

Free tiers sometimes route to different model variants. JSON shape can shift between calls.

Test: run 200 calls and count invalid parses. Above 2% unparsed, the lane needs a repair loop before it is usable.

4. The data is regulated or personal

If the payload contains health records, payment data, or identifiers, the lane choice is a compliance decision. It is not a cost decision.

Test: classify the payload schema. Any regulated field means the free lane is out of scope.

5. The workload has no observability

A lane you cannot measure is a lane you cannot defend. Logs, retries, and token spend must exist before traffic arrives.

Test: can a single request be traced end to end? If not, stop and instrument first.

6. The fallback does not exist yet

The fallback is the product here. A free lane without a rehearsed paid lane is a single point of failure with extra steps.

Test: trigger the fallback deliberately in staging. If it takes more than one flag flip, it is not ready.

A probe you can run before routing traffic

The script below measures the first four signals. It targets any OpenAI-compatible chat endpoint, including MonkeyCode's free model access.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

# freelane_probe.py - stdlib only, Python 3.10+
import json, os, statistics, time, urllib.request

BASE = os.environ["BASE_URL"].rstrip("/")
KEY = os.environ["API_KEY"]
MODEL = os.environ["MODEL"]
N = int(os.environ.get("N", "40"))

TASK = 'Return only JSON: {"intent": one of [refund,bug,other], "confidence": 0-1}'
SAMPLES = ["my card was charged twice", "app crashes on login", "how do I export data"]

def call(prompt):
    body = json.dumps({
        "model": MODEL,
        "messages": [{"role": "system", "content": TASK},
                     {"role": "user", "content": prompt}],
        "temperature": 0,
    }).encode()
    req = urllib.request.Request(
        f"{BASE}/chat/completions", data=body,
        headers={"Authorization": f"Bearer {KEY}", "Content-Type": "application/json"})
    t0 = time.perf_counter()
    try:
        with urllib.request.urlopen(req, timeout=30) as r:
            payload = json.loads(r.read())
        return True, time.perf_counter() - t0, payload
    except Exception as exc:
        return False, time.perf_counter() - t0, {"error": str(exc)}

def parse_score(payload):
    try:
        text = payload["choices"][0]["message"]["content"]
        obj = json.loads(text)
        return obj.get("intent") in {"refund", "bug", "other"} and 0 <= float(obj["confidence"]) <= 1
    except Exception:
        return False

lat, ok, retried, valid = [], 0, 0, 0
for i in range(N):
    prompt = SAMPLES[i % len(SAMPLES)]
    success, elapsed, payload = call(prompt)
    lat.append(elapsed)
    if not success:
        retried += 1
        success, elapsed, payload = call(prompt)
    if success:
        ok += 1
        valid += parse_score(payload)

lat.sort()
p95 = lat[max(0, int(len(lat) * 0.95) - 1)]
report = {
    "model": MODEL, "requests": N,
    "success_rate": round(ok / N, 3),
    "retry_rate": round(retried / N, 3),
    "valid_json_rate": round(valid / N, 3),
    "p50_s": round(statistics.median(lat), 2),
    "p95_s": round(p95, 2),
}
print(json.dumps(report, indent=2))

Run it with a fixed model identifier:

export BASE_URL="https://<your-endpoint>/v1"
export API_KEY="<key>"
export MODEL="<pinned-model-id>"
export N=60
python3 freelane_probe.py | tee probe-report.json

The probe is deliberately small. It measures shape and floor, not quality. Point it at a paid lane too, so the numbers have a baseline.

Reading the report

Thresholds below are starting points. Adjust them to the workload, not to the vendor.

Signal	Healthy	Watch	Stop
success_rate	≥ 0.99	0.95–0.99	< 0.95
retry_rate	≤ 0.05	0.05–0.15	> 0.15
valid_json_rate	≥ 0.99	0.97–0.99	< 0.97
p95_s (interactive)	≤ 2.0	2.0–4.0	> 4.0

Three or more Stop cells mean the lane is unsuitable for that path. Re-run the probe at peak hours before deciding. A single morning sample hides the tail.

The free server option fits well as a probe runner or queue worker. It does not fit signature verification, billing, or anything on the critical path. Keep those services where they already live.

Exit criteria and the rollback ledger

Write the exit criteria before the first request. Store them next to the feature flag.

Probe rerun weekly; two consecutive Stop readings trigger review.
A pinned model identifier is mandatory. Unpinned aliases drift without notice.
The fallback path is exercised in staging every release.
One owner is named. Shared ownership means no ownership.
Token spend per 100 tasks is recorded, including retries and repairs.

The rollback itself should be one switch:

# flip the routing flag, keep the mirror on for comparison
curl -sS -X POST "$FLAG_URL" \
  -H "Authorization: Bearer $FLAG_TOKEN" \
  -d '{"lane":"paid","mirror_free":true}'

Mirroring after rollback is what turns an incident into evidence. Without it, the team guesses.

When this approach is the wrong tool

This field guide is not for everyone.

Teams under contractual SLAs should buy capacity with a support agreement.
Regulated workloads need a compliance review before any lane choice.
Latency-critical paths, such as interactive autocomplete, rarely fit shared free capacity.
Workloads requiring bit-identical output across runs should use deterministic code, not a model.
Teams without logging or tracing should build observability first. A probe cannot fix blindness.

It is also the wrong guide for anyone treating a free tier as permanent. Free capacity is a stage, not a contract.

Field notes on the free tier claim

MonkeyCode's free model access and free server option are operator-stated availability claims. Quotas, model rosters, and durations change. Check the current provider page before planning around them.

The probe above does not care who runs the endpoint. That is the point. Measure the lane you have, not the lane in the announcement.

If the six signals stay quiet after two weeks of mirrored traffic, the lane has earned a wider route. Until then, keep the exit switch warm.

A short note on what changed in your own probe report is welcome in the comments.

The Do-Not-Call List for Free AI Tiers: A Request-Level Policy You Can Version

Casey Sun — Sat, 17 Oct 2026 05:56:34 +0000

A weekly dependency bot opens a pull request against a release branch. A bot posts an AI-written summary of the diff into the PR thread. Nobody reviews that summary before merge. That is the whole incident, and it needed no attacker.

This is a composite of ordinary failure modes, not a customer report. It happens because teams gate at the pipeline level. The risky call is usually a single request inside a green pipeline.

This article is an anti-use field guide. It teaches where a free AI lane must not be the caller, and what to call instead.

Free changes the risk, not the task

A paid tier and a free tier can run the same prompt. The difference is who owns the failure. Free access usually means fewer contractual guarantees, shared capacity, and terms you did not negotiate.

So the question is never "is the model good enough". The question is "what is the blast radius when this specific call is wrong". Classify the request, then pick the lane.

Four questions before any call

Ask these for every request, not every service.

Data class — public, internal, PII, or secret?
Sink — draft, comment, signed artifact, production config, or money?
Reversibility — can you undo the effect in one step?
Tool surface — does the caller hold shell, network, or write credentials?

A request that is public, reversible, and tool-free is a fine candidate for a free lane. A request that is internal, irreversible, and shell-capable is not.

Put the policy in data, not in a wiki

Policies in prose drift. Policies in a file can be hashed, diffed, and pinned per release.

Start with a small JSON policy. Keep it boring on purpose.

{
  "version": 3,
  "hard_deny": {
    "data_class": ["secret", "pii"],
    "sink": ["signed_artifact", "prod_config", "money"]
  },
  "sandbox_only": {
    "data_class": ["internal"],
    "agent_tools": ["shell", "network"]
  },
  "allow": {
    "task": ["summarize", "rename_suggestion", "changelog_draft"]
  }
}

Note the shape. Deny rules are absolute. Sandbox rules are advisory and require human review. The allow list is explicit, so unknown tasks fall to review instead of shipping.

The classifier

This runs before the model call. It is stdlib-only and short enough to read in a review.

# classify.py — Python 3.11+, no third-party deps
from __future__ import annotations

import hashlib
import json
import sys
from dataclasses import dataclass
from enum import Enum
from pathlib import Path


class Decision(str, Enum):
    ALLOW = "ALLOW"
    SANDBOX = "SANDBOX"
    DENY = "DENY"


@dataclass(frozen=True)
class Request:
    task: str
    data_class: str          # public | internal | pii | secret
    sink: str                # draft | comment | signed_artifact | prod_config | money
    reversible: bool
    human_in_loop: bool
    agent_tools: tuple[str, ...] = ()


def load_policy(path: Path) -> tuple[dict, str]:
    raw = path.read_bytes()
    return json.loads(raw), hashlib.sha256(raw).hexdigest()[:12]


def classify(req: Request, policy: dict) -> tuple[Decision, list[str]]:
    reasons: list[str] = []

    for name, blocked in policy["hard_deny"].items():
        if getattr(req, name) in blocked:
            reasons.append(f"hard_deny:{name}={getattr(req, name)}")
    if reasons:
        return Decision.DENY, reasons

    for name, gated in policy["sandbox_only"].items():
        value = getattr(req, name)
        hit = any(v in value for v in gated) if isinstance(value, tuple) else value in gated
        if hit:
            reasons.append(f"sandbox_only:{name}={value}")

    if req.task not in policy["allow"]["task"]:
        reasons.append(f"task_not_allowlisted:{req.task}")

    if not req.reversible and not req.human_in_loop:
        reasons.append("irreversible_without_review")

    return (Decision.SANDBOX if reasons else Decision.ALLOW), reasons or ["clean"]

The policy hash is the important line. Every decision record carries the exact policy version that produced it.

Log every decision, then query it

The log is JSONL. One line per request. It is the artifact you will actually use during an incident.

def emit(req: Request, decision: Decision, reasons: list[str], policy_hash: str, out=sys.stdout) -> None:
    record = {
        "policy": policy_hash,
        "decision": decision.value,
        "reasons": reasons,
        **{k: list(v) if isinstance(v, tuple) else v for k, v in req.__dict__.items()},
    }
    out.write(json.dumps(record) + "\n")

Run it against a batch and read the distribution before you trust it.

python classify.py policy.json < requests.jsonl > decisions.jsonl
jq -r 'select(.decision != "ALLOW") | .reasons[]' decisions.jsonl \
  | sort | uniq -c | sort -rn | head

That second command is the debugging workflow. It tells you which rule fires most. If task_not_allowlisted dominates, your policy is too narrow. If hard_deny:sink=signed_artifact fires often, you have a design problem upstream.

Pin the decisions with tests

A classifier without tests is a comment. Parametrize the matrix and run it in CI.

# test_classify.py
import pytest
from classify import Decision, Request, classify

POLICY = {
    "version": 3,
    "hard_deny": {"data_class": ["secret", "pii"],
                  "sink": ["signed_artifact", "prod_config", "money"]},
    "sandbox_only": {"data_class": ["internal"], "agent_tools": ["shell", "network"]},
    "allow": {"task": ["summarize", "rename_suggestion", "changelog_draft"]},
}

CASES = [
    (Request("summarize", "public", "draft", True, False), Decision.ALLOW),
    (Request("summarize", "pii", "draft", True, True), Decision.DENY),
    (Request("summarize", "public", "prod_config", True, True), Decision.DENY),
    (Request("summarize", "internal", "comment", True, True), Decision.SANDBOX),
    (Request("deploy", "public", "draft", False, False, ("shell",)), Decision.SANDBOX),
]

@pytest.mark.parametrize("req,expected", CASES)
def test_matrix(req, expected):
    assert classify(req, POLICY)[0] is expected

The fifth case is the one that saves you. A capable agent on an unlisted task lands in review, not in production.

Red flags and what to call instead

Most free-lane failures have a cheaper replacement. Reach for that first.

Red flag	Why the free lane is wrong	Better alternative
Output signs or approves anything	No accountability trail on a shared tier	Deterministic signer, human approver
Input carries PII or secrets	Terms and retention are not yours	Redaction step, then classify again
Caller holds shell or network tools	Irreversible side effects, no rollback	Sandbox with no credentials
Same prompt runs on every commit	Cost and drift scale silently	Cached fixture, golden-file test
Output lands in a config or migration	One bad token is an outage	Schema validation, then review
Task is formatting or renaming	Deterministic problem, probabilistic tool	Formatter, AST rewrite, linter
Latency sits on a user request path	Shared capacity, unknown queueing	Precompute, or a committed tier

Print that table. Tape it to the review checklist. The first three rows are hard stops.

Exit criteria for the free lane

Write these before adoption, not after the first incident. Trigger any one and move the workload.

P95 latency exceeds your user-facing budget for two consecutive weeks.
Error or refusal rate crosses the threshold you set at adoption.
Any request gains write credentials or touches a signed path.
The data classification changes, even if the task does not.
A review finds that nobody read the sandbox output for a full sprint.
Output quality drifts on a golden set you can no longer explain.

Criterion five is the quiet one. A sandbox nobody reads is a production lane with extra steps.

Where a free lane legitimately fits

MonkeyCode ships as an open-source project with a free model allowance and a free server option. Those are operator-stated availability claims.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Treat both as shared capacity, not as entitlements. Quotas, limits, and terms change, so read the project's current documentation instead of trusting any blog post, including this one.

In this policy, that lane maps to task in allow, data_class == public, sink == draft, and agent_tools == (). Summarize a public doc. Draft a changelog entry a human will edit. Suggest a test name. Keep the classifier in front of it, and the free server stays a staging surface rather than a signing surface.

If you want to test the workflow, start with the summarizer task. Leave the deploy bot out of it.

Limitations and who should not use this

This classifier is a policy check, not an enforcement boundary. A caller can pass wrong labels. Pair it with egress rules, scoped credentials, and separate identities per lane.

Skip this approach in three cases:

You have no data classification scheme. Build that first, or the labels are guesses.
You cannot review sandbox output within a stated SLA. Then every sandbox is really an allow.
You operate regulated workloads with obligations you have not read against the vendor terms.

The pattern is small on purpose. Copy the two files, replace the allow list with your real tasks, and run the test matrix once. If more than half the cases land in DENY, your pipeline shape is the problem, and no tier will fix it.

Refusing the Free Lane: A Red-Flag Gate You Can Run in CI

Casey Sun — Fri, 16 Oct 2026 04:53:29 +0000

A routing request lands in a review queue on a Friday afternoon. The workload classifies support emails and drafts replies. Someone proposes the free model lane, and someone else proposes the free server too.

The proposal looks harmless. It usually is, until the workload starts signing things, retrying forever, or holding state. Free capacity rewards small, stateless, forgiving jobs. Everything else pays later, with interest.

This is a refusal guide, not an adoption guide. It covers red flags, a CI gate, and exit criteria. Recent community threads about AI fatigue point the same way: fewer lanes, clearer boundaries.

Why refusal beats optimization

Free capacity invites overuse. Teams tune the lane before they define its boundary. The bill arrives as retries, incident load, and data exposure.

A refusal decision costs one meeting. A migration under incident pressure costs a weekend and trust. Say no early, and the lane stays boring.

Seven red flags worth acting on

Each flag below describes the workload, not a vendor. Any single flag is a conversation. Two or more usually mean refusal.

The output has legal, financial, or access-control effect. Refunds, signatures, and permission grants need a lane you control end to end.
The job must survive unbounded retries safely. If a duplicate run double-charges or double-sends, the lane is disqualified.
Prompts carry regulated or personal data. The lane's data path becomes part of your compliance scope by default.
The workload owns durable state. Sessions, locks, and queues belong in storage you can back up and restore.
A hard latency SLO is contractually binding. Shared capacity turns p99 into a weather report.
The job needs custom kernels, specific accelerator memory, or warm caches. Specialized execution rarely fits a general free tier.
Failures page a human who has no remedy. If the only runbook step is "wait", the workload is misrouted.

A refusal harness you can run in CI

The decision belongs in code, not in a Slack thread. The harness below uses only the Python standard library. It reads a policy file and a workload manifest, then exits non-zero on refusal.

// policy.json
{
  "weights": {
    "legal_or_financial_effect": 5,
    "unbounded_retries": 3,
    "regulated_data_in_prompt": 4,
    "durable_state_in_worker": 4,
    "contractual_latency_slo": 2,
    "custom_accelerator_needed": 3,
    "no_operator_remedy": 3
  },
  "thresholds": { "shadow": 5, "refuse": 12 }
}

// services/checkout-draft.json
{
  "name": "checkout-draft",
  "legal_or_financial_effect": true,
  "unbounded_retries": true,
  "regulated_data_in_prompt": false,
  "durable_state_in_worker": true,
  "contractual_latency_slo": false,
  "custom_accelerator_needed": false,
  "no_operator_remedy": false
}

#!/usr/bin/env python3
"""Refuse or admit a workload to a free model lane. Standard library only."""
import json
import pathlib
import sys


def load(path):
    return json.loads(pathlib.Path(path).read_text(encoding="utf-8"))


def evaluate(workload, policy):
    weights = policy["weights"]
    hits = [flag for flag in weights if workload.get(flag) is True]
    total = sum(weights[flag] for flag in hits)
    if total >= policy["thresholds"]["refuse"]:
        verdict = "refuse"
    elif total >= policy["thresholds"]["shadow"]:
        verdict = "shadow"
    else:
        verdict = "admit"
    return total, hits, verdict


def main(argv):
    policy = load(argv[1])
    workload = load(argv[2])
    total, hits, verdict = evaluate(workload, policy)
    print(f"workload={workload['name']} score={total} verdict={verdict}")
    for flag in hits:
        print(f"  flag={flag} weight={policy['weights'][flag]}")
    if verdict == "refuse":
        print("blocked: route this workload off the free lane before merge")
        return 1
    return 0


if __name__ == "__main__":
    sys.exit(main(sys.argv))

Running the two sample files yields score=12 verdict=refuse. The build fails, and the reviewer gets the flags instead of a debate.

python3 tools/free_lane_audit.py policy.json services/checkout-draft.json
# workload=checkout-draft score=12 verdict=refuse
#   flag=legal_or_financial_effect weight=5
#   flag=unbounded_retries weight=3
#   flag=durable_state_in_worker weight=4

Wire the gate into pull requests

# .github/workflows/free-lane-gate.yml
name: free-lane-gate
on: [pull_request]
jobs:
  audit:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: "3.12"
      - name: Audit workload manifests
        run: |
          set -euo pipefail
          for f in services/*.json; do
            python3 tools/free_lane_audit.py policy.json "$f"
          done

Decision table: admit, shadow, or refuse

Workload property	Admit	Shadow first	Refuse
Stateless text transform, retry-safe	Yes	—	—
Draft output with human approval on every item	Yes	1 week	—
Writes to your production datastore	—	—	Yes
Signs tokens, moves money, grants access	—	—	Yes
Contractual p99 commitment	—	—	Yes
Unknown cost profile, no named owner	—	2 weeks	—

Shadow mode means the free lane runs in parallel and its output is logged, not used. Admit only after the shadow window closes without unexplained diffs.

Exit criteria and order of retreat

Write the exit plan before the first deploy. The sequence matters more than the trigger.

Name one owner and one backup owner. No owner means no admission.
Log lane, model, latency, retry count, and refusal rate per request.
Open a review when p95 exceeds the internal budget for three consecutive days.
Retreat immediately after two incidents traced to the lane, regardless of cost.
Move workloads in this order: durable state first, then signing paths, then latency-critical callers, then bulk traffic last.

The last step is counterintuitive. Teams usually move bulk traffic first because it is easy. That leaves the risky paths in place longest.

Where MonkeyCode's free options fit

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The operator behind MonkeyCode describes free model access and a free server option as part of the offering. Treat both as a lane with an owner and a refusal test, exactly like any other.

The right fit is narrow and useful: experiments, draft generation, internal tooling, and shadow runs. Quotas, terms, and capacity change over time. Read the current page before planning around them, and do not copy numbers from a blog post into a capacity plan.

If the audit returns admit, the free lane is a reasonable place to start. If it returns refuse, the harness just saved a rollback.

Limitations and who should not use this

This is a proposed checklist, not a standard. It does not prove savings, and it cannot detect a red flag nobody wrote down.

Skip this approach if the team lacks request logging, has no on-call rotation, or cannot review its own data path. Skip it for regulated workloads until legal signs the lane boundary. A green audit means no known red flag is present, nothing more.

What to keep

One policy file, one script, one CI job. That is the whole artifact. It will not make a free lane safe for every workload, and it is not meant to.

Its real job is speed: a clear refusal in review beats a slow discovery in production.

The Demotion Runbook: Taking a Workload Off a Free Model Without a Bad Deploy

Casey Sun — Thu, 15 Oct 2026 11:11:29 +0000

The Demotion Runbook: Taking a Workload Off a Free Model Without a Bad Deploy

The night the free lane stopped being free

A nightly summarizer sat on a free model lane for five months. Its eval suite passed every Friday. Then an upstream schema change doubled the average input length. Retry volume tripled before Monday. A reviewer burned two hours re-reading outputs nobody trusted.

The team had an onboarding checklist. It had no removal plan. The scenario above is a composite illustration, not a customer report.

A passing eval is not a residency permit

Evals measure answer quality on a frozen sample.
They do not measure retry amplification, reviewer minutes, or blast radius.
They do not expire, so a green suite quietly becomes a permission slip.

The pattern shows up in current discussion too. One recent DEV post reports getting an eval suite to zero mistakes and still declining to make that model the default. That instinct is correct. A clean score backs a model choice. It does not license a permanent deployment.

Three tripwires that force a demotion decision

Tripwire 1: retry amplification

Count retries and divide by first attempts. A lane above 1.5 is telling you the input distribution moved. Paying for a stronger model is often cheaper than paying for retries.

Tripwire 2: reviewer minutes per 1,000 attempts

Track how long a human spends correcting or re-reading output. This number hides well. It rarely appears in a cost dashboard, and it is usually the first tripwire to fire.

Tripwire 3: blast radius

Count distinct downstream writers the lane touches. Two writers means a demotion is a config change. Five writers means a demotion is an incident.

The artifact: a lane ledger and tripwire gate

Save this as lane_ledger.py. It reads a JSONL event log, computes the three tripwires per lane, and returns exit code 1 when any lane must be demoted.

#!/usr/bin/env python3
"""lane_ledger.py — tripwire check for workloads parked on a free model lane."""
import argparse
import json
import sys
from collections import defaultdict
from dataclasses import dataclass


@dataclass
class Tripwires:
    retry_amp: float = 1.5           # retries / first attempts
    review_min_per_1k: float = 20.0  # reviewer minutes per 1k attempts
    blast_radius: int = 3            # distinct downstream writers


def load(path):
    with open(path, encoding="utf-8") as fh:
        for n, line in enumerate(fh, 1):
            line = line.strip()
            if not line:
                continue
            try:
                yield json.loads(line)
            except json.JSONDecodeError as exc:
                raise SystemExit(f"bad JSON on line {n}: {exc}")


def summarize(events):
    lanes = defaultdict(lambda: {"attempts": 0, "retries": 0, "review_min": 0.0,
                                 "writers": set(), "rejects": 0, "tokens": 0})
    for ev in events:
        row = lanes[ev["lane"]]
        kind = ev["kind"]
        if kind == "attempt":
            row["attempts"] += 1
        elif kind == "retry":
            row["retries"] += 1
        elif kind == "review":
            row["review_min"] += float(ev["minutes"])
        elif kind == "write":
            row["writers"].add(ev["target"])
        elif kind == "reject":
            row["rejects"] += 1
        else:
            raise SystemExit(f"unknown kind: {kind}")
        row["tokens"] += int(ev.get("tokens", 0))
    return lanes


def evaluate(row, t):
    attempts = row["attempts"] or 1
    amp = row["retries"] / attempts
    review = row["review_min"] * 1000 / attempts
    blast = len(row["writers"])
    hits = []
    if amp > t.retry_amp:
        hits.append(f"retry amplification {amp:.2f} exceeds {t.retry_amp}")
    if review > t.review_min_per_1k:
        hits.append(f"review load {review:.1f} min/1k exceeds {t.review_min_per_1k}")
    if blast >= t.blast_radius:
        hits.append(f"blast radius {blast} writers reaches {t.blast_radius}")
    return ("DEMOTE" if hits else "STAY"), hits, amp, review, blast


def main(argv):
    ap = argparse.ArgumentParser(description="Free-lane tripwire check")
    ap.add_argument("ledger", help="JSONL file of lane events")
    ap.add_argument("--retry-amp", type=float, default=1.5)
    ap.add_argument("--review-min-per-1k", type=float, default=20.0)
    ap.add_argument("--blast-radius", type=int, default=3)
    args = ap.parse_args(argv)
    t = Tripwires(args.retry_amp, args.review_min_per_1k, args.blast_radius)
    lanes = summarize(load(args.ledger))
    print(f"{'lane':<20}{'amp':>6}{'rev/1k':>9}{'blast':>7}  verdict")
    demoted = False
    for name, row in sorted(lanes.items()):
        verdict, hits, amp, review, blast = evaluate(row, t)
        print(f"{name:<20}{amp:>6.2f}{review:>9.1f}{blast:>7}  {verdict}")
        for hit in hits:
            print(f"    - {hit}")
        demoted = demoted or verdict == "DEMOTE"
    return 1 if demoted else 0


if __name__ == "__main__":
    sys.exit(main(sys.argv[1:]))

Record events from your own pipeline. One line per event:

{"lane":"free-summarizer","kind":"attempt","tokens":820}
{"lane":"free-summarizer","kind":"retry","tokens":810}
{"lane":"free-summarizer","kind":"review","minutes":4.5}
{"lane":"free-summarizer","kind":"write","target":"s3://reports/daily"}
{"lane":"free-summarizer","kind":"write","target":"webhook:slack"}
{"lane":"free-summarizer","kind":"reject"}

Run it against a week of traffic:

python lane_ledger.py ledger.jsonl --retry-amp 1.3 --review-min-per-1k 15

Expected output shape:

lane                    amp   rev/1k  blast  verdict
free-summarizer       1.00   4500.0      2  DEMOTE
    - review load 4500.0 min/1k exceeds 15.0

The verdict is mechanical. The thresholds are not. Set them from your own service level, and commit the values so they are reviewable.

Reading the verdict

Verdict	Signal	Next move
STAY	All three tripwires clear	Keep the lane, re-run weekly
DEMOTE, review only	Quality holds, human cost does not	Add a deterministic pre-filter
DEMOTE, retry amp	Lane is unstable on real input	Move to a paid tier or a self-hosted model
DEMOTE, blast radius	Lane touches too many writers	Split it; keep read-only paths only
Two or more hits	The lane is load-bearing	Freeze feature work and schedule a cutover

Red flags: when not to use a free lane at all

The output gates a release, an eval report, or a billing event.
The workload signs, writes to, or acknowledges a signed webhook.
The workload holds or renews a lease, lock, or idempotency key.
A legal hold, retention rule, or audit requirement applies.
p99 latency is a product promise rather than a nice-to-have.
A retry storm would hit a shared dependency you do not control.
The cost of one wrong answer exceeds the cost of a paid call.

If two or more flags apply, the free lane is the wrong tier. No amount of prompt work fixes an architecture mismatch.

Better alternatives by failure mode

Failure mode	Cheaper fix than a stronger model
Invalid JSON output	Schema validation plus one bounded repair pass
Long inputs	Chunking with a deterministic reducer
Retry storms	Token-bucket queue and a dead-letter lane
Reviewer fatigue	Confidence threshold plus sampled human review
Slow tail latency	Batch the work and move it off the request path

Three rows in that table remove the model entirely. Free model access is a cost lever, not a design decision.

Write the exit criteria before you onboard

Four numbers belong in the design doc on day one.

Retry amplification ceiling, for example 1.5.
Reviewer minutes per 1,000 attempts, for example 20.
Blast radius cap, for example two writers.
A demotion deadline: two consecutive red weeks triggers the cutover.

Anyone can add a lane. Someone has to own removing it.

Where MonkeyCode fits, and where it does not

MonkeyCode is an open-source project that provides model access and a server option for developers. Per details supplied by the project's operator, the free tier includes 10 million free tokens and a free server. Free-tier terms change, so confirm current limits and duration in the project's own repository before planning around them.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Under this runbook, MonkeyCode's free tier fits rehearsal rather than residency. Use it to run the ledger on throwaway lanes, prototype read-only paths, and practice the cutover before real traffic arrives. Do not use it for the red-flag list above. A lane that signs artifacts, renews leases, or gates a release needs a tier with an explicit contract.

The token budget is generous for prototyping. It is not a substitute for the four exit criteria. Cheap inference still costs reviewer minutes when the output is wrong.

Limitations

The composite scenario above is illustrative and not a measured benchmark.
The tripwire thresholds are examples, not industry standards.
No latency, uptime, or retention guarantee is claimed for any tier.
The ledger script covers three signals only. Add more only if you can act on each one.

One last thing

Run the ledger against your own traffic first. If the numbers clear your thresholds, MonkeyCode's free tier is a reasonable place to rehearse the next lane.

When a Free Lane Must Not Sign the Eval Report

Casey Sun — Sun, 11 Oct 2026 20:38:56 +0000

A Friday deploy waits on a single eval report. The patch came from a free model lane. That same lane then scored the patch as safe.

No second examiner signed the written ship result. The pattern looks efficient to a tired reviewer. It is not a valid release gate today.

A grader that shares quota and host cannot disprove its own work. Shared prompt history makes a clean verdict unlikely.

The failure this guide targets

Agent drafts now land faster than review staffing grows. A green self-score becomes the only stored evidence. The free lane then holds both the draft and the verdict.

This guide is a written field refusal list. It names red flags, safer placements, and exit criteria. It does not claim a measured model ranking.

Why a self-score fails closed

Shared identity

An examiner must be able to fail the author. That power disappears when both roles share one key. A quota error then threatens the verdict itself.

Disposable storage

A free server adds a second operational trap. Scratch disks get wiped, resized, or quietly reclaimed. The only trace of a failure can vanish with them.

Inflated scores

Perfect scores deserve extra doubt rather than extra trust. Case leakage, skipped rows, and hint-filled prompts inflate them. A frozen content hash is the minimum countermeasure here.

Red flags

Block auto-apply when any flag below is true. Treat the list as a hard stop, not a suggestion.

The scorer and the author share one API key.
The eval harness runs only on the free server.
A zero-mistake score has no frozen case set.
The same job can delete failures after scoring.
The report is the only artifact kept before apply.
Prompt text quietly includes the expected answers.
Quota pressure can skip cases to finish the run.
A human cannot replay the case set offline.
The grader principal is a rename of the author.
Security cases are marked optional near the quota.

One flag is enough to block auto-apply. Two flags mean the lane is only an author. Do not promote that lane into the examiner role.

Better alternatives

Keep generation on the cheap lane when blast radius stays small. Move the oracle to another principal and another host. Keep raw traces where the free server cannot delete them.

Freeze the case set in git with a content hash.
Run the scorer under a different network path.
Require a human signature for auth, data, or money diffs.
Store traces in an append-only bucket off the free host.
Treat a perfect score as a sampling prompt, not a pass.
Fail the job when any security case is skipped.
Record the grader key id beside the case-set hash.

A free model may still draft the patch. A free server may still host a scratch replay. Neither may become the system of record for the verdict.

Decision table

Use this table before any production apply step. The middle column describes a disposable sandbox only. The right column describes production authority you must withhold.

Signal	Free lane may	Free lane must not
Draft a unit-test sketch	Yes, as a marked proposal	Merge it unmarked
Score its own patch	No	Sign the ship report
Host a scratch replay	Yes, if the disk is disposable	Store the only trace
Skip cases near quota	No	Drop security cases first
Name a human reviewer	Yes, as plain text	Impersonate that reviewer
Hold release evidence	No	Be the only archive

A gate you can run

The snippet below is a labeled proposal only. It was not executed against a live account for this draft. It checks identity, case-set hash, and storage separation.

# Proposal only. Not an executed production test.
import hashlib
import json
from pathlib import Path

FREE_LANE_PRINCIPALS = {"free-model", "free-server"}

def case_set_hash(path: Path) -> str:
    payload = path.read_bytes()
    return hashlib.sha256(payload).hexdigest()

def allow_apply(report: dict, expected_hash: str) -> tuple[bool, str]:
    author = report.get("author_principal", "")
    grader = report.get("grader_principal", "")
    if not author or not grader:
        return False, "missing principal"
    if author in FREE_LANE_PRINCIPALS and grader == author:
        return False, "self-grade on a free lane"
    if author in FREE_LANE_PRINCIPALS and grader in FREE_LANE_PRINCIPALS:
        return False, "both roles still sit on the free lane"
    if report.get("case_set_sha256") != expected_hash:
        return False, "case set hash mismatch"
    if report.get("trace_store") in FREE_LANE_PRINCIPALS:
        return False, "only copy of traces is on the free lane"
    skipped = report.get("skipped_security_cases")
    if skipped is None or skipped > 0:
        return False, "security cases were skipped or omitted"
    if report.get("blast") == "high" and not report.get("human_signed"):
        return False, "high blast change lacks a human signature"
    return True, "oracle separated from the free lane"

if __name__ == "__main__":
    expected = case_set_hash(Path("cases/frozen.jsonl"))
    sample = {
        "author_principal": "free-model",
        "grader_principal": "free-model",
        "case_set_sha256": expected,
        "trace_store": "audit-bucket",
        "skipped_security_cases": 0,
        "human_signed": False,
        "blast": "high",
    }
    ok, reason = allow_apply(sample, expected)
    print(json.dumps({"ok": ok, "reason": reason}))

Pin the frozen case file before you run the script. A false result must fail the pipeline run. Do not catch that error and continue the apply.

sha256sum cases/frozen.jsonl
python gate.py
# Proposal expectation, not a captured live log:
# {"ok": false, "reason": "self-grade on a free lane"}

Fail closed in CI

Wire the script as a required check, not a comment. A proposal step looks like the block below. It is not a workflow taken from a live account.

# Proposal only. Not a published workflow.
- name: reject self-graded free lane
  run: python gate.py

Two free principals still fail the second guard. A rename must be added to the set before the next run. Otherwise the denylist becomes a mere label game.

Fixture checklist

Walk this list on every pipeline change you ship. Mark a failed row as failed closed, not skipped.

Self-grade with a matching hash returns false.
A renamed free principal must still return false.
Hash mismatch returns false even with a new grader.
A free-server trace store must always return false.
A skipped security case must always return false.
High blast without a human signature returns false.
External grader, external traces, and a signature return true.
Missing report keys return false, not a default pass.

Missing keys need an explicit guard in the runner. The sample function already fails some empty gaps. Extend that function rather than wrapping a bare except.

Where a workshop lane still fits

Disclosure: This article was prepared as part of MonkeyCode's product outreach. Operator notes for this draft describe MonkeyCode free model access. Those same notes also describe a free server option.

The notes state a free pool of ten million tokens. Confirm that figure and any expiry in current docs. This draft does not treat the number as a benchmark.

It also does not treat the pool as a permanent quota. Use that access to draft patches and sketch new cases. Use the free server for a scratch replay you can delete.

Keep the signed verdict and the trace bucket elsewhere. A workshop lane is still useful after the split. It is not a quiet second production region.

If current docs show a smaller pool, drop those steps. If docs show no free server, drop the scratch host too. Do not hard-code an allowance the vendor can change.

Exit criteria

Move the examiner role off the free lane when any line is true. Leave drafting in place only when the oracle sits elsewhere.

The case set now includes customer data or live secrets.
Apply rights can change auth, billing, or data retention.
The free host cannot prove trace durability for audits.
Quota errors start skipping cases instead of failing.
Two releases passed only because of a self-score.
Reviewers cannot replay a failure without the free host.
The same key can both write the score and delete it.
A renamed free principal bypassed the denylist once.

Exit means you split the principals and the storage. It does not always mean you abandon cheap drafting. Re-run the fixture list after every such split.

Who should not use this split

Skip this pattern if you have no second principal. A paper grader name on the same key is theater. The script will only echo the lie you typed.

Skip it for homework with no apply step. A local notebook does not need this gate. Spend that time on clearer cases instead of ceremony.

Skip it when policy already forbids model-written code. This guide assumes a draft lane already exists. If drafts are banned, the middle column is already closed.

Do not park production on the free server to dodge cost. That move recreates one failure domain under a new name. Cost relief is not an audit design by itself.

Limitations

The script checks strings you pass into it. It cannot discover a renamed free principal alone. Map those names from your real identity provider.

A content hash does not prove case quality. A frozen weak set still matches its own hash. Review the cases on a schedule you can defend.

No latency, cost, or accuracy numbers are claimed here. None of those figures were measured for this draft. Do not cite this page as a product benchmark.

Token allowances change without much notice in many products. A stated pool of ten million tokens can be revised. Re-read the vendor docs on the day you ship.

A second model on the same free pool shares one failure domain. Prefer a different host plus a human for high-blast changes. Model diversity alone is not real independence here.

This guide does not cover legal holds or webhook signatures. Lock leases and cleanup labels need their own controls. Do not reuse this gate as a universal policy engine.

Close

Keep the free lane inside the workshop boundary. Let a separate oracle sign the ship report.

If a workshop lane still helps, confirm the current docs first. Try free model access only for drafts you can discard.

A Green Suite Must Not Unlock the Free Server

Casey Sun — Sat, 10 Oct 2026 17:05:18 +0000

The scene below is a composite staging pattern, not a logged customer incident. A Friday patch looked finished before anyone read the diff. A free model had rewritten a small rate-limit parser.

The local unit suite reported zero failures that afternoon. A shared free server then received the binary. Monday traffic sent a header the tests never built.

The parser dropped that header and opened the limiter. Staging became a noisy neighbor for every other tenant. That pattern is a promotion failure, not a model-quality debate.

A green suite measured the tests already on disk. It did not measure the traffic that host would meet. A free lane can still draft the patch.

It must not own the promotion decision alone. This note is not a workload placement guide. It is a promotion gate for model-written patches.

What a green suite actually proved

A unit suite proves the cases someone already wrote. It does not prove header coverage for live traffic. It does not prove clock behavior on the host.

It does not prove neighbor isolation on a shared box. Zero failures can still hide one missing case. Treat a perfect score as a narrow lab result.

Do not treat that score as a deploy permit. CI logs often add a false kind of comfort. A green check mark names a workflow file.

It does not name the absent fixture at all. Read the test file list before the badge. If the new header is missing there, the badge is noise.

Red flags before a free-server copy

Stop the copy when any flag below is true. One flag is enough to hold the binary. Several flags mean the lane itself is wrong.

The diff touches auth, billing, webhooks, or limits.
The suite has no fixture for the new header or clock.
The target host is shared with other tenants or jobs.
Rollback needs a person who is already offline.
The model wrote both the patch and the new tests.
No human diff review is attached to the change record.
The free server has no per-tenant CPU or restart cap.
Logs on that host mix several projects in one stream.

Move the change to an isolated reviewed host. Or drop the change until the suite grows real fixtures. Do not argue from the green badge alone.

Read the diff before the badge

Run these commands from the branch under review. They are a review habit, not a claim about any one repository. Scan the name list before opening the badge.

git diff --name-only origin/main...HEAD
git diff --stat origin/main...HEAD
git log -1 --format='%an %ae %s'

Scan the name list for limit, auth, bill, and webhook paths. Open the matching test files in the next step. Confirm a new fixture fails on the old parser.

If the fixture passes on the old parser, it does not guard the change. Write a better fixture before any host copy. A passing new test on old code is a weak guard.

A clock case with the same hold

A second composite case shows the same hold without a limiter. A report job sorted dates as plain strings. A free model rewrote that date sort alone.

The tests used one local timezone only. The scratch host ran in UTC that night. Monday's report hid Sunday's rows from the office view.

The suite on that branch was fully green. The missing fixture was the host clock setting. Hold that binary before any shared host copy.

A local hold script

The script below is an unexecuted proposal only. It has not been executed for this article. It reads a small local manifest file first.

It exits non-zero when the promotion is unsafe. Wire it in front of any copy to a free server. Do not treat a zero exit as proof of correctness.

This proposal never exits zero on purpose here. A later human step can override the hold. Record that override outside the model transcript file.

#!/usr/bin/env bash
# proposed gate — not an executed benchmark
set -euo pipefail

manifest="${1:-promotion.yaml}"
test -f "$manifest" || { echo "missing manifest"; exit 2; }

python3 - "$manifest" <<'PY'
import sys, pathlib
try:
    import yaml
except ImportError:
    sys.stderr.write("pyyaml required for this proposal")
    sys.exit(2)

raw = pathlib.Path(sys.argv[1]).read_text()
data = yaml.safe_load(raw) or {}
reasons = []
touches = set(data.get("touches") or [])
banned = {"auth", "billing", "webhook", "limits"}

if data.get("model_wrote_tests"):
    reasons.append("model wrote the tests that claim coverage")
if touches & banned:
    reasons.append("diff touches a banned path for this lane")
if not data.get("human_diff_review"):
    reasons.append("no human diff review on the change record")
if data.get("target") == "shared-free-server" and not data.get("neighbor_cap"):
    reasons.append("shared host has no neighbor cap")
if "limits" in touches and int(data.get("new_header_fixtures") or 0) < 1:
    reasons.append("limit change has no new header fixture")
if "clock" in touches and int(data.get("clock_fixtures") or 0) < 1:
    reasons.append("clock change has no timezone fixture")
if not data.get("rollback_owner_online"):
    reasons.append("rollback owner is offline")

print("HOLD")
for item in reasons:
    print("- " + item)
print("- green suite is not a promotion permit")
sys.exit(1)
PY

Example manifest for the Friday parser:

# proposed fixture — not a production record
model_wrote_tests: true
touches: ["limits"]
human_diff_review: false
target: shared-free-server
neighbor_cap: false
new_header_fixtures: 0
clock_fixtures: 0
rollback_owner_online: false
suite_result: green

Expected result of this proposal is exit code 1. Printed reasons name the hold in plain text. A green suite result is present and then ignored.

The field may still exist in the notes. It must not clear the promotion gate alone. Keep that rule even if a later script grows new checks.

Decision table

Use this table before any copy step. A row can fail while the suite stays green. Clear prose is not a promotion permit here.

Signal	Free-model draft	Free-server run	Promotion
Docs typo, no runtime path	Allowed with review	Unnecessary	Human merge only
Parser change, new fixtures, isolated host	Draft only	Scratch run only	Hold for a human canary
Parser change, missing header fixture	Refuse this lane	Do not copy the binary	Rewrite tests first
Clock change, one timezone in tests	Draft only	Refuse shared hosts	Add clock fixtures first
Auth, billing, or webhook diff	Refuse this lane	Refuse this host	Use an isolated reviewed path
Model wrote the passing tests	Draft the code only	Do not trust that suite	Independent tests required
Shared host, no neighbor cap	Irrelevant	Refuse this host	Move to an isolated box

A fast draft is not a canary plan. Read the row that matches the diff. Then stop or continue on that row alone.

Better alternatives

Keep the free lane on disposable drafts only. Generate a patch, then stop that lane there. A second person writes the missing fixtures next.

Those fixtures must fail on the old code. Only then may a scratch process start safely. Prefer an isolated scratch host for that process.

A laptop container is enough for a parser. A shared free server is a poor neighbor for limit experiments. Cap restarts on any scratch box in use.

Wipe the box after the scratch run ends. Do not point live traffic at that box. Split the roles on the change record itself.

One lane may suggest a diff and nothing more. That same lane may not grade its own tests. A human owns the promotion step in full.

A human owns the rollback step as well. Write both names next to the manifest hash. Store that record outside the model chat.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. The operator states that free model access exists. The same note states a free server option.

This guide uses those options only as a draft lane and a scratch host. It does not quote quotas, hardware, duration, or model names. No token allotment is stated in this draft.

No primary quota figure was attached here. Primary docs remain the source for current terms. Terms can change between one week and the next.

This text is not a capacity or uptime promise. Where a free model is used, keep it off the promotion gate. Where a free server is used, keep it off shared tenant traffic.

Remove every product name and the method still stands. The hold is about blast radius, not a score. It is not about a vendor score at all.

Exit criteria

Leave the free lane when any line below becomes true. Exit means stop the copy to the host. It does not mean delete the local draft.

The patch now touches auth, billing, webhooks, or limits.
The suite cannot fail the old code on a new fixture.
The only host available is shared and uncapped.
The rollback owner is offline for the next traffic window.
The same model wrote the patch and the passing tests.
Primary docs no longer match the access assumed last week.
A neighbor job has already been starved on that host.
The change record has no human diff review link.

Park the patch in the open review branch. Grow the suite with one failing fixture first. Book an isolated host for the next run.

Then review the manifest with a second person. Re-entry is allowed only after those flags clear. A later green suite does not reopen a closed flag.

Clear the flag inside the manifest itself first. Run the hold script again on that file. Expect another hold from this proposal script run.

Then apply a separate human canary plan next. Do not let the model write that canary plan alone. A second person should name the abort signal.

Limits of this guide

This guide does not measure model accuracy at all. It does not rank hosts or cloud regions. It does not set a token budget here.

It does not claim a free server is private or durable. It does not claim a free lane is staffed or permanent. The script remains an unexecuted local proposal only.

PyYAML is an assumed import, not a bundled tool. Review the script locally before any real wiring. A missing import should fail closed, as written.

Do not use this gate as a compliance certificate. It will not catch a weak fixture that still passes. It will not catch a hostile dependency in the tree.

It will not replace secret scanning or policy review. Pair it with ordinary human code review steps. Pair it with an isolated canary run next.

Pair it with a rollback drill a person can run without the model. Teams with no shared staging can still use the red-flag list. They should not invent a free-server story to fit the table.

Solo experiments on disposable repos can skip the shared-host rows. They should not skip a human reading of limit or clock parsers. Disposable does not mean unread.

Who should not use this pattern

Skip the shared-host rules when the binary never leaves one laptop. Skip the product notes when no free lane sits on the path. Do not skip the self-graded test rule.

A model that writes its own passing tests is a weak witness. That rule survives without any vendor in the path. It also survives a perfect local suite score.

Staff who need a production region should not improvise. A contract or a named quota is a different lane. Do not stretch a free option into that role.

Ask the current primary docs for access terms. If those docs are silent, stay on an isolated reviewed path. Silence is not a deploy permit for anyone.

Put the hold script beside the deploy notes. Fill a manifest for the next parser or clock change. Confirm current free-access terms before any scratch host is chosen.

Keep the ship button on a human hand. Leave the model on the draft side of that line.

When a Cleanup Label Must Not Come from a Free Model

Casey Sun — Fri, 09 Oct 2026 13:17:52 +0000

A night job once treated a customer export as scratch. The label came from a free model on a borrowed host. The worker removed that export before the first page.

The scene above is a composite, not a measured outage. This account did not run that cleanup job. The same shape still appears in shared cleanup paths.

What the path got wrong

The classifier had no written retention contract to follow. The worker trusted one string returned by the model. The host kept no durable record of that choice.

Deletion is a state change without a cheap undo. A free model lane can drift, throttle, or disappear. A free server can sleep, reset, or lose local disk.

Those limits are acceptable for a draft notebook. They are unacceptable for a live delete decision. The label and the delete must not share one hop.

Red flags

One prompt both labels a prefix and enqueues the delete.
The worker role can delete outside a reviewed name list.
Approval state lives only on the free host disk.
A model outage blocks restore, not only new labels.
Prefix rules live in a chat log, not versioned policy.
The job retries a delete after the label text changes.
Apply starts before a controlled signer accepts the plan.
Planned names are not checked against a fresh inventory.

Better alternatives

Keep drafter, signer, and applier on separate systems. A drafter may suggest names that look unused. A signer must accept the exact name list.

An applier may delete only those signed names. Place the signer on the audit system already in use. Place the applier on a host with a stable identity.

Retain the decision log outside the applier host. A free server may hold a scratch notebook. It must not hold the only approval record.

It must not hold the only applier binary either. A restart must not erase the reason a prefix died. Write that reason to the audit system before apply.

Four breaks that stack

Four breaks usually stack in this incident shape. Policy lived in a prompt instead of a repo file. The model lane and the delete role were joined.

The approval file sat on a disk that can reset. No inventory diff ran after the worker finished. Remove the joined role and a bad label cannot delete.

That split is the point of the local gate. The gate does not forgive a wide cloud role. Narrow the role even when every label looks correct.

A six-step workflow

Run this workflow on an owned account first. Keep the free lane out of steps three and four. Treat every step as required, not as optional.

Export a prefix inventory from the owned account.
Draft candidate names with a script or a model.
Drop any plan that fails the local gate below.
Require a signer to freeze the exact file hash.
Apply deletes from an owned host with a retained log.
Compare remaining keys with the signed inventory.

A model may help only in step two. Step two must not receive raw object bodies. Send prefixes and sizes, never secrets or payloads.

Run a local gate

The script below is an unexecuted local example. It does not call a model or a cloud API. It only rejects a JSON plan with unsafe labels.

Save the checker in the work directory as delete_gate.py. Save the two sample plans in that same directory. Run the three commands from that same directory.

python3 delete_gate.py bad.json; echo "exit:$?"
python3 delete_gate.py good.json; echo "exit:$?"
python3 -m unittest -v test_delete_gate

A reject result is the expected outcome for bad.json. An accept result is the expected outcome for good.json. The first two exit codes are 2 and 0.

#!/usr/bin/env python3
"""Unexecuted example. Refuse a delete plan owned by a free lane."""

import json
import sys

FORBIDDEN_LANES = {"free", "free-model", "free_model"}
FORBIDDEN_HOSTS = {"free-server", "free_server", "ephemeral"}

def review(plan):
    errors = []
    lane = str(plan.get("classifier_lane", "")).lower()
    host = str(plan.get("applier_host_class", "")).lower()
    if lane in FORBIDDEN_LANES:
        errors.append("classifier lane must not be free")
    if host in FORBIDDEN_HOSTS:
        errors.append("applier host must not be free")
    if not plan.get("approval_ref"):
        errors.append("missing approval_ref")
    names = plan.get("object_names") or []
    if not names:
        errors.append("empty object_names")
    if len(names) != len(set(names)):
        errors.append("duplicate object_names")
    policy = str(plan.get("policy_version") or "")
    if (not policy) or policy.startswith("chat:"):
        errors.append("policy_version must be versioned")
    if plan.get("inventory_count") != len(names):
        errors.append("inventory_count mismatch")
    return errors

def main():
    if len(sys.argv) != 2:
        print("usage: python3 delete_gate.py plan.json")
        return 1
    with open(sys.argv[1], encoding="utf-8") as handle:
        plan = json.load(handle)
    errors = review(plan)
    if errors:
        print("reject")
        for item in errors:
            print("- " + item)
        return 2
    print("accept")
    return 0

if __name__ == "__main__":
    sys.exit(main())

Sample plans

The bad plan sets classifier_lane to the free value. The host class is owned, and a change id exists. The gate still rejects the plan on lane alone.

The good plan uses owned for both lane and host. The object names are unique and match inventory_count. The policy tag is a version, not a chat link.

{
  "classifier_lane": "free",
  "applier_host_class": "owned",
  "approval_ref": "chg-1842",
  "object_names": ["scratch/job-1.tmp"],
  "inventory_count": 1,
  "policy_version": "retention-2026-10"
}

{
  "classifier_lane": "owned",
  "applier_host_class": "owned",
  "approval_ref": "chg-1842",
  "object_names": ["scratch/job-1.tmp"],
  "inventory_count": 1,
  "policy_version": "retention-2026-10"
}

A chat prefix on policy_version is a hard fail. Chat logs are not a retention contract for deletes. Put the contract in the repo, then cite the tag.

The shell block writes both plans in a temp directory. Copy the Python files into that directory first. Then run the three commands from the earlier block.

mkdir -p /tmp/delete-gate
cd /tmp/delete-gate
cat > bad.json << 'EOF'
{
  "classifier_lane": "free",
  "applier_host_class": "owned",
  "approval_ref": "chg-1842",
  "object_names": ["scratch/job-1.tmp"],
  "inventory_count": 1,
  "policy_version": "retention-2026-10"
}
EOF
cat > good.json << 'EOF'
{
  "classifier_lane": "owned",
  "applier_host_class": "owned",
  "approval_ref": "chg-1842",
  "object_names": ["scratch/job-1.tmp"],
  "inventory_count": 1,
  "policy_version": "retention-2026-10"
}
EOF

Tests worth keeping

Save the test module locally as test_delete_gate.py. It imports the gate and checks four paths. The unit test command should report four passes.

import unittest
import delete_gate

BASE = {
    "classifier_lane": "owned",
    "applier_host_class": "owned",
    "approval_ref": "chg-1842",
    "object_names": ["scratch/job-1.tmp"],
    "inventory_count": 1,
    "policy_version": "retention-2026-10",
}

class ReviewTests(unittest.TestCase):
    def test_rejects_free_classifier(self):
        plan = dict(BASE, classifier_lane="free")
        errors = delete_gate.review(plan)
        self.assertIn("classifier lane must not be free", errors)

    def test_rejects_free_host(self):
        plan = dict(BASE, applier_host_class="free-server")
        errors = delete_gate.review(plan)
        self.assertIn("applier host must not be free", errors)

    def test_rejects_chat_policy(self):
        plan = dict(BASE, policy_version="chat:latest")
        errors = delete_gate.review(plan)
        self.assertIn("policy_version must be versioned", errors)

    def test_accepts_split_roles(self):
        self.assertEqual(delete_gate.review(dict(BASE)), [])

if __name__ == "__main__":
    unittest.main()

Why the count check exists

The count blocks a silent truncated name list. A partial list can hide the export prefix. The gate only checks that the plan is self-consistent.

Pair the gate with a server-side list before apply. The applier must list the bucket, then compare names. A mismatch stops the job before the first delete.

Decision table

Use the table as a routing guide, not a score. A single failing row is enough to block apply. Do not average the rows into a soft pass.

Signal	Free-lane draft	Free-server apply	Route
Suggest unused prefixes	Yes, without object bodies	No	Draft only
Store the approval record	No	No	Audit system
Delete signed names	No	No	Owned applier
Retry after a new label	No	No	New signed plan
Hold the only decision log	No	No	External audit log
Scratch notes on policy	Yes	Scratch disk only	Not the applier

Exit checks

Leave even the draft lane when any check below holds. Meeting one check means the team should stop. Move drafts back to a reviewed local script.

Prompts must include raw object bytes or secrets.
The team cannot keep a signer outside the model.
Measured restore time already misses the recovery target.
Policy text changes faster than the signed plan.
The free host is the only place logs survive.
A vanished lane would block a required export.
The delete role can still act without the gate.

Do not weaken the gate to force an accept. Fix the plan, then run the same command again. A second label from the model is not a signature.

Limits of the gate

The gate reads labels that the plan itself supplies. It does not inspect cloud identity or bucket rules. A lying plan can still fake the inventory count.

Keep object versioning enabled on buckets that matter. Treat this file as a seatbelt, not a vault. The example was not executed in this draft.

Run the files on a laptop before trusting the exit codes. Confirm the four tests pass on that laptop. Then wire the checker into the owned apply job.

Who should skip this pattern

Skip the gate if no production delete job exists. A personal toy bucket with disposable files needs less process. Still avoid a wide delete role on a free host.

Do not use the file as a compliance substitute. Regulated retention needs the control that counsel already chose. This checker does not certify a retention program.

Skip free-lane drafts if prompts would hold secrets. A local rules file is the better drafter then. The free server is the wrong home for that rules file.

Where a free option can still sit

A free model lane can still draft a candidate name list. A person or an owned job must sign that list. The free server, if used, stays a scratch pad.

This draft uses MonkeyCode as the named free-lane option. Disclosure: This article was prepared as part of MonkeyCode's product outreach. The operator describes free model access and a free server option.

This draft does not treat either offer as a quota promise. No model name, duration, or hardware claim is made here. Use that lane for unused-prefix drafts, not for the applier.

Confirm current terms before any workflow depends on it. If terms are unclear, keep drafts on a local script. The delete path should not wait on a vendor status page.

The Free-Lane Admission Test: Four Checks Before an AI Feature Ships on a Free Model

Casey Sun — Thu, 08 Oct 2026 09:22:48 +0000

Agent demos are everywhere this month. Most of them run on a free model lane somewhere.

Free lanes are not the problem. Shipping on one without an admission gate is.

This is a field guide for the opposite question. Not "how do I get on a free lane" but "what proves I should not be on one yet."

The Friday deploy that doubled every refund

A two-person team wired a triage agent to a free model lane. The model classified tickets and triggered refunds. Local tests passed. Nobody tested the retry path.

On Monday, finance found 41 duplicate refunds. The model had returned valid JSON both times the worker retried. It answered correctly. The integration had no gate.

That is the recurring shape of these incidents. The lane behaves. The caller absorbs the failure.

A paid lane with a contract puts a support engineer behind the failure. A free lane leaves it with whoever called it.

Where a free lane genuinely earns its place

Free model access is a real fit for some workloads. The pattern is consistent.

Low-stakes classification where a wrong label costs a re-run, not money.
Draft generation that a human edits before anything downstream reads it.
Batch enrichment over data you can recompute cheaply.
Prototypes where the goal is to learn whether the feature is worth building.

Four red flags that should stop the deploy

The output directly triggers a money movement, deletion, or signed callback.
The lane is asked to classify its own retryability instead of a deterministic rule doing it.
The caller's timeout budget is smaller than the lane's tail latency.
Business state must be inside the prompt for the answer to be correct.

Any single flag is enough to pause. Two flags means the design is wrong, not the lane.

A runnable admission test

The harness below replays frozen inputs against any lane and fails the build on three gates. It runs without network access against a local adapter. Swap one function to point it at a live endpoint.

// free-lane-admission.mjs
// Usage: node free-lane-admission.mjs fixtures/triage.jsonl
// Node 20+. No dependencies.
import { readFileSync } from "node:fs";

const REPLAY_PATH = process.argv[2] ?? "fixtures/triage.jsonl";
const GATE = { schemaRate: 0.995, p99Ms: 2500, effectKeyRate: 1.0 };

const cases = readFileSync(REPLAY_PATH, "utf8")
  .split("\n")
  .filter((line) => line.trim().length > 0)
  .map((line) => JSON.parse(line));

// Swap this body for a live call. It runs twice per case on purpose.
async function callLane(input) {
  // Unrun live example:
  // const r = await fetch(LANE_URL, { method: "POST", body: JSON.stringify(input) });
  // return r.json();
  return { label: "billing", effect_key: `refund:${input.ticket_id}` };
}

const SCHEMA = {
  label: (v) => ["billing", "bug", "howto"].includes(v),
  effect_key: (v) => typeof v === "string" && v.length > 0,
};

function schemaOk(reply) {
  return Object.entries(SCHEMA).every(([key, test]) => test(reply?.[key]));
}

function p99(samples) {
  const sorted = [...samples].sort((a, b) => a - b);
  const idx = Math.ceil(sorted.length * 0.99) - 1;
  return sorted[Math.max(0, idx)];
}

const latencies = [];
let schemaPass = 0;
let keyStable = 0;

for (const testCase of cases) {
  const t0 = performance.now();
  const first = await callLane(testCase.input);
  const second = await callLane(testCase.input);
  latencies.push(performance.now() - t0);

  if (schemaOk(first) && schemaOk(second)) schemaPass += 1;
  if (first.effect_key === second.effect_key) keyStable += 1;
}

const total = cases.length;
const result = {
  cases: total,
  schema_rate: schemaPass / total,
  effect_key_rate: keyStable / total,
  p99_ms: p99(latencies),
};

const failures = [];
if (result.schema_rate < GATE.schemaRate) failures.push("schema_rate");
if (result.effect_key_rate < GATE.effectKeyRate) failures.push("effect_key_rate");
if (result.p99_ms > GATE.p99Ms) failures.push("p99_ms");

console.table(result);
if (failures.length > 0) {
  console.error("BLOCKED on: " + failures.join(", "));
  process.exit(1);
}
console.log("ADMITTED: lane may carry this workload behind a queue.");

Exit code 1 is the useful part. It belongs in CI next to the unit tests.

What each gate is actually measuring

schema_rate catches format drift before it reaches a parser. A free lane can change behavior between deploys without notice.
effect_key_rate catches non-idempotent side effects. Calling the lane twice with the same input must produce the same effect key.
p99_ms catches the timeout trap. A lane that is fast on average and slow at the tail will exhaust retries during an incident.

What stays out of the prompt

Retry classification must be a rule table, not a model decision. The mapping below is boring and correct.

Transport signal	Classification	Caller action
Connection reset before headers	Retryable	Retry with jitter, cap 3
HTTP 429	Retryable	Back off, respect `Retry-After`
HTTP 400 with schema error	Non-retryable	Dead-letter, alert
Timeout after request sent	Unknown	Do not blind-retry writes

Exit criteria: when to leave the free lane

Adoption without exit criteria is how a prototype becomes an outage. Write these down before launch.

Schema conformance falls below 99.5% on two consecutive days.
p99 latency exceeds half the caller's timeout budget.
Any duplicate effect key appears in production logs.
Token spend per request grows threefold with no feature change.
The workload starts needing business state inside the prompt.

When a threshold trips, the migration is usually one of four moves. Move binary decisions to deterministic rules. Move high-volume low-variance work to a self-hosted small model. Move anything with an SLA to a paid lane with a contract. Move repeated inputs to a cache keyed on normalized input.

Who should not use this approach

Teams without a frozen replay set. The harness measures nothing without one.
Products where a wrong classification is a compliance event.
Workloads whose inputs change shape weekly. Fixtures rot faster than the gate pays off.
Anyone who cannot run a queue in front of the lane.

Limits of the test

The harness measures conformance, idempotency, and tail latency. It does not measure answer quality.

A lane can pass every gate and still produce wrong labels. Quality needs a labeled sample and a human review loop. That is a separate artifact, and it does not fit in a CI job.

Free lanes also change. Terms, rate limits, and model routing move without a changelog. The admission test is a snapshot, not a permanent clearance. Re-run it after any lane-side change.

MonkeyCode in this workflow

The operator describes MonkeyCode as an open-source project that offers free model access, a free token allowance, and a free server option.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Those claims are operator-supplied. Confirm current limits on the project's own page before sizing anything against them.

The admission test above is deliberately lane-agnostic. It is the same gate whether the lane is free, self-hosted, or paid. Run it on a free lane to find out what the lane cannot carry yet, not to prove it works.

That inversion is the point. If the harness reports ADMITTED, the free lane has earned a spot behind a queue. If it reports BLOCKED, the harness just saved a weekend.

Readers who want to try the free lane can point the callLane body at it and run the fixtures they already have.

A Legal Hold Does Not Expire Inside a Prompt

Casey Sun — Thu, 08 Oct 2026 06:40:45 +0000

A night operator approved a retention purge at 02:14 UTC. The plan came from a free model lane on a free server. A legal hold still covered one export bucket.

The job deleted three prefixes before a human stopped it. This composite case is a teaching story, not a customer report. No vendor log is cited as evidence here.

The boundary that failed

Retention deletes are not ordinary drafting tasks today. They destroy data that a later audit may require. A model can summarize a written retention policy.

It must not release a legal hold by itself. Free model access is useful for cheap text drafts. A free server is useful for isolated sandbox runs.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. Those two options do not change the control owner. The approve step stays on an audited path.

The free lane may only prepare text notes. It never receives live delete credentials from ops. Any state change stays outside that free lane.

How the teaching case broke

The classifier saw the hold flag and still returned allow. The same run id was copied into the approver field. Production prefixes sat beside a short sandbox note.

No second person checked the hold service first. Delete credentials lived in the same draft environment. The window opened because the plan looked complete.

That sequence is the full anti-pattern here. A fluent plan is not an approval record. A green model score is not a hold lookup.

Red flags

Stop the job when any item below is true. One flag is enough to halt the run. The fix does not belong in the same prompt.

The source lane equals free, or the field is missing.
The approver kind is model, or it copies a run id.
Any listed object still has legal hold set true.
Hold expiry is blank on a regulated export prefix.
A hold timestamp still sits in the future.
The target is production or a shared staging bucket.
Delete credentials sit beside the draft environment.
One job both classifies holds and calls the delete API.
No second human is recorded before the window opens.
Someone asks a prompt to reinterpret a failing code.

A human clears the flag in the system of record. The same prompt must not clear that flag. A second lane records the human approver id.

Field contract

A purge plan needs a small and boring schema. Extra prose from a model is ignored by the gate. Unknown fields must not grant any access.

Missing required fields must fail closed instead. The checker below follows that closed rule. Operators keep the schema in version control.

Required plan fields

The source_lane value must be audited or free.
The approver_kind value must be human or model.
The target_env value must be sandbox or production.
The objects value must be a list of key records.

Required object fields

The key is a relative prefix, never a credential URL.
The legal_hold flag is a boolean from the hold service.
The hold_until value is an ISO-8601 time, or null.

Decision table

Use this table before any purge command runs. Treat the rows as team policy, not model advice. The free lane loses every row that changes state.

Signal	Free lane may	Audited path must
Policy summary	Draft plain language	Store the signed version
Hold lookup	Read a local fixture	Query the system of record
Plan JSON	Propose a shape	Reject or accept the shape
Credential use	Never receive a token	Issue a short-lived human token
Delete API	Never call it	Call only after two checks
Exception note	Suggest wording	File the note under a human id

Mixing those columns recreates the teaching case. Draft text may move from left to right only. State change may not move back into the free lane.

Reason codes

Map each rejection to one fixed operator action. Do not invent a code that means model override. Override codes become the next quiet incident.

free_or_missing_lane means move the plan to the audited path.
approver_not_human means record a named human approver.
target_not_sandbox means drop production keys from the draft.
A hold code means refresh that flag from the source system.
hold_open means wait, or get a written counsel release.

A local checker

The script below is an unexecuted example only. It does not call a cloud delete API. It only accepts or rejects a local JSON plan.

#!/usr/bin/env python3
# Reject retention plans that a free lane must not approve.

import json
import os
import sys
from datetime import datetime, timezone

def parse_time(value):
    if not value:
        return None
    return datetime.fromisoformat(value.replace("Z", "+00:00"))

def current_time():
    override = os.environ.get("REVIEW_NOW")
    if override:
        return parse_time(override)
    return datetime.now(timezone.utc)

def review(plan, now):
    reasons = []
    lane = plan.get("source_lane")
    if lane in (None, "", "free"):
        reasons.append("free_or_missing_lane")
    if plan.get("approver_kind") != "human":
        reasons.append("approver_not_human")
    if plan.get("target_env") != "sandbox":
        reasons.append("target_not_sandbox")
    for item in plan.get("objects", []):
        key = item.get("key", "?")
        if item.get("legal_hold") is True:
            reasons.append("hold:" + key)
        expiry = parse_time(item.get("hold_until"))
        if expiry is not None and expiry > now:
            reasons.append("hold_open:" + key)
    return reasons

def main():
    plan = json.load(sys.stdin)
    now = current_time()
    if now is None:
        print(json.dumps({"allow": False, "reasons": ["bad_review_now"]}))
        return 2
    reasons = review(plan, now)
    if reasons:
        print(json.dumps({"allow": False, "reasons": reasons}))
        return 2
    print(json.dumps({"allow": True, "reasons": []}))
    return 0

if __name__ == "__main__":
    sys.exit(main())

The operator saves the file as review_purge.py. The operator runs it only on local fixtures. Production credentials stay off that review machine.

Fixture commands

These commands are a test plan, not a measured run. This draft did not execute any of the commands. No live key belongs in either fixture file.

mkdir -p fixtures
cat > fixtures/bad_free_lane.json <<'EOF'
{
  "source_lane": "free",
  "approver_kind": "model",
  "target_env": "production",
  "objects": [
    {
      "key": "exports/2026-04/a",
      "legal_hold": true,
      "hold_until": "2026-12-01T00:00:00Z"
    }
  ]
}
EOF
REVIEW_NOW=2026-10-08T00:00:00Z python3 review_purge.py < fixtures/bad_free_lane.json
echo "exit:$?"

The expected result is allow false and exit code 2. Reasons should name the lane, approver, target, and hold. A future hold time also adds hold_open.

The commands set REVIEW_NOW so the result ignores the wall clock. A bad override should fail closed with bad_review_now. After the sample date, keep the override before hold_until.

cat > fixtures/sandbox_human.json <<'EOF'
{
  "source_lane": "audited",
  "approver_kind": "human",
  "target_env": "sandbox",
  "objects": [
    {
      "key": "exports/scratch/b",
      "legal_hold": false,
      "hold_until": null
    }
  ]
}
EOF
REVIEW_NOW=2026-10-08T00:00:00Z python3 review_purge.py < fixtures/sandbox_human.json
echo "exit:$?"

The second fixture covers a narrow sandbox pass. The expected result is allow true and exit code 0. That pass still deletes nothing on any host.

A later job on another host needs its own approval. The sandbox pass does not travel with the plan. Copying a green JSON file is not a new approval.

Where a free lane still helps

The free lane can still do low-risk text work. It must not cross into a state change. Only four tasks stay inside that safe limit.

Turn a retention policy into short operator notes.
Generate extra bad fixtures for this local checker.
Explain a failed reason code to a new on-call reader.
Diff two plan shapes before a human review meeting.

A free server option can host the checker and fixtures. The operator confirms current terms before any drill. Delete tokens never land on that host.

Better alternatives

Match the control to the data class in play. A free lane is not the default owner. Use one of the narrower paths listed below.

Regulated exports stay behind the records system that owns holds.
Ordinary application logs use a storage lifecycle rule.
Scratch data stays in a sandbox bucket with no holds.
Uncertain keys wait while a human receives the page.
Model help stays on text, with no delete client installed.

A paid or internal control plane should own delete tokens. A free server should not store those tokens. A free model should not see them in a prompt.

Exit criteria

Leave the free lane when any line below becomes true. A real exit moves classification and approval together. Partial exits recreate the same control hole.

The plan targets production or a shared staging bucket.
A legal, finance, or security hold appears on any key.
The checker and the delete client share one secret file.
On-call expects the model to choose the purge window.
Fixture tests fail, and someone wants a prompt override.
Free-lane limits or uptime no longer match the drill.
An auditor asks who approved a delete, and hears a run id.

Limitations

This checker trusts the JSON fed into stdin. A false plan can omit the legal hold flag. The script cannot see the real storage bucket.

It cannot prove the hold service is current. ISO-8601 is the only time form in this example. Other time forms should fail closed in a fork.

No quota, hardware, or uptime figure is claimed here. This account has no fresh primary measurement of those terms. Operators should read current product docs before a drill.

The opening story is a composite teaching case. It is not an incident record or a log extract. It is not a performance benchmark of any kind.

Who should skip this pattern

Some teams should not adopt this split yet. A checker cannot invent a missing process owner. A free model lane will not replace counsel.

No second person exists to approve the purge.
A storage lifecycle product already owns every delete.
Counsel has not defined what a hold means.
The only environment available today is production.
Operators would rather prompt past a failing test.

A narrow next step

The operator copies the script into a scratch repo. The operator runs both fixtures on a free server. Live delete credentials stay off that sandbox host.

Teams can use MonkeyCode's free model access here. The free server option can host these fixtures. The approve path stays on an audited system.

If that split feels awkward, stop the drill. Awkwardness is the signal, not a prompt bug. Keep the next change on the audited path.

Renewing a Lock Lease Is Not a Prompt Task

Casey Sun — Thu, 24 Sep 2026 19:20:38 +0000

A billing worker woke a second replica at 02:10. Both replicas believed they held the nightly export lock. The second replica had renewed a lease it never owned.

This walkthrough uses a constructed billing export incident. No customer metric, vendor quota, or outage duration is claimed here. The failure mode remains common in agent-backed workers.

How the constructed incident unfolded

First holder

The first worker acquired the export lock at 02:00. Its fencing token was an integer from the lock service. A deploy restarted that worker before the export finished.

Bad advice

The assistant read a stale timestamp from one log line. It told the new replica to renew the lease immediately. The prompt never received the current fencing token.

Resulting write

The new replica wrote a second invoice batch. Operators reversed that batch by hand later that morning. The model had treated a lock as prose.

Failure analysis

Three independent mistakes stacked in the constructed story. The assistant acted on a timestamp instead of a token. The replica accepted prose as an authorization to write.

The platform let a completion path sit beside the lock client. Removing that path leaves the lock service as sole writer.

The boundary this guide defends

Lease renewal is a fencing decision, not a completion. A fencing token must come from the lock service. A model completion must not mint that token.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode offers free model access and a free server option. Those claims are operator-supplied availability notes, not measured limits.

Current quotas, hardware, and duration belong in current product docs. This guide deliberately states none of those numbers.

A free model can draft a test or explain a log. It must not decide who holds a lease.

A free server may host only a lab harness. The operator must keep that host off production networks.

Red flags

Teams should watch for these signs before any free-lane trial. Any single sign below is already a stop condition.

A prompt returns renew, steal, or extend as an action.
The same completion selects a tenant and a lock name.
Lease duration arrives as model text rather than config.
A free server process can reach the production lock API.
Fencing tokens are parsed from unstructured model output.
Retry advice and lease advice share one prompt.
Operators lack a kill switch outside the model path.
The lab host still carries a production lock credential.

Any one of those signs is enough to stop. Combined signs mean the design is already unsafe.

What the free lane may still do

Drafting lane

The free model lane may summarize a redacted timeout log. It may draft a unit test for the deny gate. It may list unfamiliar fields in a lock response.

Isolated lab host

The free server may run that harness in isolation. Isolation is an operator duty, not a default product promise.

The lab host should have no route to the production lock API. It should also avoid any shared secret store with production.

Neither lane may choose renew, steal, or release. Neither lane may invent a fresh fencing token.

Better alternatives

The lease path should stay boring, local, and deterministic. The steps below keep renewal out of completions.

A real lock service should issue every fencing token.
Lease deadlines should live in that service, not in prompts.
Application code should compare clocks and fencing tokens.
The model should see only redacted post-incident log text.
Explanatory drafts should stay on a separate free model lane.
The harness should run on a free server that cannot call production.
A human should merge every lease-policy change.

The model may describe a failure after the fact. The lock service still owns the next renewal.

Decision table

The table below is a policy, not a benchmark. No latency, cost, or uptime figure appears in it.

Signal in hand	Owner	Free model	Free server
Fencing token check	Lock client	Deny	Deny
Lease deadline math	Lock client	Deny	Deny
Steal or renew action	Lock service	Deny	Deny
Redacted log summary	Drafting lane	Allow	Optional
Harness unit tests	Isolated lab	Draft only	Allow if isolated
Production lock API	Lock service	Deny	Deny

Isolation means no production credentials on that host. Optional means the free server is unnecessary for a pure draft.

A local deny gate

The following Python is a proposal, not a live run. It was not executed against a live lock service. It shows the shape of a local deny gate.

Shape check

from dataclasses import dataclass

ALLOWED_DRAFT_TASKS = frozenset({"explain_log", "draft_test"})
LEASE_ACTIONS = frozenset({"renew", "steal", "extend", "release"})

@dataclass(frozen=True)
class LeaseRequest:
    action: str
    fencing_token: str
    source: str

def admit_lease(req: LeaseRequest) -> bool:
    if req.source != "lock_client":
        return False
    if req.action not in LEASE_ACTIONS:
        return False
    if not req.fencing_token.isdigit():
        return False
    return True

def admit_draft(task: str, body: str) -> bool:
    if task not in ALLOWED_DRAFT_TASKS:
        return False
    lowered = body.lower()
    banned = LEASE_ACTIONS | {"fencing", "lease_until"}
    return not any(word in lowered for word in banned)

The trusted lock client alone should call admit_lease. A draft gate should run before any free model sees text. A draft that contains renew or steal never leaves the gate.

Renewal helper

def renew_from_client(lock, token: str, now: int) -> bool:
    if lock.fencing_token != token:
        return False
    if now >= lock.lease_until:
        return False
    lock.lease_until = now + lock.lease_ttl
    return True

This helper never reads model output or log prose. The token argument must already come from the lock service.

The lease_ttl value should come from reviewed config. It should never be parsed from a completion.

Lab commands

Operators should run these checks in a disposable lab environment. These commands should not target production hosts.

python -m pytest tests/test_lease_gate.py -q
git grep -n "renew" -- app/locks app/prompts
git grep -n "fencing_token" -- app/prompts
env | awk -F= '/LOCK|FENCING/ {print $1}'

A prompt hit on fencing_token is a red flag. A lock-client hit on renew can be legitimate. The directory split is the actual code review.

An environment hit on a lock secret name fails the lab. The free server image should fail that same check.

Unexecuted test plan

Teams should treat this plan as unexecuted until a local run records output. That recording belongs next to the git revision.

A model completion source should produce denial.
An empty fencing token should produce denial.
A draft that says renew the lease should produce denial.
A redacted timeout summary should produce allow.
The lab host should hold no production lock credentials.
The free server image should lack a route to the lock API.
Unknown fields in a model completion should fail closed.
A paraphrased keep the hold alive draft should fail closed.

Teams should not invent pass rates before that local run. A green local run still does not prove production safety.

Exit criteria

Teams should leave the free lane when any item below becomes true. Leaving is a design change, not a prompt tweak.

A completion can call the lock API, even indirectly.
Lease duration is no longer a reviewed config value.
The free server shares a network with production locks.
More than one service trusts model text for fencing.
An incident review cannot name the token issuer.
Docs and code disagree on who may steal a lease.
Quotas or uptime needs exceed published free-tier terms.
A single prompt both drafts tests and renews leases.

Exit means moving lease control fully into the lock client. Drafting help can remain on the free model lane. The free server should keep hosting only the isolated harness.

Who should not use this approach

Teams without a lock service should not start here. Teams whose one process must both draft and renew should stop. Payment capture, inventory decrement, and mailbox migration should stay out.

Those paths need this boundary plus a named domain owner. Teams should also stop if current docs contradict the availability notes.

Availability language goes stale and must be rechecked. A code review habit does not go stale the same way.

Limitations

This gate checks request shape, not distributed correctness. Clock skew, session loss, and split brain need the lock service. A string deny list will miss some paraphrased hold requests.

Teams should pair that list with a typed action enum. Unknown fields from every model payload should be rejected. Denials should be logged without raw tenant payloads.

Free model access does not make a completion authoritative. A free server does not make a host trustworthy. Both remain optional lab tools whose limits operators must re-read.

The constructed incident is a teaching story, not a case study. No vendor benchmark is claimed for this deny gate.

Closing

Renewal, steal, and release should stay inside the lock client. A free model may explain the redacted log afterward. A free server may run only an isolated harness.

Operators should read current MonkeyCode docs before any free-lane trial. Every quota and hardware note there should be treated as time-bound.

When a Free Model Must Not Classify Retryability

Casey Sun — Wed, 23 Sep 2026 16:16:42 +0000

On a Monday deploy, a refund agent started auto-retrying ledger writes. Each 503 response was labeled retryable by a helper model. The queue doubled while the ledger stayed dark.

Customers then saw duplicate authorization holds by noon. The model had treated unavailability as safe to replay. Retryability was never a natural-language task for agents.

Retryability is a control-plane decision

A retry changes money, inventory, and on-call load. The classifier sits on the hottest failure path. Latency, determinism, and auditability all matter on that path.

A free model can draft fixtures and comments. It must not vote on any production retry. The live vote belongs to pinned status rules.

Tool-calling agents make this failure mode common. The model already chooses arguments for an HTTP tool. Teams then let it also label the tool error. That second vote is how duplicate POSTs are born.

Red flags before the model is wired in

Stop the integration if any item below is true.

The outbound call mutates money, inventory, or access rights.
The upstream documents Retry-After or required idempotency keys.
Duplicate execution is user-visible, billable, or legally binding.
The helper model is not pinned, logged, or replayable.
Backoff state lives only on an ephemeral free server.
On-call cannot explain a retry without reading a prompt.

A yes on any row means keep the model off the path. Incident chat is not an exception to that rule. Comfort text can wait until the worker has failed closed.

What the classifier is allowed to read

The live classifier may read only transport facts.

HTTP status code from the failed upstream attempt
Retry-After header when present and numerically valid
A caller-supplied idempotency key for mutating methods
Static method allowlist covering GET and keyed PUT
Per-route maximum attempt budget stored in config

The classifier must not read the response body as prose. It must not send that body to a model for labeling. Body text stays untrusted on the hot path. Status codes remain the only allowed retry contract.

Artifact: a pinned retry classifier

The following Node module is a proposal for local tests. It is not a production SLA for vendors. Teams should adapt the status map for each vendor.

// retryability.mjs
export const RETRYABLE_STATUS = new Set([408, 425, 429, 502, 503, 504]);

export const MUTATING = new Set(["POST", "PATCH", "DELETE"]);

export function parseRetryAfter(header, nowMs = Date.now()) {
  if (header == null || header === "") return null;
  const raw = String(header).trim();
  const asInt = Number.parseInt(raw, 10);
  if (Number.isFinite(asInt) && String(asInt) === raw) {
    if (asInt < 0 || asInt > 120) return null;
    return asInt * 1000;
  }
  const when = Date.parse(raw);
  if (Number.isNaN(when)) return null;
  const delta = when - nowMs;
  if (delta < 0 || delta > 120_000) return null;
  return delta;
}

export function classifyRetry(input) {
  const {
    method,
    status,
    retryAfterHeader,
    attempt,
    maxAttempts,
    idempotencyKey,
    source,
  } = input;

  if (source && source !== "pinned-map") {
    return { retry: false, reason: "unpinned-source", delayMs: 0 };
  }
  if (attempt >= maxAttempts) {
    return { retry: false, reason: "budget-exhausted", delayMs: 0 };
  }
  if (!RETRYABLE_STATUS.has(status)) {
    return { retry: false, reason: "status-not-retryable", delayMs: 0 };
  }
  if (MUTATING.has(method) && !idempotencyKey) {
    return { retry: false, reason: "mutating-without-idempotency", delayMs: 0 };
  }

  const hinted = parseRetryAfter(retryAfterHeader);
  const delayMs = hinted ?? Math.min(1000 * 2 ** attempt, 30_000);
  return { retry: true, reason: "pinned-status", delayMs };
}

The source guard against unpinned maps is the lesson. A model adapter cannot sneak in without failing closed. Backoff remains a numeric delay for every retry. Prose comments and hidden tool calls stay out.

Tests that fail if a model joins the path

Run the file with Node's built-in test runner.

// retryability.test.mjs
import { test } from "node:test";
import assert from "node:assert/strict";
import { classifyRetry, parseRetryAfter } from "./retryability.mjs";

const base = {
  method: "POST",
  status: 503,
  retryAfterHeader: null,
  attempt: 0,
  maxAttempts: 3,
  idempotencyKey: "refund-9f3c",
  source: "pinned-map",
};

test("503 with idempotency key may retry", () => {
  const out = classifyRetry(base);
  assert.equal(out.retry, true);
  assert.equal(out.reason, "pinned-status");
});

test("POST without an idempotency key must not retry", () => {
  const out = classifyRetry({ ...base, idempotencyKey: "" });
  assert.equal(out.retry, false);
  assert.equal(out.reason, "mutating-without-idempotency");
});

test("404 is never retryable", () => {
  const out = classifyRetry({ ...base, status: 404 });
  assert.equal(out.retry, false);
});

test("409 conflict is never retryable", () => {
  const out = classifyRetry({ ...base, status: 409 });
  assert.equal(out.retry, false);
});

test("unpinned source fails closed", () => {
  const out = classifyRetry({ ...base, source: "free-model" });
  assert.equal(out.retry, false);
  assert.equal(out.reason, "unpinned-source");
});

test("Retry-After seconds are capped", () => {
  assert.equal(parseRetryAfter("3"), 3000);
  assert.equal(parseRetryAfter("9999"), null);
});

test("response body never reaches classifyRetry", () => {
  const keys = Object.keys(base);
  assert.equal(keys.includes("body"), false);
  assert.equal(keys.includes("prompt"), false);
});

node --test retryability.test.mjs

A green run does not prove production safety. It only proves the classifier rejects unpinned sources. That is the contract this article cares about.

Decision table

Signal	Live retry vote	Offline model use
503 + idempotency key	Pinned map may retry	Draft extra fixtures
409 conflict	Never retry	Summarize runbooks
429 with Retry-After	Honor capped delay	Not consulted
500 on POST, no key	Never retry	Generate negative tests
Timeout, unknown status	Never retry	Propose new map rows
Free server preemption	Fail closed	Rebuild the harness

The model column stays as optional offline work. The left column remains the only production vote. Do not merge the columns to save a round trip.

How a model sneaks onto the path

Most leaks arrive as a small helper during an incident. Someone pastes the 503 body into a prompt for comfort. The helper then gets imported into the worker loop.

The next module is an anti-pattern for review drills only. Do not ship it. Do not wrap it.

// do-not-ship.mjs — anti-pattern, review drills only
export async function classifyWithModel(res, promptClient) {
  const text = await res.text();
  const vote = await promptClient.complete(
    `Retry this ${res.status}? ${text.slice(0, 500)}`
  );
  return String(vote).toLowerCase().includes("yes");
}

That function reads prose and returns an unlogged boolean. It ignores idempotency, budgets, and Retry-After caps. Reviewers should reject any import of this shape.

A second leak is schema drift inside the tool layer. The agent regenerates an OpenAPI snippet after a 503. Required idempotency headers disappear from the new snippet. The next call looks unique to the ledger. Pin the tool schema the same way the status map is pinned.

Load tests will lie if retries are model-voted

Soak tests assume an authored traffic mix. A model-voted retry loop rewrites that mix under failure. One planted 503 can multiply a 100 POST script into four hundred attempts.

Percentiles then describe the retry storm, not the service. Error budgets then describe the storm as well. Keep the classifier pinned so the load profile stays authored.

Record attempt counts as a metric beside latency. Record reason from classifyRetry on every reject. A rising unpinned-source count is a ship-stop, not a dashboard curiosity.

# proposal: fail a soak when retry reasons drift
node soak.mjs --max-retry-ratio 0.05 --forbid-reason unpinned-source

The command is a local proposal, not a vendor benchmark. Teams still need their own traffic files and SLOs. The point is to measure retries as a first-class output.

Pull request checklist

Reject the change if any box fails.

Function arguments still exclude body and prompt fields.
New status codes include a vendor doc link in comments.
Mutating methods still require an idempotency key.
Tests cover 404, 409, 429, and unpinned source.
No new network client is imported by the classifier.
Worker hosting is not a scratch free server.

A checklist is slower than a prompt during a 503 burst. That slowness is the feature. Retry policy should be boring under pressure.

Where a free model still helps

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

MonkeyCode offers free model access and a free server option. Those lanes fit fixture drafting and isolated harness runs. They do not fit live retry classification at all. For that offline fixture loop, those free lanes are enough to try.

A useful split of duties looks like this.

Freeze the status map in source control today.
Ask a free model to propose missing negative tests.
Run those tests on a free server scratch box.
Copy accepted tests into the pinned suite afterward.
Keep the production worker on a stable host.

The free model never sees live 503 bodies. The free server never stores the attempt budget. Human reviewers still read every new map row.

Where the free server must not sit

A retry coordinator holds attempt counts and lease timers. A free server can vanish between two 503s. Lost timers become either silence or a stampede.

Keep the following components off the free server.

The worker that increments attempt counters on each 503
The durable store for idempotency keys still in flight
The clock used to honor Retry-After without drift
The publisher that re-queues delayed jobs after backoff

Scratch generation of tests can live there. The retry control plane cannot live there. Cold starts are acceptable for fixture jobs. Cold starts are not acceptable for Retry-After math.

Exit criteria

Move the live path off free lanes when any trigger fires.

Duplicate side effects appear in staging replay logs.
Source logs show any value except pinned-map.
p95 classify time exceeds the worker ack deadline.
On-call cannot recite the status map from memory.
A model-written test was merged without a human diff.
Backoff state was lost after a scratch host recycle.

Exit means code and hosting, not a prompt tweak. Restore the pinned map in the worker first. Restore a durable worker for the retry loop. Then delete the adapter that called the model.

Do not keep the adapter behind a feature flag for later. Flags drift. The next incident will flip the flag under fatigue. Deletion is the only honest exit.

Who should not use this approach

Teams should skip this split in some cases.

Purely local CLIs with no networked side effects at all
Read-only crawlers that already treat every error as fatal
Teams without a test runner or a review path
Protocols that already pin retryability in-band today

Those groups need less machinery, not a model vote. Do not add a classifier only for fashion. A fatal-on-any-error client is already a pinned policy.

Limitations

The status map above is incomplete on purpose. gRPC codes, WebSocket drops, and exactly-once queues need their own tables. Retry-After parsing here rejects long delays rather than scheduling them.

These tests do not measure any vendor uptime. They do not claim a model quality score. They only lock the hot path to deterministic inputs.

Status maps drift whenever vendors change error contracts. Revisit the table when an upstream publishes a new error catalog. Do not let a model apply that catalog live.

Idempotency keys have their own lifetime rules. This article does not define key expiry, hashing, or storage. A pinned retry vote still fails if keys collide or vanish.

Closing

Retry storms often start as helpful agent features. They end as duplicate charges and angry ledgers. Keep classification boring, pinned, and out of the prompt.

Free models can still write the tests that keep it boring. The production vote stays in the module above.