DEV Community: Harper Xu

A Free Server Is a Failure-Domain Map

Harper Xu — Tue, 27 Oct 2026 13:18:55 +0000

Free compute is rarely your bottleneck. Unowned failure is. Map the failure domains before you deploy on a free server.

That sounds abstract, so here is the concrete version. You get a machine you did not buy, running a model you did not train. Your job is not to trust it. Your job is to decide what breaks when it disappears.

Start with ownership, not features

Draw four boxes and label the owner of each.

client ──▶ gateway (you) ──▶ free server (borrowed) ──▶ model endpoint (borrowed)
              │                    │                          │
        deadlines, retries    process lifetime           model version
        logs, cache keys      disk, env vars             rate limits
        fallback path         restart policy             response shape

Only the second column is yours. Everything to the right can change without a changelog. You will not get a migration guide when that machine reboots.

So the review question is not "does the free server work?" It is "what does my gateway do while it does not?"

Borrowed disk and environment variables are state you never designed. Any file written there can vanish between requests. Any secret placed there can outlive your intent. Keep both out of the critical path.

Turn the review into drills

Opinions about failure are cheap. Reproductions are not. Write three drills and run them on a schedule.

# drills/run.sh — template; point BASE at your own gateway
set -u
BASE="${BASE:-http://localhost:8080}"
for drill in cold-start rate-limit truncation; do
  python drills/drill.py --target "$BASE" --drill "$drill" --runs 5
done

The driver stays small. It only injects one fault at a time.

# drills/drill.py — template, not executed against a live endpoint for this article
import random, time, urllib.request

def call(target, prompt, timeout):
    body = json.dumps({"prompt": prompt}).encode()
    req = urllib.request.Request(
        f"{target}/generate", data=body,
        headers={"Content-Type": "application/json"},
    )
    with urllib.request.urlopen(req, timeout=timeout) as resp:
        return resp.read().decode()

def cold_start(target, runs):
    for i in range(runs):
        time.sleep(random.uniform(0, 45))   # aim at an idle window
        t0 = time.monotonic()
        try:
            call(target, "warm up", timeout=30)
            print("cold-start", i, round(time.monotonic() - t0, 2), "s", "ok")
        except Exception as exc:
            print("cold-start", i, "fail", type(exc).__name__)

def rate_limit(target, runs):
    for i in range(runs):
        status = "ok"
        try:
            call(target, "burst", timeout=10)
        except Exception as exc:
            status = type(exc).__name__
        print("rate-limit", i, status)

def truncation(target, runs):
    for i in range(runs):
        try:
            out = call(target, "long answer please", timeout=20)
        except Exception as exc:
            print("truncation", i, "error", type(exc).__name__)
            continue
        ok = out.rstrip().endswith((".", "}", "!"))
        print("truncation", i, len(out), "terminated" if ok else "cut")

Run the rate-limit drill with concurrency, not a serial loop. A serial loop will pass and teach you nothing. That is the most common way this review produces false comfort.

Read the results as architecture

Cold start tells you about your first-paint budget. If an idle machine answers in tens of seconds, your synchronous endpoint is the bug. Move the call behind a queue, or keep a short warm path with a cheap probe.

Rate limit tells you who owns retries. If both the client and the gateway retry, you built a multiplier. Pick one owner, and give that owner a single deadline that all attempts share.

Truncation tells you whether you validate output. A response that stops mid-sentence is not an error code. Your parser must reject it explicitly, or your users become the validator.

Write the outcome down. A drill you ran once is an anecdote. A drill wired into CI is a constraint you can defend in review.

A decision table you can defend

Use plain rules, written before the incident, not after it.

Workload	Runs where	Reason
Interactive request	gateway, hard timeout, fail fast	someone is waiting
Batch summarization	queue, generous retry	nobody is waiting
Secrets or customer records	local only	boundary you do not own
Deterministic transforms	local, no model at all	model adds risk, not value

The last row saves the most money and the most incident reviews. Not every step deserves a model call.

What I would change next

Three changes, in order of payoff.

First, one owner for retries. The gateway retries. The client does not. Every attempt spends from the same budget.

# gateway.py — deadline-carrying wrapper (template)
import time

class Deadline(Exception):
    pass

def call_with_budget(fn, budget_s, attempts=3):
    end = time.monotonic() + budget_s
    for _ in range(attempts):
        left = end - time.monotonic()
        if left <= 0:
            raise Deadline("budget exhausted")
        try:
            return fn(timeout=left)
        except TimeoutError:
            continue          # same deadline, smaller remaining budget
    raise Deadline("attempts exhausted")

Second, a health probe that records the remote build. Log the version string you saw with each rollout. When response shape changes, you want a timestamp, not a guess.

Third, a fallback that is boring. Serve a cached answer or a clear error. Never silently swap in another model and pretend nothing happened. Quiet substitution turns a small outage into a data quality incident.

Each change is small. Together they turn a borrowed machine into a bounded dependency.

Limits and who should skip this

Drills measure your gateway, not the provider. They cannot promise availability, and they cannot detect a limit you never hit.

Free capacity can change shape without notice. Treat every number you measure as a snapshot with a date attached. Re-run the drills after any change on the far side.

Skip this approach entirely if you have data residency rules, sub-second latency targets, or contractual uptime. Rent capacity you can audit instead, and keep the drills anyway.

For a low-stakes sandbox, MonkeyCode provides free model access and a free server option. That is enough room to run these three drills before you point real traffic at anything.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Point BASE at your own gateway. Keep the drills in CI. Then you can replace the machine without losing sleep over it.

Treat the Free Lane as an Evictable Cache

Harper Xu — Mon, 26 Oct 2026 11:50:39 +0000

A free lane is a cache, not a contract. Cache it well and an eviction costs you a delay. Wire it as a dependency and an eviction costs you an outage.

I run a small batch summarizer over an internal corpus. It calls MonkeyCode's free model access and runs its worker on the free server option. Both are useful, and both are availability I did not buy. Disclosure: This article was prepared as part of MonkeyCode's product outreach. Check the plan page yourself before you size anything, because free tiers move.

The constraint comes first

You cannot promise what you do not control. Free capacity can be rate limited, rotated, or retired. Assume revocation without notice, at the worst hour.

So the first design question is not which model. It is what your job does when the lane closes. If the answer is "it raises", you built a dependency. Fix that before you write a single prompt.

The second question is what your job does when the answer arrives twice. Retries are cheap on your side and expensive on theirs. Duplicate work is how a free lane becomes a slow lane.

Data flow, one direction

The path is short on purpose. A producer writes jobs to a local queue file. A worker reserves budget, calls the lane, writes the result to disk, then commits.

The worker runs on the free server, but it holds no state you cannot rebuild. Its ledger is a SQLite file it owns. Its dead-letter file is append-only. Kill the box and you lose a lease, not a decision.

That split matters. Anything that can be reclaimed must be disposable. Anything that must survive lives in a file you can copy off the box in one command.

tar czf rescue.tgz ledger.db dead_letter.jsonl out/

Failure domains, ranked

The interesting failures are not crashes. They are slow, partial, and silent. Here is the table I actually review before each change.

Failure	How you see it	Blast radius	First move
Rate limit	HTTP 429 on a burst	One job, then all jobs	Open the breaker for a cool-off window
Upstream error	5xx or reset mid-body	Retry storm	Cap retries, jitter the backoff
Allowance spent	Ledger refuses the reserve	Whole lane	Stop the queue, keep the exit
Server reclaimed	Health probe goes quiet	Worker only	Rerun from ledger, do not re-charge
Partial write	Result file is truncated	One job	Write to `.tmp`, rename after fsync
Clock skew	`at` values go backwards	Audit only	Store monotonic deltas, not wall time

The third row is the one people miss. Quota exhaustion is not an error path you sprinkle on top. It is a state the worker must enter cleanly and leave cleanly.

The lane as code

Two classes are enough. One holds the budget. One holds the exit. Everything else is a caller.

"""lane.py - one free lane, one ledger, one exit."""
import json, sqlite3, time, urllib.error, urllib.request


class LaneClosed(Exception):
    """The lane is unavailable right now. That is a state, not a bug."""


class Lane:
    def __init__(self, url, cap_tokens=200_000, db_path="ledger.db",
                 fails_to_open=3, cooloff=60.0, timeout=20):
        self.url, self.cap, self.db_path = url, cap_tokens, db_path
        self.fails_to_open, self.cooloff, self.timeout = fails_to_open, cooloff, timeout
        self.fails, self.open_until = 0, 0.0

    def _db(self):
        db = sqlite3.connect(self.db_path)
        db.execute("CREATE TABLE IF NOT EXISTS spend ("
                   "job_id TEXT PRIMARY KEY, tokens INTEGER NOT NULL, at REAL NOT NULL)")
        return db

    def spent(self):
        db = self._db()
        try:
            return db.execute("SELECT COALESCE(SUM(tokens),0) FROM spend").fetchone()[0]
        finally:
            db.close()

    def reserve(self, job_id, tokens):
        """Meter first. Idempotent per job_id. Returns False at the ceiling."""
        db = self._db()
        try:
            total = db.execute("SELECT COALESCE(SUM(tokens),0) FROM spend").fetchone()[0]
            done = db.execute("SELECT 1 FROM spend WHERE job_id=?", (job_id,)).fetchone()
            if done:
                return True
            if total + tokens > self.cap:
                return False
            db.execute("INSERT INTO spend VALUES (?,?,?)", (job_id, tokens, time.time()))
            db.commit()
            return True
        finally:
            db.close()

    def seen(self, job_id):
        db = self._db()
        try:
            return db.execute("SELECT 1 FROM spend WHERE job_id=?", (job_id,)).fetchone() is not None
        finally:
            db.close()

    def call(self, payload):
        if time.time() < self.open_until:
            raise LaneClosed(f"breaker open for {self.open_until - time.time():.0f}s")
        req = urllib.request.Request(
            self.url, data=json.dumps(payload).encode(),
            headers={"Content-Type": "application/json",
                     "Idempotency-Key": payload["job_id"]})
        try:
            with urllib.request.urlopen(req, timeout=self.timeout) as resp:
                self.fails = 0
                return json.loads(resp.read())
        except urllib.error.HTTPError as err:
            if err.code == 429 or err.code >= 500:
                self._trip(f"http {err.code}")
            raise
        except (urllib.error.URLError, TimeoutError) as err:
            self._trip(repr(err))
            raise

    def _trip(self, why):
        self.fails += 1
        if self.fails >= self.fails_to_open:
            self.open_until = time.time() + self.cooloff
        raise LaneClosed(why)

The caller stays boring, which is the point. The worker checks, reserves, calls, and defers.

"""run.py - spend the lane, keep the exit."""
import json, os
from lane import Lane, LaneClosed

lane = Lane(os.environ["LANE_URL"], cap_tokens=int(os.environ.get("LANE_CAP", "200000")))


def run(job):
    if lane.seen(job["job_id"]):
        return {"job_id": job["job_id"], "state": "already_done"}
    estimate = len(json.dumps(job).split()) * 2          # rough on purpose
    if not lane.reserve(job["job_id"], estimate):
        return {"job_id": job["job_id"], "state": "budget_stop"}
    try:
        out = lane.call(job)
        with open(f"out/{job['job_id']}.tmp", "w") as fh:
            fh.write(json.dumps(out))
        os.replace(f"out/{job['job_id']}.tmp", f"out/{job['job_id']}.json")
        return {"job_id": job["job_id"], "state": "done"}
    except LaneClosed as why:
        with open("dead_letter.jsonl", "a") as fh:
            fh.write(json.dumps({**job, "why": str(why)}) + "\n")
        return {"job_id": job["job_id"], "state": "deferred", "why": str(why)}

Prove the exit with a fake lane

You do not test eviction against a live endpoint. You test it against a server that misbehaves on demand. Three tests cover most of the risk.

"""test_lane.py - no network, no vendor, just the failure modes."""
import http.server, json, threading
import pytest
from lane import Lane, LaneClosed


class Fake(http.server.BaseHTTPRequestHandler):
    statuses = [200]

    def do_POST(self):
        code = Fake.statuses.pop(0) if Fake.statuses else 200
        self.rfile.read(int(self.headers.get("Content-Length", 0)))
        self.send_response(code)
        self.end_headers()
        self.wfile.write(json.dumps({"ok": code == 200}).encode())

    def log_message(self, *args):
        pass


@pytest.fixture
def lane(tmp_path):
    srv = http.server.HTTPServer(("127.0.0.1", 0), Fake)
    threading.Thread(target=srv.serve_forever, daemon=True).start()
    yield Lane(f"http://127.0.0.1:{srv.server_port}/v1/chat", cap_tokens=100,
               db_path=str(tmp_path / "l.db"), fails_to_open=2, cooloff=30)
    srv.shutdown()


def test_two_429s_open_the_exit(lane):
    Fake.statuses = [429, 429]
    for _ in range(2):
        with pytest.raises(LaneClosed):
            lane.call({"job_id": "a"})
    with pytest.raises(LaneClosed, match="breaker open"):
        lane.call({"job_id": "b"})


def test_budget_is_a_hard_stop(lane):
    assert lane.reserve("j1", 60) is True
    assert lane.reserve("j2", 60) is False


def test_reserve_is_idempotent(lane):
    assert lane.reserve("j1", 10) is True
    assert lane.reserve("j1", 10) is True
    assert lane.spent() == 10

Run it with two commands. There is nothing else to install beyond pytest.

python -m pytest -q test_lane.py
>>> 3 passed

Now point LANE_URL at the free endpoint and replay the same drill. If the breaker holds, you own the exit. If the worker crashes on a 429, you do not.

What I would change next

The single queue is my weakest decision. Bulk backlog and fresh jobs compete for the same allowance. Split them, and let the fresh lane borrow only when the bulk lane is idle.

I would also cache by content hash before spending anything. Repeats are common in a corpus, and a hit costs zero. Right now the ledger sees them as fresh work.

A warm standby key is the third change. One paid fallback behind the same interface turns a hard stop into a slower day. It is not free, and that is exactly why it works.

Finally, I would sample egress logs weekly. Prompts leave your box and land somewhere else. Count what left, and diff it against what you meant to send.

Who should skip this

Skip this design if you have a hard latency SLO. A breaker is honest, but it is not fast. Skip it for regulated data that cannot leave your network boundary at all.

Skip it if you need per-token billing you can reconcile to the cent. A free allowance is fine for experiments and poor for accounting. Skip it if your job needs hardware you cannot reclaim in one command.

For everything else, the pattern holds. Assume eviction, keep the ledger, and make the exit boring.

If you want to watch a lane close safely before you trust it, start on MonkeyCode's free model access and free server, then run the three tests above against it. A lane you can lose is a lane you can actually use.

Meter First, Call Second: A Ledger for Free Inference

Harper Xu — Tue, 20 Oct 2026 09:58:18 +0000

Free inference does not fail on quality. It fails on accounting. You cannot subtract a number you never recorded.

Most teams wire a free lane in an afternoon. They add a key, a base URL, and a retry loop. Then a worker dies mid-call and the budget line goes quiet. Nobody knows what was spent, because nothing was written down.

This is an architecture review of the fix. Put a ledger in front of the lane. Record before you call, not after.

The constraints you actually have

You do not own the upstream. Quota can vanish between two requests, and the error text changes with it. You cannot plan capacity against a number you cannot read.

Workers are cheap and disposable. A process can die between the request and the response. Any budget held in memory dies with it, silently.

Retries are the default, not the exception. A timeout retry can double your spend on a call that already succeeded upstream. Your ledger has to survive that without human cleanup.

The free server option makes this sharper. Cold starts stretch tail latency, so timeouts fire more often. More timeouts mean more retries. That is the loop that eats a free allowance.

Data flow: reserve, call, settle

Keep one rule. No call without an open lease. The lease is a row, not a variable.

job ──▶ reserve(job_id, budget) ──▶ lease: open, ledger -budget
             │
             ├──▶ upstream lane (free model access)
             │
        commit(spent)  ──▶ ledger +(budget - spent)
        release()      ──▶ ledger +budget
        sweeper()      ──▶ release expired leases

The encumbrance is the sum of open budgets and committed spend. The ledger sum must always equal minus that number. If the two drift, you have a bug, not a rounding error.

The artifact: a ledger you can run today

Standard library only. One SQLite file. This runs on a laptop or a small always-on box.

# ledger.py
import sqlite3, time, uuid

DDL = """
CREATE TABLE IF NOT EXISTS lease (
  lease_id TEXT PRIMARY KEY,
  job_id   TEXT NOT NULL UNIQUE,
  budget   INTEGER NOT NULL CHECK (budget >= 0),
  spent    INTEGER NOT NULL DEFAULT 0,
  state    TEXT NOT NULL CHECK (state IN ('open','committed','released')),
  expires  REAL NOT NULL
);
CREATE TABLE IF NOT EXISTS ledger (
  entry_id INTEGER PRIMARY KEY AUTOINCREMENT,
  lease_id TEXT NOT NULL,
  delta    INTEGER NOT NULL,
  reason   TEXT NOT NULL,
  at       REAL NOT NULL
);
"""


class Broker:
    def __init__(self, path="budget.db", ttl=120.0, now=time.time):
        self.db = sqlite3.connect(path, isolation_level=None)
        self.db.execute("PRAGMA journal_mode=WAL")
        self.db.executescript(DDL)
        self.ttl, self.now = ttl, now

    def _tx(self, fn):
        self.db.execute("BEGIN IMMEDIATE")
        try:
            out = fn()
        except Exception:
            self.db.execute("ROLLBACK")
            raise
        self.db.execute("COMMIT")
        return out

    def _entry(self, lease_id, delta, reason):
        self.db.execute(
            "INSERT INTO ledger(lease_id,delta,reason,at) VALUES(?,?,?,?)",
            (lease_id, delta, reason, self.now()),
        )

    def reserve(self, job_id, budget):
        def run():
            row = self.db.execute(
                "SELECT lease_id, state FROM lease WHERE job_id=?", (job_id,)
            ).fetchone()
            if row:
                if row[1] != "open":
                    raise ValueError(f"job {job_id} is {row[1]}")
                return row[0]  # retry reuses the open lease
            lease_id = uuid.uuid4().hex
            self.db.execute(
                "INSERT INTO lease(lease_id,job_id,budget,expires) VALUES(?,?,?,?)",
                (lease_id, job_id, budget, self.now() + self.ttl),
            )
            self._entry(lease_id, -budget, "reserve")
            return lease_id
        return self._tx(run)

    def commit(self, lease_id, spent):
        def run():
            budget, state = self.db.execute(
                "SELECT budget, state FROM lease WHERE lease_id=?", (lease_id,)
            ).fetchone()
            if state != "open":
                return  # idempotent settle
            if spent > budget:
                self._entry(lease_id, -(spent - budget), "overrun")
            else:
                self._entry(lease_id, budget - spent, "refund")
            self.db.execute(
                "UPDATE lease SET spent=?, state='committed' WHERE lease_id=?",
                (spent, lease_id),
            )
        return self._tx(run)

    def release(self, lease_id):
        def run():
            budget, state = self.db.execute(
                "SELECT budget, state FROM lease WHERE lease_id=?", (lease_id,)
            ).fetchone()
            if state != "open":
                return
            self._entry(lease_id, budget, "release")
            self.db.execute(
                "UPDATE lease SET state='released' WHERE lease_id=?", (lease_id,)
            )
        return self._tx(run)

    def sweep(self):
        rows = self.db.execute(
            "SELECT lease_id FROM lease WHERE state='open' AND expires < ?",
            (self.now(),),
        ).fetchall()
        for (lease_id,) in rows:
            self.release(lease_id)
        return len(rows)


def encumbrance(broker):
    return broker.db.execute(
        "SELECT COALESCE(SUM(CASE state WHEN 'open' THEN budget ELSE spent END),0) FROM lease"
    ).fetchone()[0]


def ledger_sum(broker):
    return broker.db.execute("SELECT COALESCE(SUM(delta),0) FROM ledger").fetchone()[0]

Call it with a settle path that branches, not a bare finally. A finally commit would bill a failed call.

lease = broker.reserve(job_id="job-8241", budget=4000)
try:
    text = call_upstream(prompt, max_tokens=4000)
except Exception:
    broker.release(lease)      # upstream gave you nothing
    raise
else:
    broker.commit(lease, spent=count_tokens(prompt) + count_tokens(text))

Run the sweeper on a short interval. A crashed worker then costs you one TTL, not a frozen budget.

$ python -c "from ledger import Broker; print(Broker().sweep())"
0

The test that matters

Do not test the happy path. Test the invariant under failure. Use a fake upstream that fails on a fixed fraction of jobs.

import random
from ledger import Broker, encumbrance, ledger_sum

def fake_call(rand):
    if rand.random() < 0.3:
        raise TimeoutError("lane timeout")
    return rand.randint(1, 5000)   # simulated spent

def test_invariant_holds_under_failure():
    b, rand = Broker(":memory:", ttl=0.0), random.Random(7)
    for i in range(200):
        lease = b.reserve(f"job-{i}", 4000)
        try:
            spent = fake_call(rand)
        except TimeoutError:
            b.release(lease)
        else:
            b.commit(lease, spent)
    b.sweep()
    assert ledger_sum(b) == -encumbrance(b)

ttl=0.0 forces every open lease to expire immediately. The sweep then proves the release path works. A single drift means a code path skipped the ledger.

Failure domains, one by one

Failure	What you see	Containment
Worker dies after reserve	Budget stuck, no spend	TTL sweeper
Retry of a live job	Two leases for one job	`job_id UNIQUE`, reuse open lease
Response exceeds budget	`spent > budget`	Separate `overrun` entry, alert on it
Ledger write fails	Call runs unledgered	Reserve first; no row, no call
Upstream quota gone	429 storm, retry pileup	Per-lane breaker, queue instead of retry
Cold start on free server	Timeouts, then retries	Warm ping, cap queue depth

Read that table as a budget. You get to fix three of these today. Pick the two that touch money and the one that touches latency.

Where MonkeyCode fits

One honest lane for this. MonkeyCode offers free model access and a free server option, and the operator states a 10M-token free allowance as of publication. Free-tier numbers move, so check the current terms page before you size anything.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Use it for the parts the ledger protects. Run the broker on the free server and point one lane at free model access. Keep your paid lane behind the same broker. Then a free-lane outage is a routing event, not an incident.

What I would not do is put the broker in the same process as the worker. A crash takes both down. Separate the ledger, even onto the same box.

Who should not use this

If you have one user and one key, this is overhead. Your bill is small enough to read by hand.

If you cannot count tokens before the call, budget in requests instead. The schema works unchanged; the numbers get coarser.

If you need multi-region writes, SQLite is the wrong store. Keep the reserve/commit/release contract and move it to a transactional service.

This ledger records spend. It does not prevent an upstream from billing you anyway. Treat it as your source of truth, not as a firewall.

What I would change next

Move the reserve to the edge closest to the caller. A lease taken one network hop earlier survives more failures.

Add per-tenant budgets as a second column. One noisy job should not spend a shared pool.

Export the encumbrance as a metric. Alert when it sits flat while traffic is up. That is a leak, not a quiet period.

If you want to try the free model access and the free server, start with the ledger above. Point real jobs at the lane only after the invariant test passes twice.

Put the Free Lane Behind a Kill Switch

Harper Xu — Mon, 19 Oct 2026 08:49:19 +0000

Put the Free Lane Behind a Kill Switch

Borrowed capacity fails differently than capacity you own. Design the exit first, then wire the discount.

That is the whole review in one line. Everything below covers the seam, the data flow, the failure domains, and the drill that proves the seam holds.

The constraint you cannot negotiate

A free model endpoint has no SLA you can enforce. A free server has no restart notice you can plan around. You do not get to page anyone when they vanish.

So the boundary must absorb the loss. Your call sites should not move an inch.

You also cannot assume the terms stay the same. Free today is a fact about today. It is not a contract, a roadmap, or a promise about next quarter.

Write the code as if the lane is already scheduled for removal. Then the removal is a config change, not an incident.

Treat borrowed capacity as one module with one door. One interface, one deadline, one fallback path. Delete the module and the system still starts and still serves.

That is the test I apply to every free-tier dependency. If deleting it breaks startup, it is not a dependency. It is the foundation, and you built on sand.

Data flow, drawn as arrows

A job enters a dispatcher. The dispatcher asks a lane for a completion, with a deadline attached to the call. The lane either answers inside that deadline or the dispatcher moves on.

Every prompt is also an egress event. Your text leaves your network and lands on a machine you do not control.

Log a hash and a byte count, not the full body. Unless your own policy says the body is safe to keep, keep the hash.

Keep the response shape identical across lanes. Callers should never branch on which lane answered the request.

If any caller asks "was this free?", your abstraction already leaked. Fix the interface, not the caller.

Four failure domains worth naming

Domain	Symptom	Blast radius if unhandled
Deadline	Slow response, hung socket	Worker parks forever
Exhaustion	429 or a hard stop	Retry storm at a closed door
Quality drift	Plausible but wrong output	Silent bad data downstream
Server reboot	Connection refused	Startup path dies

The first two are loud. The third is the one that costs you a weekend.

A free lane can swap model versions without telling you. Your parser still succeeds. Your schema still validates. Your answers are just quietly worse.

Pin one prompt set, store the outputs, and diff them on a schedule. Ten fixed prompts beat a long postmortem written three weeks later.

Deadline math you can argue about

The remaining budget is the job deadline minus elapsed time. Never hand a lane more time than the job has left.

This one rule kills most of the retry storms I see. A retry that starts after the deadline is wasted work with extra load.

Track the deadline in a monotonic clock. Wall clocks jump, and a jump turns your timeout into a hang.

The seam, in about forty lines

Here is a runnable sketch. It is illustrative Python, not a published library, and you should adapt the client calls to your SDK.

# lane.py
from __future__ import annotations

import time
from dataclasses import dataclass
from typing import Protocol


@dataclass(frozen=True)
class Job:
    prompt: str
    deadline_s: float = 8.0


class Lane(Protocol):
    name: str
    def complete(self, job: Job, budget_s: float) -> str: ...


class FreeLane:
    name = "free"

    def __init__(self, client):
        self._client = client

    def complete(self, job: Job, budget_s: float) -> str:
        return self._client.complete(job.prompt, timeout=budget_s)


class PaidLane:
    name = "paid"

    def __init__(self, client):
        self._client = client

    def complete(self, job: Job, budget_s: float) -> str:
        return self._client.complete(job.prompt, timeout=budget_s)


class Breaker:
    def __init__(self, fails: int = 3, cooldown_s: float = 60.0):
        self.fails = fails
        self.cooldown_s = cooldown_s
        self._count = 0
        self._open_until = 0.0

    def allow(self, now: float) -> bool:
        return now >= self._open_until

    def record(self, ok: bool, now: float) -> None:
        if ok:
            self._count = 0
            return
        self._count += 1
        if self._count >= self.fails:
            self._open_until = now + self.cooldown_s
            self._count = 0

Now the router. It walks lanes in order and stops at the first answer inside the deadline.

# router.py
def route(job, lanes, breaker, clock=time.monotonic, audit=print):
    start = clock()
    for lane in lanes:
        remaining = job.deadline_s - (clock() - start)
        if remaining <= 0:
            break
        if lane.name == "free" and not breaker.allow(clock()):
            audit("skip free: breaker open")
            continue
        try:
            out = lane.complete(job, remaining)
        except Exception as exc:
            audit(f"{lane.name} failed: {type(exc).__name__}")
            breaker.record(False, clock())
            continue
        breaker.record(True, clock())
        audit(f"{lane.name} served, {len(out)} chars")
        return out, lane.name
    raise TimeoutError("no lane answered inside deadline")

Notice what the router does not know. It does not know model names, prices, or hosts. It knows order, deadline, and whether the breaker is open.

That ignorance is the feature. It is why you can delete the free lane in one commit.

The drill that proves it

The test below fails the free lane on purpose. Then it asserts the paid lane served the request.

# tests/test_router.py
import pytest
from lane import Job, FreeLane, PaidLane, Breaker
from router import route


class SlowClient:
    def complete(self, prompt, timeout):
        raise TimeoutError("deadline")


class FastClient:
    def __init__(self):
        self.calls = 0

    def complete(self, prompt, timeout):
        self.calls += 1
        return "ok"


def test_free_failure_falls_back_to_paid():
    paid = FastClient()
    lanes = [FreeLane(SlowClient()), PaidLane(paid)]
    out, name = route(Job("hello"), lanes, Breaker(fails=1))
    assert (out, name, paid.calls) == ("ok", "paid", 1)


def test_breaker_opens_after_repeated_failures():
    b = Breaker(fails=2, cooldown_s=30)
    b.record(False, 100.0)
    b.record(False, 101.0)
    assert b.allow(102.0) is False
    assert b.allow(131.0) is True

Run it before you trust the free path, not after.

python -m pytest -q tests/test_router.py

Then do the part most people skip. Turn the lane off and run every test you have.

FREE_LANE=off python -m pytest -q

If anything beyond a latency assertion breaks, you leaked the free lane into your logic. Fix the leak before you add features on top of it.

Do the same against the free server. Kill it and watch your startup path.

curl -sS -m 3 http://127.0.0.1:8080/health || echo "free server down"

Your app should start, degrade, and write one clear log line. It should not crash. It should not hang waiting for a socket that will never answer.

Where MonkeyCode fits, honestly

If you need a free lane to rehearse this against, MonkeyCode is an open project that offers free model access and a free server option, per the team. Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Verify the current terms on their site before you plan around them. I am not quoting quotas, model names, or hardware here. Those numbers move, and I have not measured them on your workload.

Use it as the removable part, not the foundation. The lesson lives in the drill, not in the discount.

What I would change next

Replace the hardcoded lane order with a list read from config. Add a per-lane counter so exhaustion becomes a state you observe, not an exception you catch by accident.

Then add a quality canary. Fixed prompts, stored answers, a nightly diff. When the free lane drifts, a diff tells you first.

Last, record which lane served each job in your audit log. Without that field you cannot answer the only question that matters later. What changed, and when did it start?

Who should skip this

Skip this if you have a hard latency SLO you must meet on every call. Skip it if the data cannot leave your network boundary. Skip it if you need a pinned model version to reproduce results months later.

Free capacity is a fine place to prototype and a bad place to depend. That sentence survives every pricing page you will ever read.

Run the kill-switch drill on whatever free lane you already have. If you want a practice target, MonkeyCode's free tier is one option to point it at.

Put the Free Lane Behind a Kill Switch

Harper Xu — Mon, 19 Oct 2026 08:39:04 +0000

Put the Free Lane Behind a Kill Switch

Borrowed capacity fails differently than capacity you own. Design the exit first, then wire the discount.

That is the whole review in one line. Everything below covers the seam, the data flow, the failure domains, and the drill that proves the seam holds.

The constraint you cannot negotiate

A free model endpoint has no SLA you can enforce. A free server has no restart notice you can plan around. You do not get to page anyone when they vanish.

So the boundary must absorb the loss. Your call sites should not move an inch.

You also cannot assume the terms stay the same. Free today is a fact about today. It is not a contract, a roadmap, or a promise about next quarter.

Write the code as if the lane is already scheduled for removal. Then the removal is a config change, not an incident.

Treat borrowed capacity as one module with one door. One interface, one deadline, one fallback path. Delete the module and the system still starts and still serves.

That is the test I apply to every free-tier dependency. If deleting it breaks startup, it is not a dependency. It is the foundation, and you built on sand.

Data flow, drawn as arrows

A job enters a dispatcher. The dispatcher asks a lane for a completion, with a deadline attached to the call. The lane either answers inside that deadline or the dispatcher moves on.

Every prompt is also an egress event. Your text leaves your network and lands on a machine you do not control.

Log a hash and a byte count, not the full body. Unless your own policy says the body is safe to keep, keep the hash.

Keep the response shape identical across lanes. Callers should never branch on which lane answered the request.

If any caller asks "was this free?", your abstraction already leaked. Fix the interface, not the caller.

Four failure domains worth naming

Domain	Symptom	Blast radius if unhandled
Deadline	Slow response, hung socket	Worker parks forever
Exhaustion	429 or a hard stop	Retry storm at a closed door
Quality drift	Plausible but wrong output	Silent bad data downstream
Server reboot	Connection refused	Startup path dies

The first two are loud. The third is the one that costs you a weekend.

A free lane can swap model versions without telling you. Your parser still succeeds. Your schema still validates. Your answers are just quietly worse.

Pin one prompt set, store the outputs, and diff them on a schedule. Ten fixed prompts beat a long postmortem written three weeks later.

Deadline math you can argue about

The remaining budget is the job deadline minus elapsed time. Never hand a lane more time than the job has left.

This one rule kills most of the retry storms I see. A retry that starts after the deadline is wasted work with extra load.

Track the deadline in a monotonic clock. Wall clocks jump, and a jump turns your timeout into a hang.

The seam, in about forty lines

Here is a runnable sketch. It is illustrative Python, not a published library, and you should adapt the client calls to your SDK.

# lane.py
from __future__ import annotations

import time
from dataclasses import dataclass
from typing import Protocol


@dataclass(frozen=True)
class Job:
    prompt: str
    deadline_s: float = 8.0


class Lane(Protocol):
    name: str
    def complete(self, job: Job, budget_s: float) -> str: ...


class FreeLane:
    name = "free"

    def __init__(self, client):
        self._client = client

    def complete(self, job: Job, budget_s: float) -> str:
        return self._client.complete(job.prompt, timeout=budget_s)


class PaidLane:
    name = "paid"

    def __init__(self, client):
        self._client = client

    def complete(self, job: Job, budget_s: float) -> str:
        return self._client.complete(job.prompt, timeout=budget_s)


class Breaker:
    def __init__(self, fails: int = 3, cooldown_s: float = 60.0):
        self.fails = fails
        self.cooldown_s = cooldown_s
        self._count = 0
        self._open_until = 0.0

    def allow(self, now: float) -> bool:
        return now >= self._open_until

    def record(self, ok: bool, now: float) -> None:
        if ok:
            self._count = 0
            return
        self._count += 1
        if self._count >= self.fails:
            self._open_until = now + self.cooldown_s
            self._count = 0

Now the router. It walks lanes in order and stops at the first answer inside the deadline.

# router.py
def route(job, lanes, breaker, clock=time.monotonic, audit=print):
    start = clock()
    for lane in lanes:
        remaining = job.deadline_s - (clock() - start)
        if remaining <= 0:
            break
        if lane.name == "free" and not breaker.allow(clock()):
            audit("skip free: breaker open")
            continue
        try:
            out = lane.complete(job, remaining)
        except Exception as exc:
            audit(f"{lane.name} failed: {type(exc).__name__}")
            breaker.record(False, clock())
            continue
        breaker.record(True, clock())
        audit(f"{lane.name} served, {len(out)} chars")
        return out, lane.name
    raise TimeoutError("no lane answered inside deadline")

Notice what the router does not know. It does not know model names, prices, or hosts. It knows order, deadline, and whether the breaker is open.

That ignorance is the feature. It is why you can delete the free lane in one commit.

The drill that proves it

The test below fails the free lane on purpose. Then it asserts the paid lane served the request.

# tests/test_router.py
import pytest
from lane import Job, FreeLane, PaidLane, Breaker
from router import route


class SlowClient:
    def complete(self, prompt, timeout):
        raise TimeoutError("deadline")


class FastClient:
    def __init__(self):
        self.calls = 0

    def complete(self, prompt, timeout):
        self.calls += 1
        return "ok"


def test_free_failure_falls_back_to_paid():
    paid = FastClient()
    lanes = [FreeLane(SlowClient()), PaidLane(paid)]
    out, name = route(Job("hello"), lanes, Breaker(fails=1))
    assert (out, name, paid.calls) == ("ok", "paid", 1)


def test_breaker_opens_after_repeated_failures():
    b = Breaker(fails=2, cooldown_s=30)
    b.record(False, 100.0)
    b.record(False, 101.0)
    assert b.allow(102.0) is False
    assert b.allow(131.0) is True

Run it before you trust the free path, not after.

python -m pytest -q tests/test_router.py

Then do the part most people skip. Turn the lane off and run every test you have.

FREE_LANE=off python -m pytest -q

If anything beyond a latency assertion breaks, you leaked the free lane into your logic. Fix the leak before you add features on top of it.

Do the same against the free server. Kill it and watch your startup path.

curl -sS -m 3 http://127.0.0.1:8080/health || echo "free server down"

Your app should start, degrade, and write one clear log line. It should not crash. It should not hang waiting for a socket that will never answer.

Where MonkeyCode fits, honestly

Verify the current terms on their site before you plan around them. I am not quoting quotas, model names, or hardware here. Those numbers move, and I have not measured them on your workload.

Use it as the removable part, not the foundation. The lesson lives in the drill, not in the discount.

What I would change next

Replace the hardcoded lane order with a list read from config. Add a per-lane counter so exhaustion becomes a state you observe, not an exception you catch by accident.

Then add a quality canary. Fixed prompts, stored answers, a nightly diff. When the free lane drifts, a diff tells you first.

Last, record which lane served each job in your audit log. Without that field you cannot answer the only question that matters later. What changed, and when did it start?

Who should skip this

Free capacity is a fine place to prototype and a bad place to depend. That sentence survives every pricing page you will ever read.

Run the kill-switch drill on whatever free lane you already have. If you want a practice target, MonkeyCode's free tier is one option to point it at.

Architecture Review: Borrowed Capacity, Owned Exit

Harper Xu — Sun, 18 Oct 2026 07:18:48 +0000

Borrow the model. Own the exit. That is the whole review.

Most teams add a free inference lane the same way. They wire a key, run a demo, and move on. Nothing breaks until the lane gets slow or tight. Then the demo becomes an outage with no owner.

So I run the review in a fixed order. Constraints first. Data flow second. Failure domains third. A small script keeps everyone honest.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

MonkeyCode is an open-source project that offers free model access and a free server option. The operator states the free tier includes 10 million tokens. Treat that number and the server terms as current-at-writing claims. Confirm both on the project page before you size anything against them.

The constraint shapes everything

You do not control the lane's capacity, its model version, or its retention. That is the constraint, and it is not fixable. Design as if the lane vanishes at 3 a.m.

Your free server has a second constraint. It is borrowed compute, so treat its disk and its uptime as temporary. Nothing precious lives there by default.

Follow one job through the system

A job enters a queue. A router picks a backend. The backend returns text. A parser turns that text into a structured artifact. The artifact lands in your own store.

The free server can host the queue, the router, and the store. The free lane touches exactly one step. Keep that boundary sharp and the rest follows.

The lane produces text. Your code owns truth. Write that sentence on the whiteboard before anyone writes a prompt.

Name the failure domains out loud

The first domain is refusal by volume. You hit a rate limit and get a 429 back. Your router must fall through to a deterministic path, not retry forever.

Second is silent schema drift. The lane returns prose where you expected JSON. Your parser must reject it loudly instead of guessing.

Third is egress. Every prompt leaves your machine. If a job carries customer data, the free lane is the wrong lane. No exceptions, no allowlist tricks.

Fourth is the host itself. A free server can restart under you. Any file your job needs must be rebuildable from the store.

Artifact: a lane audit you can run today

This script probes the lane, classifies the outcome, and appends a line to a ledger. Run it before CI, and again every morning. Python 3.11 or newer.

# lane_audit.py
import json, os, time, datetime, urllib.request, urllib.error

LANE_URL = os.environ.get("FREE_LANE_URL", "")
LANE_KEY = os.environ.get("FREE_LANE_KEY", "")
LEDGER = os.environ.get("LANE_LEDGER", "lane_ledger.jsonl")
CANARY = "Reply with the single word: ok"

def classify(code):
    return {401: "auth", 403: "auth", 404: "bad_route",
            429: "rate_limited", 503: "lane_down"}.get(code, f"http_{code}")

def probe(timeout=20):
    body = json.dumps({"messages": [{"role": "user", "content": CANARY}]}).encode()
    req = urllib.request.Request(
        LANE_URL, data=body,
        headers={"Content-Type": "application/json",
                 "Authorization": f"Bearer {LANE_KEY}"})
    t0 = time.monotonic()
    try:
        with urllib.request.urlopen(req, timeout=timeout) as resp:
            payload = resp.read().decode()
        return {"class": "ok", "ms": int((time.monotonic() - t0) * 1000),
                "bytes": len(payload)}
    except urllib.error.HTTPError as e:
        return {"class": classify(e.code), "ms": int((time.monotonic() - t0) * 1000)}
    except Exception as e:
        return {"class": type(e).__name__, "ms": int((time.monotonic() - t0) * 1000)}

def record(result):
    result["at"] = datetime.datetime.now(datetime.UTC).isoformat()
    with open(LEDGER, "a") as f:
        f.write(json.dumps(result) + "\n")
    return result

if __name__ == "__main__":
    r = record(probe())
    print(json.dumps(r))
    raise SystemExit(0 if r["class"] == "ok" else 1)

The exit code matters more than the JSON. Wire it into CI and you get a red build the day the lane changes shape. That is cheaper than a red demo.

Now read the ledger weekly. Count the classes. A rising rate_limited share means your batch jobs need a budget, not a retry loop.

Test the product with the lane removed

This is the test almost nobody writes. Delete the lane and the app must still boot. Point the lane at a dead port and the job must still finish in a degraded mode.

# test_degraded_lane.py
import importlib

def test_boots_without_lane(monkeypatch):
    monkeypatch.delenv("FREE_LANE_URL", raising=False)
    import app
    importlib.reload(app)
    assert app.backend_name() == "static"

def test_job_survives_dead_lane(monkeypatch):
    monkeypatch.setenv("FREE_LANE_URL", "http://127.0.0.1:9/dead")
    import app
    importlib.reload(app)
    out = app.run_job({"input": "hello"})
    assert out["status"] in {"ok", "degraded"}
    assert out["source"] == "static"

If the second test fails, you do not have a lane. You have a dependency. Those are different things, and only one of them belongs in a demo.

What belongs on the free lane

Workload	Fits?	Why	Guard
Public demo for a post	Yes	Bounded, low stakes	Cache the output
Internal tool, five users	Yes	Small blast radius	Daily audit run
Customer API with an SLA	No	Capacity is not yours	Paid lane only
Regulated or personal data	No	Every prompt is egress	Keep it local
50k-row batch backfill	Caution	Will hit limits	Chunk, then pause

Read the middle column honestly. Most incidents come from rows three and four, not from row one.

What I would change next

Move the artifact store off the free server first. That single change removes the worst restart scenario.

Then add a budget counter next to the router. Count tokens per job, not per day. A per-day cap hides the one job that eats the month.

Finally, version your prompts like code. When the lane's model changes under you, a prompt diff tells you what actually moved.

Limits, and who should walk away

Free capacity is fine for demos, prototypes, and internal tools. It is a poor fit for anything with a latency promise. It is the wrong fit for regulated data or for state that cannot be rebuilt.

If your team cannot yet write the degraded-path test above, fix that first. Adding a second borrowed lane just doubles the uncertainty.

The project is open source, so you can read the code before you trust the claims. Start with the audit script, not the demo.

Draw Your Free Tier as Three Planes

Harper Xu — Sat, 17 Oct 2026 06:21:12 +0000

Free inference is not a socket you plug in. It is a plane you place in a diagram.

Sketch three planes before you write a client. One plane holds your state. One plane answers prompts. One plane runs work that outlives a single request.

If you cannot name which plane failed, you are not ready to ship on a free tier. Here is the review I would run on that sketch.

Constraints set the shape

Three constraints decide the design. The model behind a free endpoint can change without telling you. Throughput is shared, so latency is a distribution and not a number. The machine that runs your jobs is not yours to patch at 2 a.m.

A wallet mental model fails here. Budgets feel finite. Shared capacity is not yours to reason about. You need a pipeline view, not a balance.

None of those constraints are defects. They are inputs. Design against them, not around them.

MonkeyCode is the concrete case in this review. It offers free model access and a free server option, which map to the middle and last planes. Disclosure: This article was prepared as part of MonkeyCode's product outreach. The advertised allowance is 10M tokens at the time of writing. Re-read the current terms before you size a batch.

One rule keeps the flow one-way

Your ledger owns state. The model plane owns nothing. The compute plane owns nothing you cannot rebuild.

That rule turns data flow into six steps. Create a job row, assemble the prompt, hold a token estimate, capture the response, store the artifact, trigger the run. Every step writes to your side of the fence. None of them read state back from the provider.

Now the part that saves you a night. Refuse work before you spend, not after.

# budget.py — starter, no dependencies, single-flight only
from dataclasses import dataclass

@dataclass
class Ledger:
    ceiling: int
    spent: int = 0
    held: int = 0

    def hold(self, estimate: int) -> bool:
        if self.spent + self.held + estimate > self.ceiling:
            return False
        self.held = estimate
        return True

    def settle(self, actual: int) -> None:
        self.held = 0
        self.spent += actual

Call hold with a rough estimate from your prompt length. Call settle with the usage the response reports. If hold returns false, the job waits in a queue you can see.

That queue is the whole point. A refusal you can inspect beats a 429 you cannot.

Run it once with a tiny ceiling to prove the refusal path fires. You want that failure in a test, not in a demo.

Check the lane before the batch

Free endpoints move. Treat reachability as a measurement, not an assumption. Run a canary before a batch, never during one.

#!/usr/bin/env bash
set -euo pipefail
: "${MC_BASE_URL:?set MC_BASE_URL}"
: "${MC_API_KEY:?set MC_API_KEY}"

curl -sS --max-time 20 "$MC_BASE_URL/v1/chat/completions" \
  -H "Authorization: Bearer $MC_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"messages":[{"role":"user","content":"reply with ok"}],"max_tokens":8}' \
  | grep -q '"content"' && echo "canary: up" || { echo "canary: down"; exit 1; }

Adjust the path to your provider's shape. The check is not the point. The habit is. Gate the batch on the canary, and a bad afternoon becomes a delayed job instead of a half-written artifact.

Three blast radii, not one

Naming failure domains is cheap. Bounding them is the work. Draw a circle around each plane and ask what leaks out of it.

The ingress radius covers prompt assembly. A bad template corrupts every job downstream. Version the template and stamp the version on each row.

The model radius covers inference. A timeout here costs one held estimate, nothing more. Keep it that way by never letting a response write directly to your system of record.

The compute radius covers the free server. Restarts are weather, not incidents. Make every job idempotent so a restart replays cleanly and the artifact lands once.

Plane	What breaks first	Blast radius	Next change
Ingress	Prompt template edit	Every queued job	Version and stamp
Model	Timeout or shape change	One held estimate	Fallback behind one interface
Compute	Restart mid-job	One artifact	Idempotent replay
Ledger	Concurrent writes	Silent overspend	Single-flight lock

Read the table top to bottom. The last row is the one people skip. It is also the one that quietly spends your allowance while you sleep.

What I would change next

Move prompt assembly behind a versioned template first. Your ledger is only as honest as the thing that produced the estimate.

Then make the ledger durable and single-flight. The starter above assumes one writer. Real batch runners need a lock or a queue. Until then, keep concurrency at one and say so in the README.

Finally, put a fallback model behind the same interface. Do not build a router with five branches. Build one seam and change what sits behind it.

Measure queue depth weekly. If it grows, your estimate is wrong, not the model.

Who should not use this

Do not build on a free tier if you need pinned model versions in a contract. Do not send regulated data you cannot share with a third party. Do not accept cold starts if you sold an availability number. Pay for the guarantee instead.

Everyone else should start here. Sketch the planes, prove the refusal path, gate on a canary. Then move real work through it.

Try the free model access and the free server with one canary job, not your backlog. If the diagram survives that, it survives Tuesday.

Quota Exhaustion Is a Failure Domain

Harper Xu — Fri, 16 Oct 2026 05:23:21 +0000

Your free inference budget will run out mid-job. That is not bad luck. It is a failure domain.

Teams keep treating quota errors as billing noise. They wrap the call in a try block and move on. Then a four-step job dies at step three, and nobody knows what was already written.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The free lane in this review is MonkeyCode's. The operator supplies free model access with 10 million tokens, plus a free server option. I have not benchmarked either one. I quote no latency, no success rate, and no throughput here, because I have no verified numbers for them.

The constraints you actually have

Three constraints shape any design around a free lane. Finite budget. Revocable access. Expensive redo.

Ten million tokens is a real amount of work. It is also gone in one afternoon if you pipe a monorepo through it. So stop treating the free tier as a floor you stand on. Treat it as a lane you drive in, with a sign that says the lane may close.

Access is revocable by design. I am not predicting anything about MonkeyCode's roadmap. I am stating a design assumption: no free lane belongs in your critical path.

Redo is the expensive part. Each replayed step costs tokens a second time. A resume that skips finished steps is cheaper than a retry that starts over.

The last week of DEV front-page posts was heavy on AI fatigue and tool churn. That matches production. The lane you use today is not the lane you use in six months. Design for the swap, not for the vendor.

Data flow

Keep the flow boring. Four boxes, one direction.

step list -> budget ledger -> lane adapter -> checkpoint log -> next step

The ledger charges before the call, never after. A timeout comes after the response, and it still spends tokens. If you charge on success only, your counter drifts low. Drift is worse than a hard stop, because you plan against a number that is wrong.

The checkpoint log is append-only. One JSON line per finished step. On restart, you read the log and skip what is done.

The adapter has exactly one method. That is the whole contract: complete(prompt) -> str. If leaving a lane takes three methods, you cannot leave it.

Where it breaks

Failure domain	Trigger	Today's symptom	Recovery
Quota	ledger hits zero	exception mid-job	resume from checkpoint
Rate limit	burst of calls	retries pile up	back off, single lane
Model drift	lane swaps the model	output shape changes	validate every step
Egress	prompt leaves host	data outside boundary	redact before send
Process death	runner is killed	in-memory state lost	append to disk, not RAM

Each row gets its own trigger and its own recovery. Do not collapse them into one except Exception. Name them, then test them.

The harness

Here is a runnable version. It uses the standard library only and never touches the network. Save it as lane_ledger.py.

# lane_ledger.py - Python 3.11+, standard library only.
import json, pathlib, tempfile

class QuotaExhausted(Exception):
    pass

class Ledger:
    """Charge before the call. Never after."""
    def __init__(self, budget: int):
        self.budget = budget
        self.spent = 0
    def charge(self, tokens: int) -> None:
        if self.spent + tokens > self.budget:
            raise QuotaExhausted(f"need {tokens}, left {self.budget - self.spent}")
        self.spent += tokens

class StubLane:
    """One adapter, one method. Swapping it means leaving the lane."""
    def __init__(self, ledger: Ledger, cost: int = 3):
        self.ledger = ledger
        self.cost = cost
    def complete(self, prompt: str) -> str:
        self.ledger.charge(self.cost)
        return "stub:" + prompt[::-1]

class JobRunner:
    """Append-only checkpoints. A resume skips finished steps."""
    def __init__(self, lane: StubLane, checkpoint: str):
        self.lane = lane
        self.path = pathlib.Path(checkpoint)
        self.executed: list[int] = []
    def done(self) -> set[int]:
        if not self.path.exists():
            return set()
        rows = [json.loads(l) for l in self.path.read_text().splitlines() if l.strip()]
        return {r["step"] for r in rows}
    def run(self, steps: list[str]) -> list[int]:
        self.executed = []
        done = self.done()
        for i, step in enumerate(steps):
            if i in done:
                continue
            out = self.lane.complete(step)
            with self.path.open("a") as f:
                f.write(json.dumps({"step": i, "input": step, "output": out}) + "\n")
            self.executed.append(i)
        return self.executed

def chaos_test() -> None:
    ckpt = str(pathlib.Path(tempfile.mkdtemp()) / "job.jsonl")
    steps = ["draft", "critique", "patch", "changelog"]
    ledger = Ledger(budget=7)
    runner = JobRunner(StubLane(ledger, cost=3), ckpt)

    try:
        runner.run(steps)          # 3 + 3 tokens, then the third call dies
    except QuotaExhausted:
        pass
    first = runner.executed
    assert first == [0, 1], first
    assert ledger.spent == 6, ledger.spent

    ledger.budget = 12             # top up, then resume
    second = runner.run(steps)
    assert second == [2, 3], second

    rows = [json.loads(l) for l in pathlib.Path(ckpt).read_text().splitlines()]
    assert [r["step"] for r in rows] == [0, 1, 2, 3], rows
    print("first run:", first, "tokens: 6")
    print("resume:", second, "tokens:", ledger.spent)
    print("checkpoints:", len(rows), "duplicate steps: 0")

if __name__ == "__main__":
    chaos_test()

Run it.

python lane_ledger.py
first run: [0, 1] tokens: 6
resume: [2, 3] tokens: 12
checkpoints: 4 duplicate steps: 0

Read that output like an architecture review. The first run spends six tokens and dies on the third call. The ledger refuses the call instead of overdrawing. The resume executes only steps two and three. Four checkpoints exist, and none repeats.

Now count where the lane's name appears in your own repo.

grep -rn "StubLane" --include="*.py" . | wc -l

One file is healthy. Ten files means your lane is load-bearing. That count belongs in the design review, not in a postmortem.

What the test does not prove

Be honest about scope. This harness proves resume logic, not inference quality. It never touches a network, a model, or a free server.

It cannot tell you how a real free lane behaves under load. It cannot promise that an allowance survives an account change. Treat the 10 million tokens as a starting allowance, not a guarantee.

The stub also always succeeds while it has budget. Real lanes can fail halfway through a stream. Your adapter still needs a partial-response path for that case, and this test does not cover it.

Who should not use this

Skip this approach if you need data residency guarantees. Prompts leave your host, and the checkpoint log writes model output to disk.

Skip it for latency-critical paths. A resume loop is slower than one direct call, by design.

Skip it if the work is regulated or audited. Pinning a model version for reproducibility is hard when you do not control that version.

What I would change next

Add a second adapter behind the same interface. Two lanes make the swap test real instead of theoretical. Then make chaos fail mid-stream, between tokens, not neatly between calls.

Move the ledger out of the process once more than one worker runs jobs. A file-based counter races under concurrency.

Log a stop reason for every halt. Quota exhaustion, rate limiting, and schema drift need different alerts and different humans.

The goal is not a reliable free lane. You cannot build one. The goal is a system that does not care which lane answers.

If you want to point this harness at a real endpoint, the free tier is a cheap place to start, and the operator supplies 10M tokens plus a free server option.

Treat Free Inference as a Removable Part

Harper Xu — Thu, 15 Oct 2026 11:28:07 +0000

A free inference lane looks like capacity you gained.
It is also a dependency you accepted.

If you cannot unplug it in one commit, review it as architecture.
That framing changes which questions you ask first.

Most teams adopt a free lane during a spike.
The spike ends, the lane stays, and nobody writes the exit test.
Six months later the exit cost lands on release day.

Review against four constraints

A removable part has a documented boundary.
A free lane usually has none of that written down.

Constraint	What you can assume	What to verify before relying on it
Availability	Best effort, shared capacity	Whether any SLA exists in writing
Permanence	Terms and limits can change	The current published terms, on the day you ship
Quota	Some allowance exists	The exact allowance and what happens at zero
Data handling	Your prompt leaves your machine	Retention, logging, and training policy

None of these are reasons to avoid a free lane.
They are reasons to keep the lane behind a seam.

Trace the data flow first

Draw the hops before you write the adapter.
Here is the flow I would expect to find.

job spec -> lane adapter -> network egress -> provider
         -> response -> artifact -> receipt -> your repo

Each arrow is a failure domain you own.
The provider owns only the middle one, and only partly.

Your adapter is the only hop you fully control.
That is why the seam belongs there, not in your business logic.

Artifact 1: one seam for every lane

The scaffold below is a proposal, not a tested release.
It exists to make lane swapping a config change, not a refactor.

# lane.py - one seam for every inference lane
import os, time, json, hashlib, urllib.request

class Lane:
    name = "abstract"
    model = "unset"

    def complete(self, prompt: str) -> str:
        raise NotImplementedError

class HttpLane(Lane):
    def __init__(self, name, model, url, key_env):
        self.name, self.model, self.url = name, model, url
        self.key = os.environ.get(key_env, "")

    def complete(self, prompt: str) -> str:
        payload = {"model": self.model,
                   "messages": [{"role": "user", "content": prompt}]}
        req = urllib.request.Request(
            self.url,
            data=json.dumps(payload).encode(),
            headers={"Content-Type": "application/json",
                     "Authorization": "Bearer " + self.key})
        with urllib.request.urlopen(req, timeout=30) as resp:
            data = json.loads(resp.read())
        return data["choices"][0]["message"]["content"]

def receipt(lane, prompt, out, ms, attempt, ok):
    return {"lane": lane.name, "model": lane.model,
            "prompt_sha256": hashlib.sha256(prompt.encode()).hexdigest()[:16],
            "out_len": len(out), "ms": ms,
            "attempt": attempt, "ok": ok, "at": int(time.time())}

Two details matter more than the code.
The model string is recorded, never assumed.
And the receipt stores a prompt hash, not the prompt.

A hash proves you ran the same input twice.
Storing the prompt would move your egress problem into your logs.

Name the failure domains

A free lane fails differently from a paid one.
Build the detection path before the retry path.

Failure	Detection	Degradation you should define
Quota is exhausted	Non-success status on first attempt	Stop the batch, keep the queue
Provider returns errors	Timeout or 5xx in the receipt	Two retries, then switch lanes
Free server is unreachable	Connection error before any response	Fall back, never fake output
Terms or limits changed	Weekly config check fails	Fail the build, not the user
Output quality drifts	Swap test agreement drops	Route that job type to review

Retries are where free lanes quietly become expensive.
Cap them at two per job and count them in the receipt.

Artifact 2: the swap test

The only honest way to judge a lane is to run your jobs on two.
Here is a small harness that does one job set through both.

# swap_test.py - run one job set through two lanes and diff
import json, time, difflib, pathlib
from lane import HttpLane, receipt

LANES = [HttpLane("free", "MODEL_A", "https://free.example/v1/chat", "FREE_KEY"),
         HttpLane("paid", "MODEL_B", "https://paid.example/v1/chat", "PAID_KEY")]

raw = pathlib.Path("jobs.txt").read_text()
prompts = [p.strip() for p in raw.split("\n---\n") if p.strip()]
rows = []

for p in prompts:
    outs = {}
    for lane in LANES:
        t0, ok, out = time.time(), False, ""
        try:
            out = lane.complete(p)
            ok = True
        except Exception as err:
            out = "ERROR: " + type(err).__name__
        ms = int((time.time() - t0) * 1000)
        outs[lane.name] = out
        rows.append(receipt(lane, p, out, ms, 1, ok))
    ratio = difflib.SequenceMatcher(None, outs["free"], outs["paid"]).ratio()
    rows.append({"kind": "agreement", "ratio": round(ratio, 3)})

print(json.dumps(rows, indent=2))

Run it and keep the output as your baseline.

printf 'job one\n---\njob two\n---\njob three\n' > jobs.txt
python swap_test.py > receipts.json
python -c "import json;d=json.load(open('receipts.json'));print([r['ratio'] for r in d if 'ratio' in r])"

Agreement is a routing signal, not a quality score.
Read it as a heuristic and re-run it after every lane change.

My working rule, still untested on real workloads:
run the set three times, and treat a job type as lane-portable only if agreement stays high across all three.
Below that line, the free lane still runs, but a human reads the output.

What I would change next

Cap retries at two and record each attempt separately.
A single "failed" flag hides a lane that burned your quota slowly.

Keep a fallback lane configured but disabled.
An enabled fallback becomes the primary lane within a month.

Hash prompts in receipts and keep the raw text in your own store.
That keeps the audit trail inside your boundary.

Pin the model string you sent, not the model you intended.
Aliases move, and your receipts will not tell you.

Add one config check per week that reads the provider's current terms page.
Fail the check loudly when the text changes.

Re-run the swap test whenever the lane, the model, or the job set changes.
Cheap tests age badly, and so do free lanes.

Where a free lane fits in this design

A free lane is a good fit when the seam exists and the exit is tested.
That is the whole point of building the adapter first.

The operator of this account has supplied two availability claims for MonkeyCode, an open-source coding project: free model access, and a free server option.
I have not benchmarked either, and I have not verified the current allowance.
The allowance has been described to me as a fixed token amount with a free server, and that description should be re-checked against the published terms on the day you decide.
Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Where it genuinely participates in this workflow is as a lane behind the seam.
You point one HttpLane at it, run the swap test, and read the agreement numbers for your own jobs.
The free server option matters for the same reason: a box you do not manage is a failure domain you must still name.

That is a testable claim, which is the only kind worth repeating.
If the seam works, the free lane is an easy addition and an easy removal.

When this approach is wrong

Do not send regulated or licensed data to any lane you cannot audit.
No free tier changes that constraint, and no adapter hides it.

Do not use this if your job needs a hard latency target.
Shared capacity and retries do not mix with tight SLOs.

Do not use it in a no-egress environment, even for drafts.
The first prompt is already an egress event.

And do not adopt a free lane without a fallback you have actually exercised.
A fallback you never ran is a comment, not a plan.

A free lane you cannot remove is not capacity.
It is a deadline you scheduled for someone else.

File the Next Change Before Run

Harper Xu — Tue, 13 Oct 2026 02:00:42 +0000

You file the next change before any free run starts. A cheap answer still carries your real system. Free capacity never replaces a written architecture review.

You feel the pull when a model replies in seconds. Speed looks like progress, yet it is not. That speed is only a faster untrusted draft.

Your job is the seam between that draft and a host. The seam has constraints, a data flow, and failure domains. You name the next change before you cross.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode offers free model access and a free server option. Those two facts are availability claims, not a design.

You still own the architecture of the path. A free model does not know your blast radius. A free server does not inherit your production trust.

Constraints before comfort

You start with the box you cannot move. Your repo holds secrets, licenses, and user data. None of those may ride inside a prompt.

The free model lane writes text and does not merge code. You send the smallest context that can answer one question. You keep apply rights on a host you control.

The free server is a rented bench, not your factory floor. You do not know its neighbors, its lifetime, or its disk after you leave. Treat every file there as public scratch paper.

Confirm current limits on the product page before you plan. Quotas, hardware, and duration are not facts in this note. If the page disagrees with your memory, the page wins.

This review assumes you can read a diff and reject it. It assumes you can run tests on a machine you administer. If either assumption fails, you should stop here.

Walk the data flow

Picture a package that must cross a busy city. The label is the contract you wrote at home. The truck is the model call, and the dock is the server.

You write the contract on your machine first. It lists the question, the allowed files, and the forbidden paths. It also lists the next change you would keep.

The contract leaves your machine as plain text. That exit still counts, even when the vendor is generous. You strip tokens, keys, and customer rows before the text moves.

The model returns a draft, not a verdict. You save that draft beside the contract on disk. You diff it like mail from a careful stranger.

Only a bounded check may travel to the free server. The check is a command with a timeout and a fixture. It must not apply migrations or open production paths.

Results travel back as logs and exit codes. You read them on the host you already trust. The rented server does not get a vote.

Name the domains that break together

A failure domain is the set of things that break together. Free pricing does not draw that set for you. You draw the set before the run begins.

Domain one is the prompt you send away. A bad prompt can leak a key or a private path. The blast radius is every system that key can touch.

Domain two is the draft you might accept. A confident patch can invert a check or drop a guard. The blast radius is the next merge if you skip review.

Domain three is the free server you borrowed. A shared bench can keep files, logs, or a half-written env. The blast radius is whatever you copied onto that disk.

Domain four is the apply host you still own. That host can change real users if you let it. You keep it outside the free loop on purpose.

If two domains share a credential, you have one domain. Split them before you celebrate a green log. A green log on scratch is not a release.

A contract the shell can refuse

The script below is a labeled proposal, not a live client. No live vendor run is claimed in this note. You run it locally, then you judge the ticket.

It reads a small contract from your disk. It refuses secret-shaped strings and a missing next change. It writes a run ticket only when the review fields exist.

#!/usr/bin/env python3
"""Path review for a bounded check. Proposal, not a vendor client."""
import json, re, sys
from pathlib import Path

SECRET = re.compile(r"(?i)(api[_-]?key|secret|token|password)\s*[:=]\s*\S+")
REQUIRED = ("question", "allowed_files", "forbidden_paths",
            "failure_domains", "next_change", "check_cmd")

def load(path):
    data = json.loads(Path(path).read_text())
    if not isinstance(data, dict):
        raise SystemExit("contract must be an object")
    return data

def review(data):
    missing = [k for k in REQUIRED if not data.get(k)]
    blob = json.dumps(data)
    leaks = SECRET.findall(blob)
    notes = []
    if missing:
        notes.append("missing:" + ",".join(missing))
    if leaks:
        notes.append("secret_shape_detected")
    if "apply" in data.get("check_cmd", "").lower():
        notes.append("check_cmd_looks_like_apply")
    domains = data.get("failure_domains") or []
    if len(domains) < 3:
        notes.append("name_at_least_three_domains")
    return notes

def main():
    data = load(sys.argv[1])
    notes = review(data)
    if notes:
        print(json.dumps({"ok": False, "blockers": notes}, indent=2))
        raise SystemExit(2)
    ticket = {
        "ok": True,
        "question": data["question"],
        "check_cmd": data["check_cmd"],
        "next_change": data["next_change"],
        "run_on": "free_server_scratch_only",
        "apply_on": "operator_host_after_review",
    }
    print(json.dumps(ticket, indent=2))

if __name__ == "__main__":
    main()

You save a narrow contract next to the script. You keep the question small enough to test once. You keep the check command boring and bounded.

{
  "question": "Does the retry helper stop after three attempts?",
  "allowed_files": ["src/retry.py", "tests/test_retry.py"],
  "forbidden_paths": [".env", "secrets/", "prod.tfvars"],
  "failure_domains": ["prompt_text", "draft_patch", "free_server_disk", "apply_host"],
  "next_change": "Add a test that the helper refuses a fourth attempt.",
  "check_cmd": "python -m pytest tests/test_retry.py -q --timeout=30"
}

Run the review on the machine you administer. A blocked ticket means you are not ready to copy files. A clean ticket permits fixtures, not a deploy.

python3 review_path.py contract.json

If the script exits with code two, fix the contract. Do not bargain the blocker away in a side channel. The side channel is not your architecture record.

Copy only the allowed files to the free server. Run only the check command printed on the ticket. Pull the logs home, then delete the scratch tree.

scp -r src tests user@free-server:~/scratch/retry-check/
ssh user@free-server 'cd ~/scratch/retry-check && python -m pytest tests/test_retry.py -q --timeout=30'
ssh user@free-server 'rm -rf ~/scratch/retry-check'

Those shell lines are a sketch, not a promise. You replace the host with the name your plan gives. If the plan has no shell, you do not invent one.

What you change next

The next change is a sentence in the contract. In the sample, that sentence is a fourth-attempt test. You write it before the model is allowed to speak.

After the logs return, you edit that sentence or you keep it. You do not let the model replace it in silence. Your change note is the memory of the architecture.

You would change the script next by hashing allowed files. The ticket should name the bytes you meant to send. A later run should fail if those bytes moved.

You would also stamp an end time on the ticket. A free server session should expire in that stamp. If you cannot state an end, scratch has become home.

You would not add an auto-apply step to this ticket. The moment the ticket can merge, four domains collapse. That collapse is the bug this review exists to prevent.

Who should leave this path

This script catches shape, not the author's intent. A secret pasted without a keyword can still leave. A harmful check can look boring and still be wrong.

It does not measure the free option you might rent. It does not prove isolation, uptime, or data deletion. You must read the current vendor terms yourself.

Do not use this path for a production apply. Do not use it on repos that hold live secrets. Do not use it if you cannot reject a fluent patch.

Skip the path when the check needs private customer data. Skip it when the failure domain includes safety hardware. A free bench is the wrong room for that weight.

Skip it when your team needs an audited record you do not control. This note is a local habit you can show a reviewer. It is not a compliance product and should not pretend.

Close the review

File the change note on your own machine first. Run the local review until the ticket is clean. Then spend a free model call on the question you already wrote.

If a free server is still the right bench, use the ticket only. Read the current MonkeyCode plan once before you copy files. Decide with your constraints in view, not with a slogan.

Name Failure Domains Before Apply

Harper Xu — Sun, 11 Oct 2026 21:50:30 +0000

You do not start an architecture review inside the diff. You start by naming the failure domain that can hurt you. A clean patch can still cross a boundary you never drew.

Think of the change as water in a set of pipes. The pipes matter more than the color of the water. If a pipe leaks, the patch quality will not save you.

Your first constraint is the scope of files. You name the files the change may touch. You also name the files it must not touch.

A suggestion that wanders past that fence fails review. That remains true even when the code compiles. Scope is a wall, not a hopeful wish.

Your second constraint is the path data takes. Prompts, file snippets, and test logs either stay on your machine or they leave. You write that choice down before you ask any model for help.

Your third constraint is where the patch lands. Apply stays on a branch that you control. Or it lands on a machine you do not administer.

Those two targets are different failure domains entirely. Do not blur them because the suggestion looks small. A small diff can still open the wrong door.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode offers free model access and a free server option. This review uses only those two supplied facts.

It does not assume a model name or a quota. It also skips hardware shape and any duration. You read the live plan yourself before you rely on either offer.

Treat the free server as a rented workbench, not as your archive. You can try a patch there when your laptop is the wrong place. You still keep the source of truth on a host you can inspect.

Here is the data flow you should draw before you paste anything. Your intent stays written in your own notes. The context you select then moves to the model.

The suggestion comes back to you as text. That text then sits in a review file. Apply happens only after the domain map says yes.

That flow has four hops, and each hop can fail alone. A failure in hop two does not excuse a failure in hop four. You review them as separate rooms in one building.

Hop one is the intent you already wrote. You write the constraint in a short file, not in a chat scroll. If the constraint is fuzzy, the model will invent a goal.

You cannot review an invented goal after the fact. A fuzzy intent is how drift enters the building. Write the goal in one sentence you can test.

Hop two is the egress of selected context. You choose snippets, not the whole tree, unless the tree is already public. Secrets, tokens, and customer paths do not ride along.

If you cannot redact, you do not send. Egress is a door you choose to open. Once the snippet leaves, you cannot pull the meaning back.

Hop three is the proposal text you received. The model returns a suggestion you can store. You store it as a patch file with a timestamp.

You do not apply it from the chat window. The chat is a transcript, not a release. A transcript can vanish, while a patch file can be diffed.

Hop four is where you apply the patch. You apply on a branch, or you apply inside the free server sandbox. You run the tests that belong to the touched files.

A green test on the wrong host is not evidence. Evidence is a file you can still open tomorrow. If that file is missing tomorrow, the hop already failed.

Failure domains are the rooms where a hop can burn you. Name each one before you admire the diff. You can use five names and keep them stable.

The egress domain fails when private data leaves. The drift domain fails when the patch solves a different problem. The apply domain fails when the patch lands on the wrong ref.

The custody domain fails when you cannot retrieve remote files. The evidence domain fails when no file can prove what ran. Five rooms share one building and no excuse.

The custody domain is about getting files back. It is not about how much room you were given. A free server may hold a workspace for a while.

That hold is not a backup you can trust. It is also not a promise you should invent. Read the current plan, then assume the workspace can vanish.

You can encode this review as a small gate. The gate reads a manifest you write by hand. It refuses apply when the map is incomplete or unsafe.

The script below is a proposal you can run locally. This proposal claims no benchmark and no vendor API. It only checks the map you already wrote.

Use hold when the map is fine but you are not ready. Hold stores the decision without landing any patch. Apply stays closed until you change that target on purpose.

#!/usr/bin/env python3
# Local domain gate. Proposal only. It makes no network calls.

import json
import sys

REQUIRED = (
    "intent",
    "touched",
    "forbidden",
    "egress",
    "apply_target",
    "evidence",
)
TARGETS = {"local-branch", "free-server-sandbox", "hold"}
EVIDENCE = {"patch-file", "patch-file+tests"}

def load(path):
    with open(path, encoding="utf-8") as handle:
        return json.load(handle)

def domain_failures(spec):
    failures = []
    missing = [key for key in REQUIRED if not spec.get(key)]
    if missing:
        failures.append("map-incomplete:" + ",".join(missing))
        return failures
    if spec.get("egress") == "external" and spec.get("secrets") is True:
        failures.append("egress-domain:secrets-would-leave")
    overlap = set(spec.get("touched") or []) & set(spec.get("forbidden") or [])
    if overlap:
        failures.append("drift-domain:forbidden-overlap")
    target = spec.get("apply_target")
    if target not in TARGETS:
        failures.append("apply-domain:unknown-target")
    if target == "free-server-sandbox" and not spec.get("source_of_truth"):
        failures.append("custody-domain:no-local-source-of-truth")
    if spec.get("evidence") not in EVIDENCE:
        failures.append("evidence-domain:no-local-artifact")
    if target != "hold" and spec.get("waiver") != "reviewed":
        failures.append("gate:apply-without-waiver")
    return failures

def paths_in_diff(diff_text):
    paths = []
    for line in diff_text.splitlines():
        if line.startswith("+++ b/"):
            paths.append(line[6:])
    return paths

def extra_paths(diff_text, touched):
    allowed = set(touched)
    return [path for path in paths_in_diff(diff_text) if path not in allowed]

def main():
    spec = load(sys.argv[1])
    failures = domain_failures(spec)
    if len(sys.argv) > 2:
        with open(sys.argv[2], encoding="utf-8") as handle:
            extra = extra_paths(handle.read(), spec.get("touched") or [])
        if extra:
            failures.append("drift-domain:patch-outside-touched")
    if failures:
        print("REFUSE")
        for item in failures:
            print(item)
        return 2
    print("ALLOW")
    print("target=" + spec["apply_target"])
    return 0

if __name__ == "__main__":
    sys.exit(main())

Save that checker as domain_gate.py on your machine. Then write a manifest that matches a real change. Do not let a model fill the forbidden list.

That list is your constraint, not a suggestion. A model that edits the fence is already inside drift. You keep the pen when you write that list.

{
  "intent": "Rename the retry helper without changing its public signature.",
  "touched": ["src/retry.py", "tests/test_retry.py"],
  "forbidden": ["src/secrets.py", ".env", "deploy/"],
  "egress": "external",
  "secrets": false,
  "apply_target": "local-branch",
  "source_of_truth": "git@origin/main",
  "evidence": "patch-file+tests",
  "waiver": "reviewed"
}

Run the domain gate before you apply anything. A refuse result is the review, not a nuisance. Read the exit code before you read the diff.

python3 domain_gate.py review.json suggestion.patch
status=$?
echo "exit=${status}"
test "${status}" -eq 0 || exit "${status}"
git switch -c review/retry-helper
git apply --check suggestion.patch
git apply suggestion.patch
python3 -m pytest tests/test_retry.py -q

If the gate prints an egress failure, stop and redact. If it prints a custody failure, keep the patch file local. Do not treat the free server as the archive.

Walk a concrete miss so the rooms feel real. You asked for a rename of a retry helper. The suggestion also edits the config loader to be helpful.

The touched paths and forbidden paths now overlap. The gate refuses, and that refuse is the architecture talking. You do not negotiate with the patch to stay green.

REFUSE
drift-domain:forbidden-overlap

You shrink the request and then ask again. You keep the same forbidden list in place. Helpful extra edits are drift, even when they look kind.

Walk a second miss on the free server. Tests pass in the sandbox during the afternoon. Next morning the workspace is gone, and so is your only copy.

The evidence domain failed when the patch never came home. The free server did not betray a quota you invented. It simply was never meant to be your archive.

You change the order of hops, not the slogan. Write the patch file locally before any remote run. Copy out logs you care about before you close the session.

A second manifest shows the safer sandbox path. You still name a source of truth you control. The free server is where you try the hop.

{
  "intent": "Try the same rename where the local machine is busy.",
  "touched": ["src/retry.py", "tests/test_retry.py"],
  "forbidden": [".env", "secrets/"],
  "egress": "external",
  "secrets": false,
  "apply_target": "free-server-sandbox",
  "source_of_truth": "/work/repo",
  "evidence": "patch-file+tests",
  "waiver": "reviewed"
}

python3 domain_gate.py sandbox.json suggestion.patch
# copy only touched files, run the tests, bring the patch file back

You should change the gate next, not the model. Turn the waiver into a commit message line, not a loose flag. Add a check that the patch edits only touched paths.

Expire sandbox copies on a clock you choose. A vendor clock is not your own constraint. Bring the patch home before you call the hop done.

Do not change the model as your first move. A sharper model does not shrink an unnamed failure domain. It only writes a more confident patch in the same pipes.

Here is the analogy to keep on the wall. A building inspector does not start by praising the paint. The inspector walks the load path, the exits, and the fire spread.

Your domain map is the inspection walk itself. The diff is only the paint on the wall. Fresh paint on a cracked beam is still a cracked beam.

Limits sit in plain sight for this gate. This gate cannot see secrets you forgot to mark. It cannot prove the free server isolated your work.

It cannot tell you a quota or a region. It cannot tell you how long a workspace lives. Those facts change, so you read them at use time.

This article will not freeze those live facts for you. Do not use this approach for regulated data that must stay local. Do not use it when you need a durable build farm.

Do not use it if a tool should merge to the default branch. The gate is a review aid, not a release system. If the manifest and the patch disagree, believe the manifest.

Throw the patch away and ask for a smaller suggestion. You should not widen touched paths just to make the gate go green. That move hides drift instead of naming it.

A green gate on a lying map is worse than a red gate. Free model access helps when the hop is small and the map is complete. A free server helps when you need a disposable place to run that hop.

Neither offer replaces the walk through failure domains. If you try that walk with MonkeyCode, confirm the live plan first. Then run this gate on a scratch branch before you trust a suggestion.

Review the Free Lane Before You Cross

Harper Xu — Sat, 10 Oct 2026 18:25:19 +0000

A free lane is useful only after you name its failure domains. You do not receive custody because the invoice says zero. You receive a temporary neighbor that can propose and can check.

People can generate a patch in minutes, and that speed is the trap. The patch still has to cross rooms that do not share a fate. You review the crossing before you celebrate the speed.

That neighbor can leak, stall, or invent a confident diff. Your review starts with boundaries, not with excitement about price. Price is a constraint, and constraints belong on the diagram.

The outreach brief supplied two availability claims for this review. One claim is free model access, and the other is a free server option. Neither claim names a token cap, a machine size, or an end date.

The brief presents MonkeyCode as an open source project you can inspect. You place that project in the neighbor seat, not in the custody seat. Disclosure: This article was prepared as part of MonkeyCode's product outreach.

This review uses it only as that remote neighbor, not as a proven benchmark. Do not freeze a quota from a chat, a slide, or an old post. A number you cannot open on the project page is not a plan.

You check the live page on the day you depend on the lane. A stale allowance will bend the design around a promise you cannot keep. You would rather drop the lane than invent a limit.

Hold the constraints

You keep four constraints even when the lane costs nothing. The packet may describe the work, but it may not carry secrets. The free server may check code, but it must not hold production keys.

The proposal may look finished, but it must not merge itself. A fifth rule sits quietly beneath those four hard constraints. The free workspace is ephemeral, so you do not store state there.

If a check needs a cache, you rebuild that cache inside the job. You also cap time, because a free neighbor can stall without warning. A hung check should fail closed, and your queue should move on.

Walk the four rooms

Picture the path as four rooms with one-way doors. Room one is your editor, where the spec and the tests already live. Room two is the packet builder, which strips names, tokens, and host paths.

Room three is the free model, which may only see that stripped packet. It returns a patch proposal and a short note about assumed versions. Room four is the free server, which may run only an allowed check.

Those doors do not reverse, and that direction is the point. Logs may come home, but home state does not chase them. You apply nothing until a local gate accepts the returned diff.

The model room and the server room are different failure domains. A clever diff is not permission to execute that diff abroad. You insert a broker so those rooms cannot talk in raw editor text.

Watch a retry bug tempt you

Take a retry helper that fails only when a key is missing. You will want to paste the environment file, because the trace mentions it. That urge is the transit failure, and it shows up early.

The packet builder should drop the file and keep the error class. The model can still suggest a test that stubs the missing key. The free server can run that test if the command is allowed.

You do not send the real key to make the foreign run look real. A leaked run is a worse design, not a more honest one. Realism stops where custody starts, and custody stays on your machine.

Name the failure domains

Four failure domains sit on that path, and they do not share a fate. The model domain fails by being wrong, stale, or overly sure. You treat its patch as a draft from a fast colleague, not as law.

The transit domain fails when the packet holds more than the task. A send crosses a boundary, even when the destination looks friendly and free. You redact before send, and you log the redaction, not the secret.

The free server domain fails as a shared, short-lived workshop. A noisy neighbor can slow you, and a reset can delete scratch files. You assume the disk vanishes, and you assume the network is hostile.

The local domain fails when you skip the gate because the run looked green. Green on a foreign host is a clue, not a release decision. Your machine still owns format, review, merge, and the final test.

Encode the review

You can encode this review as a small Python gate. The script below is a proposal, and this draft did not execute it. It refuses a packet that crosses a domain line you already named.

Run it against a fixture before you trust it in a hook. A reject on a dirty packet is the success case you want first. An accept should be rare, boring, and easy to explain.

#!/usr/bin/env python3
'''Proposal only. This gate was not run against a live service.'''

import json
import sys
from pathlib import Path

FORBIDDEN_KEYS = {
    'api_key', 'token', 'secret', 'password', 'private_key', 'authorization',
}
FORBIDDEN_PARTS = {'ssh', 'scp', 'curl', 'wget', 'nc', 'kubectl', 'terraform'}
ALLOWED_HEADS = {'pytest', 'python', 'ruff', 'git'}


def load_packet(path):
    data = json.loads(path.read_text(encoding='utf-8'))
    if not isinstance(data, dict):
        raise SystemExit('packet must be an object')
    return data


def looks_like_secret(text):
    markers = ('sk-', 'ghp_', 'AKIA', '-----BEGIN', 'xoxb-')
    return any(marker in text for marker in markers)


def find_secret_paths(value, prefix=''):
    hits = []
    if isinstance(value, dict):
        for key, item in value.items():
            path = f'{prefix}.{key}' if prefix else key
            if str(key).lower() in FORBIDDEN_KEYS:
                hits.append(path)
            hits.extend(find_secret_paths(item, path))
    elif isinstance(value, list):
        for index, item in enumerate(value):
            hits.extend(find_secret_paths(item, f'{prefix}[{index}]'))
    elif isinstance(value, str) and looks_like_secret(value):
        hits.append(prefix or 'value')
    return hits


def review(packet):
    failures = []
    domain = packet.get('domain')
    if domain not in {'proposal', 'check'}:
        failures.append('domain must be proposal or check')
    if packet.get('contains_secrets') is True:
        failures.append('packet admits secrets')
    for hit in find_secret_paths(packet.get('inputs', {})):
        failures.append(f'secret-like field: {hit}')
    command = str(packet.get('command') or '')
    parts = command.split()
    head = parts[0] if parts else ''
    if domain == 'proposal' and command:
        failures.append('proposal domain cannot carry a command')
    if domain == 'check':
        if head not in ALLOWED_HEADS:
            failures.append('command not allowed: ' + (head or 'empty'))
        if any(part in FORBIDDEN_PARTS for part in parts):
            failures.append('command crosses a network or admin line')
        timeout = int(packet.get('timeout_seconds') or 0)
        if timeout <= 0 or timeout > 120:
            failures.append('timeout must sit between 1 and 120 seconds')
    if packet.get('persist_workspace') is True:
        failures.append('remote workspace must not persist')
    if packet.get('apply_on_remote') is True:
        failures.append('remote apply is forbidden')
    return failures


def main():
    if len(sys.argv) != 2:
        raise SystemExit('usage: review_free_lane.py <packet.json>')
    packet = load_packet(Path(sys.argv[1]))
    failures = review(packet)
    if failures:
        print('reject')
        for item in failures:
            print(item)
        return 2
    print('accept')
    print(packet.get('domain'))
    return 0


if __name__ == '__main__':
    raise SystemExit(main())

Save the proposal as review_free_lane.py and keep the fixtures beside it. The dirty fixture is the case you want to see fail first. The clean fixture is the narrow case you are willing to send.

{
  "domain": "check",
  "contains_secrets": false,
  "persist_workspace": false,
  "apply_on_remote": false,
  "timeout_seconds": 30,
  "command": "curl https://example.invalid",
  "inputs": {
    "note": "retry failed",
    "api_key": "sk-demo-not-real"
  }
}

{
  "domain": "check",
  "contains_secrets": false,
  "persist_workspace": false,
  "apply_on_remote": false,
  "timeout_seconds": 60,
  "command": "pytest tests/test_retry.py",
  "inputs": {
    "error_class": "MissingKey",
    "note": "stub the key in the test"
  }
}

mkdir -p fixtures
jq -n --arg note 'stub the missing key' \
  '{domain:"check",contains_secrets:false,persist_workspace:false,apply_on_remote:false,timeout_seconds:60,command:"pytest tests/test_retry.py",inputs:{error_class:"MissingKey",note:$note}}' \
  > fixtures/clean.json
python3 review_free_lane.py fixtures/dirty.json
echo dirty_exit:$?
python3 review_free_lane.py fixtures/clean.json
echo clean_exit:$?

A dirty fixture should fail on a secret-shaped field and a network command. A clean fixture should pass with pytest and a short timeout. You keep both fixtures beside the script so the rule stays visible.

Expect the dirty file to print reject and exit two. Expect the clean file to print accept and exit zero. Those outcomes are designed behavior, not a measured run from this draft.

You do not pipe a raw chat log into this gate. A chat log is a swamp of paths, names, and half secrets. The packet is a small contract, and the contract is what you review.

Change the design next

The next design change is a narrow broker between the two remote rooms. The broker may forward a proposal id, but not a raw buffer. That cut stops a bad draft from becoming a server script.

After the first quiet week, you split reads from any write tool. You pin allowed commands in a file the gate reads each run. You store job ids locally, beside the spec, not on the server.

A later run should rebuild the check from the spec and the job id. That leaves durable history on the machine that already holds the repo. You would not add deploy rights just to make the lane feel finished.

You would also record which domain produced each artifact in the log. A proposal diff and a check log should never wear the same label. Mixed labels are how a foreign green bar sneaks into a release note.

Say the limits

This design has sharp edges, and you should say them out loud. The gate scans shapes, but it can miss a secret hidden in prose. A determined paste can still bypass a script that you forget to run.

Shared tenancy means timing noise, so do not benchmark the free server here. This draft records no latency, no token count, and no hardware claim. If a page quotes those figures, cite that page, not this review.

The model can recommend an API that your lockfile no longer contains. You confirm package versions locally before you accept the patch. Free access can also shrink or pause, so keep a local fallback flow.

Who should walk away

Do not use this path for regulated data, customer dumps, or private keys. Do not use it when you cannot explain the packet to a reviewer. Do not use it as a production runner or as a durable disk.

If policy forbids third-party inference, the free model is already out. If you need a guaranteed quota, wait until the live page states one. You can still learn the gate on a toy repo with fake values.

The useful move is boring, and that boredom is why it survives. You name the domains, cut the packet, and keep the merge local. A free neighbor can speed a draft, but it cannot own the system.

Read the current MonkeyCode limits, then try one toy repo only. Stop if the live page no longer matches the brief you were given.