DEV Community: Emery Li

Measuring Before Migrating: A Residency Smoke Test for Local-First Agents and Free Servers

Emery Li — Tue, 27 Oct 2026 12:49:28 +0000

An agent that feels instant on a laptop can feel broken the moment its model call crosses a network boundary, and the postmortem usually blames the model. The regression more often comes from three numbers nobody recorded before the move: cold-start latency, payload size leaving the device, and behaviour when the remote seat is unreachable. The scenario below is illustrative rather than a customer story, but the failure shape is common enough to plan around.

This article builds a small probe that records those three numbers for any endpoint pair and turns them into a residency decision. It assumes you already have an agent that runs end to end and you are choosing where the model call happens. Nothing here depends on one vendor, and every threshold below is a starting point you should retune for your own product.

Why three measurements beat one benchmark

Public leaderboards measure model quality on fixed prompts, and they say almost nothing about the machine your agent actually runs on. Residency is a property of your process, your network path and your prompt size, so it has to be measured in place. A single latency number is also easy to game: a warm local runtime can win on the median and lose badly on the first request after a laptop wakes up.

The probe therefore records a distribution rather than a point estimate, and it pairs that distribution with two non-latency signals. The payload audit answers what content leaves the device, which matters when prompts carry repository context or customer records. The offline check answers whether the agent degrades into a queue or into an exception, and that difference decides how much of the fallback path you have to build.

Step 1: Fix one request shape for both endpoints

The comparison is only meaningful when both sides receive the same bytes, so define one request body and reuse it. Keep the system prompt and user prompt in files, and let environment variables supply the base URLs. This also makes the probe cheap to rerun whenever a runtime updates.

mkdir -p prompts
printf 'Summarise the three functions in this diff.\n' > prompts/short.txt

export LOCAL_BASE_URL="http://127.0.0.1:8080/v1/complete"
export REMOTE_BASE_URL="https://your-endpoint.example/v1/complete"

The two URLs are placeholders, and the script below does no vendor-specific shaping. If your local runtime expects a different JSON envelope, adapt build_body once and keep the rest identical. That single edit point is what makes the numbers comparable later.

Step 2: Measure first body byte, with explicit honesty about it

Time to first token is the metric developers feel, but most HTTP clients only expose time to first body byte. The probe reads exactly one byte and stops, then labels the result as an approximation. It also reports p50 and p95 separately, because tail behaviour is where remote seats and cold local runtimes diverge.

The script also counts failures instead of hiding them behind a mean. A run that times out is data, and a remote seat that fails two of seven attempts changes the architecture conversation more than a hundred milliseconds of median latency.

#!/usr/bin/env python3
"""residency_probe.py - reference implementation, not a benchmark suite.

Records first-body-byte latency, payload size, secret-pattern hits and
forced-offline behaviour for up to two endpoints. Every number it prints
is yours to collect; no vendor performance is asserted here.
"""
from __future__ import annotations

import argparse
import json
import os
import re
import socket
import statistics
import time
import urllib.error
import urllib.request

SECRET_PATTERNS = {
    "provider_key": re.compile(r"sk-[A-Za-z0-9]{16,}"),
    "bearer_value": re.compile(r"(?i)bearer[ ]+[A-Za-z0-9._-]{12,}"),
    "aws_secret": re.compile(r"(?i)aws_secret_access_key[ ]*[:=]"),
}


def build_body(prompt: str, system: str) -> bytes:
    return json.dumps({"system": system, "prompt": prompt}).encode("utf-8")


def scan_for_secrets(raw: bytes) -> list[str]:
    text = raw.decode("utf-8", errors="replace")
    hits = [name for name, pat in SECRET_PATTERNS.items() if pat.search(text)]
    for key in ("API_KEY", "ACCESS_TOKEN", "SECRET"):
        value = os.environ.get(key, "")
        if len(value) >= 12 and value in text:
            hits.append("env:" + key)
    return hits


def first_body_byte_ms(url: str, body: bytes, timeout: float) -> float:
    req = urllib.request.Request(
        url, data=body, headers={"Content-Type": "application/json"}
    )
    start = time.perf_counter()
    with urllib.request.urlopen(req, timeout=timeout) as resp:
        resp.read(1)  # first body byte, NOT first token
    return (time.perf_counter() - start) * 1000.0


def probe(url: str, body: bytes, attempts: int, timeout: float) -> dict:
    samples, failures = [], 0
    for _ in range(attempts):
        try:
            samples.append(first_body_byte_ms(url, body, timeout))
        except (urllib.error.URLError, socket.timeout, TimeoutError, ConnectionError):
            failures += 1
    ordered = sorted(samples)

    def pct(p: float) -> float | None:
        if not ordered:
            return None
        idx = min(len(ordered) - 1, int(round(p * (len(ordered) - 1))))
        return round(ordered[idx], 1)

    return {
        "url": url,
        "attempts": attempts,
        "failures": failures,
        "p50_ms": pct(0.50),
        "p95_ms": pct(0.95),
        "mean_ms": round(statistics.fmean(samples), 1) if samples else None,
    }


def offline_behaviour(fallback) -> dict:
    """Point at the discard port, then check the fallback still returns work."""
    dead = "http://127.0.0.1:9/v1/complete"
    started = time.perf_counter()
    try:
        first_body_byte_ms(dead, b"{}", timeout=2.0)
        reached = True
    except (urllib.error.URLError, socket.timeout, TimeoutError, ConnectionError):
        reached = False
    result = fallback()
    return {
        "endpoint_reachable": reached,
        "fallback_returned": result is not None,
        "fallback_seconds": round(time.perf_counter() - started, 3),
    }


def main() -> None:
    ap = argparse.ArgumentParser()
    ap.add_argument("--prompt", default="prompts/short.txt")
    ap.add_argument("--attempts", type=int, default=7)
    ap.add_argument("--timeout", type=float, default=20.0)
    ap.add_argument("--out", default="residency_report.json")
    args = ap.parse_args()

    prompt = open(args.prompt, encoding="utf-8").read().strip()
    body = build_body(prompt, system="residency-probe")
    report = {
        "payload_bytes": len(body),
        "payload_secret_hits": scan_for_secrets(body),
        "endpoints": [],
        "offline": offline_behaviour(lambda: {"queued": True}),
    }

    for name in ("LOCAL_BASE_URL", "REMOTE_BASE_URL"):
        url = os.environ.get(name)
        if not url:
            continue
        row = probe(url, body, args.attempts, args.timeout)
        row["name"] = name
        report["endpoints"].append(row)

    with open(args.out, "w", encoding="utf-8") as fh:
        json.dump(report, fh, indent=2)
    print(json.dumps(report, indent=2))


if __name__ == "__main__":
    main()

Run it twice: once with the local runtime freshly started, and once after it has served a few requests. Those two runs are the cheapest way to see cold-start cost without building a load generator.

Step 3: Audit what actually leaves the device

The payload audit is deliberately blunt, and it is not a data-loss-prevention product. It flags obvious credential shapes and any long environment value that leaked into the serialised body, which catches the common accident of pasting a token into a system prompt. A hit is a stop sign for a remote seat and a warning for a local one, because local runtimes log too.

The byte count matters as much as the hits. A prompt carrying an entire diff can be orders of magnitude larger than a chat message, and upload time on a home connection scales with it. Record the byte count next to the latency figures, or you will attribute a bandwidth problem to model speed.

Step 4: Force the offline path on purpose

The probe points at a dead port and then calls the same fallback your agent would call. If the fallback raises instead of returning queued work, the agent needs a fallback contract before any residency change ships. If it returns queued work in a few seconds, the offline story is already credible.

This check is cheap and it prevents the most expensive migration surprise. A remote seat that disappears during a demo is survivable when work queues locally and can be resumed. It is fatal when the whole run lives in a request handler that has already returned.

A decision table you can argue with

The thresholds below are operator-tunable starting points rather than published guidance. Fill the middle columns with your own probe output and use the last two as the interpretive lens.

Signal	What local-first tends to give	What a free server tends to give	Read it as
Warm p95 first body byte	Low and stable once the runtime is pinned	Depends on queueing at the shared endpoint	Local wins on steady interactive loops
Cold start	Long first request after sleep or restart	Endpoint is already warm	Remote wins on sporadic use
Payload bytes off device	Zero for the model call	Full prompt, including context	Sensitive context argues for local
Secret-pattern hits	Still a bug, still logged	Blocks the send until redacted	Any hit stops the remote option
Offline behaviour	Usable if you persist state	Fails closed without a queue	Queue design is the real work
Concurrency shape	Bounded by one machine	Bursts absorbed elsewhere	Spiky workloads favour remote

The table is a discussion device, not a verdict. Two teams with identical numbers can land on different rows because their data classification and support burden differ.

Where a free server genuinely wins

A recurring discussion in developer circles concerns how many intermediaries sit between a developer and a model, and residency is the concrete version of that question. The honest answer is that self-hosting is sometimes the wrong trade, especially when the local machine cold-starts constantly or has no accelerator at all. Spiky workloads, shared team artifacts and short-lived experiments all favour a remote seat that is already warm.

MonkeyCode, an open-source coding agent project, offers free model access and a free server option, which makes it a reasonable second column for the probe above. Disclosure: This article was prepared as part of MonkeyCode's product outreach. Token allowances and server terms change, so read the project page for the current limits before you design around them, and never assume a free tier is permanent.

The useful pattern is to run the probe against a free remote seat and against your laptop on the same day with the same prompt file. That produces a defensible comparison instead of an intuition, and it costs one environment variable.

Limitations and who should not use this

The probe measures first body byte, not time to first token, and the two diverge when a server streams headers early. It is a single-machine instrument, so it is not a load test and it will not reveal behaviour under ten concurrent runs. The secret scan is a regex tripwire, not a compliance control, and it will miss bespoke credential formats.

Teams with hard data-residency obligations should keep the model call local and treat the remote column as unavailable rather than slow. Interactive products with tight latency budgets should not adopt a shared endpoint without a warm-up strategy and a measured tail. Anyone expecting deterministic throughput from a free tier should build the queue first and treat the server as an accelerator, not a guarantee.

If you want a second column without provisioning anything, point the probe at a free remote seat first and keep the local runtime as the fallback. The numbers you collect will tell you more than any migration plan written before them.

Routing Agent Jobs on p95 Time-to-First-Token, Not Average Latency

Emery Li — Mon, 26 Oct 2026 11:28:14 +0000

A missed deadline the averages never showed

A two-person team runs three agent jobs from one laptop: inbox triage that must land inside forty seconds, a log summariser that can take three minutes, and a nightly report with a nine-minute window. On most days all three finish early. On the day the laptop suspended mid-run, triage resumed into a stale socket, retried twice, and delivered at 4 minutes 12 seconds. The dashboard reported mean latency of 3.1 seconds for the same period. This scene is a composite rather than a measured customer case, but the shape is familiar: the mean stayed green because the tail was rare, and the tail was the only part with a deadline.

The fix is not a faster model. It is to treat routing as admission control over jobs with deadlines, and to rank candidate lanes by a tail statistic instead of an average.

Why p95 time-to-first-token is the useful unit

Time-to-first-token (TTFT) covers everything a caller waits through before any useful output exists: queueing, cold start, connection setup, and prompt prefill. Total completion time still matters, but for streaming agents the perceived stall is TTFT, and the tail is where deadlines break. A lane advertising a 900 ms mean can hide a 6 s p95, which means a job with a 5 s budget fails roughly once in twenty runs while every average-based chart stays flat.

Measure the distribution per lane and per prompt-size class, and keep the raw samples on disk. Two lanes that look identical on averages routinely differ by a factor of three at p95 once cold starts are included.

Step 1: Probe each lane and store a distribution

The probe below records TTFT samples for streaming chat completions against any OpenAI-compatible endpoint. It uses only the standard library, so it runs on the same restricted box as the agent itself. Collect at least fifty samples per lane before trusting the quantiles.

# probe_ttft.py - measure TTFT samples for one lane
import json, os, time, urllib.request

BASE   = os.environ['LANE_BASE_URL']        # e.g. https://host/v1
KEY    = os.environ.get('LANE_API_KEY', '')
MODEL  = os.environ['LANE_MODEL']
PROMPT = 'Return the word ready and nothing else.'

def one_probe(timeout=30.0):
    body = json.dumps({'model': MODEL, 'stream': True,
                       'messages': [{'role': 'user', 'content': PROMPT}]}).encode()
    req = urllib.request.Request(
        f'{BASE}/chat/completions', data=body,
        headers={'Content-Type': 'application/json',
                 'Authorization': f'Bearer {KEY}'})
    t0 = time.monotonic()
    with urllib.request.urlopen(req, timeout=timeout) as resp:
        for raw in resp:
            line = raw.decode('utf-8', 'replace').strip()
            if line.startswith('data:') and '[DONE]' not in line:
                return time.monotonic() - t0
    return time.monotonic() - t0

def quantile(xs, q):
    xs = sorted(xs)
    if not xs:
        return None
    return xs[min(len(xs) - 1, int(round(q * (len(xs) - 1))))]

if __name__ == '__main__':
    samples, errors = [], 0
    for _ in range(int(os.environ.get('LANE_SAMPLES', '50'))):
        try:
            samples.append(one_probe())
        except Exception:
            errors += 1
        time.sleep(0.2)
    print(json.dumps({'lane': os.environ['LANE_NAME'], 'n': len(samples),
                      'errors': errors, 'p50': quantile(samples, 0.50),
                      'p95': quantile(samples, 0.95),
                      'max': max(samples, default=None)}))

Run it once per candidate lane and keep the error count alongside the quantiles. A lane that times out on ten percent of probes is not a fallback; it is a second outage arriving on schedule.

Step 2: Turn measurements into a lane scorecard

The scorecard adds the fields a probe cannot measure: whether the lane can hold a secret, and whether it survives with no network. The values below are illustrative placeholders, not benchmarks.

# lanes.py - replace every number with your own probe output
LANES = [
    {'name': 'local',       'p95_ttft_s': 1.4, 'retry_rate': 0.05, 'handoff_s': 0.2,
     'offline_capable': True,  'can_hold_secret': True,  'rank': 0},
    {'name': 'free_server', 'p95_ttft_s': 2.6, 'retry_rate': 0.20, 'handoff_s': 0.8,
     'offline_capable': False, 'can_hold_secret': False, 'rank': 1},
]

This is where a hosted lane earns its place. The operator describes MonkeyCode as an open-source project offering free model access and a free server option, along with a 10M-token free allowance; I have not independently audited those terms, so check the project's current pages before depending on them. What matters for routing is narrower: a free server lane adds a third column to the scorecard that costs nothing when idle, and that idle capacity is exactly what a deadline-aware controller needs. Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Step 3: Encode the policy as an admission controller

The controller answers one question per job: which lane can meet this deadline with the least expected cost. Expected cost multiplies p95 by the observed retry rate and adds the handoff penalty, which penalises flaky lanes far more honestly than a mean does.

# admit.py - choose a lane for a job with a deadline
SAFETY_MARGIN_S = 1.0

def eligible(lane, job):
    if job['secret_class'] == 'restricted' and not lane['can_hold_secret']:
        return False
    if job['requires_offline'] and not lane['offline_capable']:
        return False
    return True

def expected_cost(lane):
    return lane['p95_ttft_s'] * (1 + lane['retry_rate']) + lane['handoff_s']

def choose(job, lanes):
    budget = job['deadline_s'] - SAFETY_MARGIN_S
    ranked = sorted((l for l in lanes if eligible(l, job)),
                    key=lambda l: (expected_cost(l), l['rank']))
    for lane in ranked:
        if expected_cost(lane) <= budget:
            return lane['name'], expected_cost(lane)
    return None, None   # refuse early instead of missing silently

Two behaviours matter more than the ranking rule. A job carrying a restricted credential can never be admitted to a lane with can_hold_secret = False, regardless of how much deadline headroom exists. And when no lane fits, the controller returns None immediately, so the caller degrades its output instead of discovering the miss four minutes later.

Step 4: Log every lane decision

An admission decision that is not logged cannot be audited. Emit one line per job with the chosen lane, the expected cost, the budget, and the eventual outcome.

python probe_ttft.py >> ttft_samples.jsonl
python -c 'from admit import choose; from lanes import LANES; \
print(choose({"deadline_s": 40, "secret_class": "restricted", \
"requires_offline": False}, LANES))'

After a week, compare the logged expected_cost against the real completion time. A persistent gap in one direction means the scorecard is stale, and re-probing that lane is cheaper than debugging a mystery deadline miss later.

The tradeoff, stated plainly

Job property	Local-first wins when	Free server lane wins when
Latency	p95 stays under budget once the model is warm	local p95 exceeds the deadline on cold start
Secrets	credentials must never leave the machine	the job carries no restricted material
Offline	work must continue with no network	connectivity is assumed and monitored
Capacity	job volume fits one device at night	bursts outrun the device, and idle capacity is free

The table is not a recommendation to move everything. It is a filter: most jobs are local-first, a narrow band of stateless, network-tolerant, secret-free jobs benefits from an external lane, and the controller should make that band explicit rather than accidental.

Where this breaks, and who should skip it

Single-sample probing produces noise, so anyone unwilling to schedule repeated probes will get a scorecard that misroutes confidently. Lanes with wildly bimodal latency need separate distributions per prompt-size class, which doubles the bookkeeping. Teams already running a real queue with priority and preemption should use that scheduler instead, because a sixty-line controller duplicates its weakest parts.

Skip this approach entirely if your jobs have no deadlines, if every job carries restricted credentials, or if you cannot store and review per-lane samples. The controller is a deadline tool, not a general load balancer.

The script and the scorecard format run unchanged against any OpenAI-compatible endpoint, including the free tier the operator provides, so a first comparison costs one afternoon and fifty probes per lane.

A Residency Lease for Agent Runs: Expiring Placement Decisions Instead of Static Config

Emery Li — Tue, 20 Oct 2026 09:33:29 +0000

A Residency Lease for Agent Runs: Expiring Placement Decisions Instead of Static Config

Reproducing the failure takes about a minute on any laptop: start an agent turn against a remote model endpoint, then drop the network with nmcli networking off halfway through. The run rarely fails loudly. It stalls, retries, and eventually writes a truncated artifact that still validates against the schema.

The placement config said remote, and nothing in that config knew the link had disappeared ten minutes earlier. A boolean flag cannot expire, so the run kept honouring a decision whose supporting evidence was gone. The fix is not a better flag; it is a decision that carries its own expiry date.

Why a static placement flag cannot survive a long run

Placement is normally decided once, at process start, on the assumption that the conditions measured at start still hold. Agent runs break that assumption in ordinary ways: tool calls take minutes, laptops sleep, tunnels drop, and a shared office uplink degrades at the exact moment a batch job begins. A static config records intent; it does not record evidence, and it cannot tell you when it stopped being true.

A lease replaces the flag with a small, immutable record: the chosen placement, the moment it expires, and the measurements that justified it. Invariants such as "this payload must not leave the host" produce long leases because they do not change with the weather. Measurements such as round-trip latency and packet loss produce short leases because they do.

The lease record

The data model is deliberately boring. Constraints are inputs, a lease is an output, and validity is a pure function of the clock, which makes every decision testable without a network.

# lease.py — a self-expiring placement lease for local-first agent runs
from __future__ import annotations
import os, re, statistics, socket, time
from dataclasses import dataclass
from enum import Enum

class Placement(str, Enum):
    LOCAL = "local"
    REMOTE = "remote"

@dataclass(frozen=True)
class Constraints:
    secret_free_local: bool   # payload may not leave the host
    offline_capable: bool     # must finish with no network at all
    p95_budget_ms: int        # per-turn latency budget

@dataclass(frozen=True)
class Lease:
    placement: Placement
    expires_at: float
    evidence: dict

    def valid(self, now: float | None = None) -> bool:
        return (time.time() if now is None else now) < self.expires_at

Because Lease is frozen and carries evidence, a failing run can log the exact probe numbers that produced it. That single property removes most of the "why did it go remote last Tuesday" archaeology from incident reviews.

Step 1 — Probe the remote seat before trusting it

The probe measures the transport path to the server seat, not model quality. Point it at whatever health endpoint the deployment exposes; the socket version below is the minimum viable substitute when no HTTP endpoint is published.

# lease.py (continued) — transport probe, five samples by default
def probe(samples: int = 5, timeout: float = 2.0) -> dict:
    host = os.environ["SEAT_HOST"]
    port = int(os.environ.get("SEAT_PORT", "443"))
    latencies: list[float] = []
    for _ in range(samples):
        t0 = time.perf_counter()
        try:
            with socket.create_connection((host, port), timeout=timeout):
                latencies.append((time.perf_counter() - t0) * 1000)
        except OSError:
            latencies.append(float("inf"))
    ok = [x for x in latencies if x != float("inf")]
    return {
        "samples": samples,
        "p50_ms": round(statistics.median(ok), 2) if ok else None,
        "p95_ms": round(max(ok), 2) if ok else None,
        "loss": round(1 - len(ok) / samples, 2),
        "at": round(time.time()),
    }

Illustrative output on a degraded link looks like {"samples": 5, "p50_ms": 88.1, "p95_ms": 1940.6, "loss": 0.2, "at": 1792471200}. The gap between p50 and p95 matters more than either number alone: a median of 88 ms with a 1.9 s tail is a link that will pass a smoke test and ruin a long run.

Step 2 — Turn constraints into invariants

Two constraints should outrank every measurement. If the payload contains secrets, or if the run must complete with no network, the lease goes local with a long expiry and no probe at all, because no amount of good latency changes the requirement.

SECRET_PATTERNS = [re.compile(p) for p in (
    r"sk-[A-Za-z0-9]{16,}",
    r"-----BEGIN [A-Z ]*PRIVATE KEY-----",
    r"(?i)aws_secret_access_key\s*=\s*\S+",
)]

def looks_secret_bearing(payload: str) -> bool:
    return any(p.search(payload) for p in SECRET_PATTERNS)

This pattern list is a coarse gate and nothing more. A dedicated redaction gate for the payload itself belongs upstream; the check here exists to catch the case where a secret appears mid-run, in turn seven, long after placement was settled.

Step 3 — Derive the expiry from measured latency

The expiry is the interesting part, because it determines how quickly a bad decision gets reconsidered. Short leases re-probe often and cost little; long leases save probe overhead and risk staleness. The table below is the rule set this module implements.

Condition observed	Placement	Lease TTL	Evidence recorded
Payload secret-bearing	local	3600 s	constraint flag, no probe
Run must survive offline	local	3600 s	constraint flag
Probe loss at or above 34%	local	90 s	samples, loss, timestamp
p95 above per-turn budget	local	120 s	p95, budget
p95 at or below half the budget, zero loss	remote	300 s	p95, loss
Everything else	remote	120 s	p95, loss

# lease.py (continued) — probe result becomes a lease
def decide(c: Constraints, now: float | None = None) -> Lease:
    now = time.time() if now is None else now
    if c.secret_free_local or c.offline_capable:
        return Lease(Placement.LOCAL, now + 3600,
                     {"reason": "invariant", "probe": None})
    p = probe()
    if p["p95_ms"] is None or p["loss"] >= 0.34:
        return Lease(Placement.LOCAL, now + 90, {"reason": "loss", **p})
    if p["p95_ms"] > c.p95_budget_ms:
        return Lease(Placement.LOCAL, now + 120, {"reason": "latency", **p})
    ttl = 300 if p["p95_ms"] * 2 <= c.p95_budget_ms else 120
    return Lease(Placement.REMOTE, now + ttl, {"reason": "measured", **p})

Note that a healthy link does not earn an unlimited lease. It earns five minutes, which is roughly the interval at which a moving laptop or a shared uplink can change its mind.

Step 4 — Check the lease at every turn boundary

Enforcement belongs at the boundary between turns, never inside a tool call. Migrating mid-call produces the duplicated-side-effect bug that takes a week to diagnose; letting the call finish locally or remotely and re-deciding afterwards does not.

# lease.py (continued) — per-turn enforcement
def next_lease(current: Lease | None, c: Constraints) -> Lease:
    if current is not None and current.valid():
        return current
    return decide(c)

def run_turn(state, payload: str, c: Constraints) -> str:
    lease = next_lease(state.lease, c)
    if lease.placement is Placement.REMOTE and looks_secret_bearing(payload):
        lease = Lease(Placement.LOCAL, time.time() + 3600,
                      {"reason": "secret_in_turn", "probe": None})
    state.lease = lease
    if lease.placement is Placement.REMOTE:
        try:
            return state.remote(payload)
        except RemoteUnavailable as exc:
            state.lease = Lease(Placement.LOCAL, time.time() + 90,
                                {"reason": type(exc).__name__, "probe": None})
    return state.local(payload)

The state object is deliberately unspecified here, because the interesting contract is the lease, not the client library. Any existing local-first runner can adopt the four functions above behind its current dispatch point.

Step 5 — Revoke, do not restart

Three signals should revoke a lease immediately rather than wait for expiry. A secret appearing in the current turn revokes to local for an hour, because the constraint is structural. A transport exception revokes to local for ninety seconds, giving the link time to recover before the next probe. An explicit rate-limit response from the server revokes for at least the advertised backoff window, and the run stops rather than probing around the limit.

Clock handling deserves its own line. After a host suspend, wall-clock time can jump while monotonic time does not, so a lease that looks valid may be hours stale. Comparing time.monotonic() deltas against the lease TTL, and treating a mismatch as expiry, catches the resume case that a pure time.time() comparison hides.

A fault matrix worth running before you trust it

Fault injected	Expected behaviour	Assertion
Link dropped mid-run	renewal probe fails, placement flips to local	no remote call after drop
Lease expires during a tool call	call completes, re-decision happens after	exactly one execution per turn
Secret appears in turn 7	immediate revoke to local, one hour TTL	remote client untouched after turn 7
Host suspend and resume	wall-clock jump treated as expiry	fresh probe on next turn
Server returns a rate-limit response	revoke and stop, no retry loop	request count stays flat

The decision function itself is testable in milliseconds by replacing the probe, which keeps the suite fast enough to run on every commit.

import pytest, lease

@pytest.mark.parametrize("constraints,probe_result,expected", [
    (lease.Constraints(True, False, 800), None, lease.Placement.LOCAL),
    (lease.Constraints(False, True, 800), None, lease.Placement.LOCAL),
    (lease.Constraints(False, False, 800),
     {"p95_ms": None, "loss": 0.6}, lease.Placement.LOCAL),
    (lease.Constraints(False, False, 200),
     {"p95_ms": 640.0, "loss": 0.0}, lease.Placement.LOCAL),
    (lease.Constraints(False, False, 900),
     {"p95_ms": 120.0, "loss": 0.0}, lease.Placement.REMOTE),
])
def test_decide(constraints, probe_result, expected, monkeypatch):
    if probe_result is not None:
        monkeypatch.setattr(lease, "probe", lambda samples=5: probe_result)
    assert lease.decide(constraints).placement is expected

The module and tests above were written for this article and exercised only against a stubbed probe; treat the probe target as the part you must adapt to your own deployment.

Where the free tier actually fits in the lease

The lease does not care which remote seat it points at, only whether that seat is reachable inside the budget. That is precisely why a free server option is useful as the remote side of a first lease: it makes the probe step cheap to run on real traffic, and the remote branch of the decision table can be exercised without a procurement conversation.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

MonkeyCode is described by its maintainers as an open source project, and the operator states that it offers free model access together with a free server option, with a ten-million-token allowance. Both are availability claims that can change without notice, so verify the current terms in the project's own materials before sizing a batch run against them; this article makes no claim about model names, hardware, throughput, or how long the offer lasts.

Placed inside the lease, the free server is the default remote branch for turns that carry no secrets and tolerate a measured p95. The free models are the reason the remote branch is worth probing at all for routine summarisation, classification, and drafting work, where a two-second tail is annoying rather than disqualifying.

Limitations and who should skip this

The probe measures transport, not output quality, so a lease can be perfectly healthy while the remote model produces worse results than the local one. Teams with contractual data-residency requirements need attested infrastructure, and a lease record is not an attestation. Anyone whose per-turn budget sits under about 50 ms on a LAN will find the lease flipping to remote and back for no operational benefit.

Single-host runners get the most from this pattern. Multi-tenant systems need a lease per tenant and per credential scope, which is a larger design than four functions. If the run is a one-shot script under thirty seconds, a static flag plus a timeout is genuinely the better engineering choice.

A reasonable next step is to implement only probe() and log its output for a week before wiring the decision in; the numbers will tell you whether your workload has the latency variance this lease exists to handle.

A Local Token Ledger for Free-Tier Agents: Surviving Crashes, Retries, and Drift

Emery Li — Mon, 19 Oct 2026 08:16:10 +0000

The last week's DEV front page leaned hard toward browser-side AI demos and long, open-ended discussions about AI fatigue, and both threads circle the same practical worry. Teams can now call a model from almost anywhere, but almost nobody can answer a simple operational question: which job spent the allowance, and did that job actually finish. The failure is rarely dramatic. It shows up as a nightly batch that dies halfway through with the pool already empty, while the dashboard shows only a single aggregate number that resets on someone else's schedule.

The scenario below is a composite of that failure mode rather than a report on a named customer. A small team runs a nightly agent job against a free model tier, retries twice on timeouts, and discovers on Monday that the pool is exhausted. Nobody can name the culprit, because usage lives only in a remote console, and the local logs stop at the point where the request was dispatched. The lesson is not that free tiers are unreliable. The lesson is that token accounting was treated as a vendor concern when it is really an application concern.

Why the accounting layer belongs on the local machine

The prompt is the part of an agent run that carries the most risk, because it often contains internal identifiers, file fragments, or customer text. An accounting layer positioned remotely sees that payload by construction, which quietly turns a billing helper into a data processor. A local ledger sees only counts, keys, and timestamps, so the sensitive material never leaves the machine that already holds it.

Local placement also survives the failure modes that matter here. A ledger written to a local SQLite file keeps working when the network drops, when the free server is slow to respond, and when the machine reboots mid-run. The heavy generation step can still be delegated, and often should be, but the record of what was spent must not depend on the same connection that spent it.

Step 1 — Fix the accounting unit before writing any code

Ambiguity about what counts as a token is the root cause of most reconciliation arguments. The ledger in this article treats billable tokens as provider-reported input tokens plus provider-reported output tokens, and it records retries as separate reservations with their own idempotency keys. Estimates are allowed at reservation time, but only reported numbers are allowed to close a run.

Two rules follow from that definition. First, a failed request that still consumed tokens must be committed, not silently released, because providers may bill for truncated streams. Second, a released reservation means the provider was never called or explicitly reported no usage, which is a much stricter condition than a client-side timeout.

Step 2 — Model reservations instead of running totals

A single spent counter cannot answer the question that matters during a crash: was this request paid for? Reservations solve that by giving every attempt a row with an explicit phase, so an interrupted process leaves a reserved row that is visibly unaccounted for rather than an invisible gap. The schema stays small enough to inspect by hand during an incident.

# ledger.py -- Python 3.10+, standard library only
import sqlite3
import time
import uuid

SCHEMA = """
CREATE TABLE IF NOT EXISTS reservations (
  idem_key        TEXT PRIMARY KEY,
  run_id          TEXT NOT NULL,
  phase           TEXT NOT NULL CHECK (phase IN ('reserved','committed','released')),
  reserved_tokens INTEGER NOT NULL,
  reported_tokens INTEGER,
  created_at      REAL NOT NULL,
  updated_at      REAL NOT NULL
);
CREATE INDEX IF NOT EXISTS idx_res_phase ON reservations(phase);
"""


class BudgetExceeded(RuntimeError):
    pass


class TokenLedger:
    def __init__(self, path: str, budget_tokens: int, floor_ratio: float = 0.15):
        self.db = sqlite3.connect(path, isolation_level=None, timeout=5.0)
        self.db.execute("PRAGMA journal_mode=WAL")
        self.db.executescript(SCHEMA)
        self.budget = budget_tokens
        self.floor_ratio = floor_ratio

    def _spent(self) -> int:
        row = self.db.execute(
            "SELECT COALESCE(SUM(COALESCE(reported_tokens, reserved_tokens)), 0) "
            "FROM reservations WHERE phase IN ('reserved','committed')"
        ).fetchone()
        return int(row[0])

    def remaining(self) -> int:
        return self.budget - self._spent()

    def reserve(self, idem_key: str, estimate_tokens: int) -> str:
        run_id = str(uuid.uuid4())
        now = time.time()
        floor = int(self.budget * self.floor_ratio)
        self.db.execute("BEGIN IMMEDIATE")
        try:
            if self.db.execute(
                "SELECT 1 FROM reservations WHERE idem_key = ?", (idem_key,)
            ).fetchone():
                self.db.execute("ROLLBACK")
                return run_id  # replayed call, nothing new reserved
            if self.remaining() - estimate_tokens < floor:
                raise BudgetExceeded(
                    f"remaining={self.remaining()} need={estimate_tokens} floor={floor}"
                )
            self.db.execute(
                "INSERT INTO reservations "
                "(idem_key, run_id, phase, reserved_tokens, created_at, updated_at) "
                "VALUES (?, ?, 'reserved', ?, ?, ?)",
                (idem_key, run_id, estimate_tokens, now, now),
            )
            self.db.execute("COMMIT")
            return run_id
        except Exception:
            self.db.execute("ROLLBACK")
            raise

Step 3 — Close runs with reported usage, not hopes

The commit path is deliberately boring, because boring code is what gets called from an exception handler at two in the morning. It moves a row from reserved to committed exactly once, and the WHERE phase = 'reserved' clause makes double-commit a no-op rather than a double charge. The release path exists for the narrow case where the provider was never reached.

    def commit(self, idem_key: str, reported_tokens: int) -> bool:
        cur = self.db.execute(
            "UPDATE reservations SET phase = 'committed', reported_tokens = ?, updated_at = ? "
            "WHERE idem_key = ? AND phase = 'reserved'",
            (reported_tokens, time.time(), idem_key),
        )
        return cur.rowcount == 1

    def release(self, idem_key: str) -> bool:
        cur = self.db.execute(
            "UPDATE reservations SET phase = 'released', updated_at = ? "
            "WHERE idem_key = ? AND phase = 'reserved'",
            (time.time(), idem_key),
        )
        return cur.rowcount == 1

The calling job then reads almost like a transaction script. The estimate is a guard rail, the commit uses the provider's own numbers, and an unexpected exception releases the hold so the pool does not leak capacity for a request that never happened.

ledger = TokenLedger("tokens.db", budget_tokens=10_000_000)
key = f"nightly-2026-10-19-{job_id}"

ledger.reserve(key, estimate_tokens=48_000)
try:
    result = client.generate(prompt)          # free model access, remote call
    ledger.commit(key, result.usage.input + result.usage.output)
except Exception:
    ledger.release(key)
    raise

Step 4 — Reconcile drift and reap abandoned reservations

Two housekeeping queries turn the ledger from a log into a control. The drift check compares reserves against reported usage and lists runs whose error exceeds a tolerance, which reveals tokenizer mismatches and prompt bloat early. The reaper closes reservations left behind by killed processes, using an age threshold long enough to avoid racing a slow but live request.

    def drift(self, tolerance: float = 0.05):
        rows = self.db.execute(
            "SELECT idem_key, reserved_tokens, reported_tokens FROM reservations "
            "WHERE phase = 'committed' AND reported_tokens IS NOT NULL"
        ).fetchall()
        return [
            (k, res, rep)
            for k, res, rep in rows
            if res and abs(rep - res) / res > tolerance
        ]

    def reap(self, older_than_s: float = 3600) -> int:
        cur = self.db.execute(
            "UPDATE reservations SET phase = 'released', updated_at = ? "
            "WHERE phase = 'reserved' AND created_at < ?",
            (time.time(), time.time() - older_than_s),
        )
        return cur.rowcount

Step 5 — Gate dispatch on the ledger, not on the console

The gate is one comparison, and it belongs before the request is built rather than after. When remaining() drops below a configured floor, the job should degrade to a cheaper path, shrink its context, or stop cleanly with a message that names the ledger file. That behaviour is what prevents the Monday-morning surprise, because the decision is made locally and instantly.

A test plan that fails in the useful direction

Scenario	Injection	Expected result
Crash after reserve	`kill -9` between reserve and commit	Row stays `reserved`; reaper releases it after the age threshold
Duplicate dispatch	Call `reserve` twice with the same key	Second call returns the first `run_id`, no new row
Double commit	Call `commit` twice	Second call returns `False`, totals unchanged
Under-reported usage	Commit fewer tokens than reserved	`drift` flags the run, `remaining` rises by the difference
Over-reported usage	Commit more tokens than reserved	Gate trips earlier on the next run, no negative spend
Parallel workers	Ten processes, one database file	`BEGIN IMMEDIATE` serialises writes; no lost updates
Read-only filesystem	Mount the directory read-only	Initialisation fails loudly before any request is sent

Run the crash and duplicate cases first, since they are the two that produce silent budget loss in practice. A short shell loop that kills the worker at random offsets covers the first case cheaply, and asserting remaining() afterwards catches leaks without instrumenting the provider.

Where the free server earns its seat

Generation itself is the step worth delegating, because a weak or sleeping local host should not decide whether a batch finishes. In that split, the prompt is sent to a hosted endpoint while the ledger stays home, and the local machine only needs to survive long enough to record the result. MonkeyCode's free model access and free server option fit this shape: the remote side does the inference and the local side keeps the books. The brief for this piece also states a ten-million-token free allowance, and that figure is operator-supplied like the rest; check the current project terms before planning capacity around it.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Concern	Keep local	Delegate to free server
Secrets and prompt text	Yes, or redact before send	No
Token ledger and gates	Yes	No
Inference on a sleeping laptop	Not reliable	Preferred
Offline replay of past runs	Yes, from the ledger	Not needed
Crash recovery	Local file, WAL mode	Remote session may expire

Limitations and who should skip this

The ledger counts what the provider reports, so any client sharing the same account or key can still spend outside its view. It also cannot enforce a hard cap upstream; provider-side budget controls remain the only real ceiling, and this design is a guard rail rather than a guarantee. SQLite handles a handful of concurrent workers comfortably but is not a multi-tenant billing system, and tokenizer differences make estimates approximate until reconciliation.

Teams running a single interactive session should skip all of this, because a script and a spreadsheet answer their question faster than a schema does. Teams that need invoices, chargeback, or per-customer attribution should use a real metering service instead of a SQLite file on a laptop.

A closing note on scope

The useful habit is smaller than a platform: record reservations locally, close them with reported numbers, and reconcile before the pool is empty. If this workflow is useful, the project is open source and the free tier is the cheapest way to test whether the gate fires at the right moment for your own jobs.

Fault-Injecting the Free Seat: An Offline Fallback Contract for Local-First Agents

Emery Li — Sun, 18 Oct 2026 06:59:08 +0000

The incident report was three lines long: a refactor turn had been running against a remote seat, the office Wi-Fi dropped for eleven seconds, and the turn returned nothing at all. The local model was still loaded, the repository was still on disk, and the user was still waiting on a spinner. Nothing in the codebase had ever defined what should happen next, so the answer was a stack trace and a lost turn.

That gap is not a networking bug, and retrying harder does not close it. It is a missing contract between two seats that can both serve the same request. The pattern below is a composite of common local-first agent setups rather than a single team's incident, but the failure shape repeats often enough to be worth writing down.

Why a fallback contract beats a retry loop

A retry loop answers one question: how many times do we try again. A fallback contract answers four: which failures are worth retrying, which should hand the turn to the local seat, how long the user waits in each case, and how the interface tells the user which seat actually answered. The last question is the one most local-first projects skip, and it is the one that erodes trust fastest.

When provenance is invisible, a degraded local answer looks identical to a healthy remote answer. Users then attribute the smaller model's weaker output to the product rather than to the network. A contract that labels provenance turns an invisible degradation into an explainable event, which is a far cheaper fix than making the local path match the remote one.

Step 1: Classify the failure before you choose a residency

The useful unit of design is not "online versus offline" but a small set of failure classes with different correct responses. The budgets in the table are illustrative starting points for a discussion, not measured values; each team should set them from its own latency data.

Failure class	Observable signal	Wrong response	Contract response
Hard partition	Connect error in under a second	Retry three times, then fail the turn	Local seat takes the turn immediately, output labeled `local`
Soft timeout	No first byte before the deadline	Wait indefinitely	Local takes over with the remaining budget; remote work continues
Error-free stall	Socket open, no tokens emitted	Spinner with no telemetry	Watchdog emits a `degraded` event and the UI shows provenance
Remote 5xx burst	Three failures inside the window	Stack trace to the user	Circuit opens for the window, local serves the queue

The key insight is that a timeout is not a cancellation. A request abandoned after the deadline may still occupy the remote seat and still consume whatever budget that seat has. That is why the contract needs an idempotency key stamped on the turn before the first attempt, so a later local answer can suppress a duplicate remote result instead of merging two divergent outputs.

Step 2: Encode the budget as data

Budgets that live in prose drift away from the code within a sprint. Put them in a frozen dataclass so tests can construct a tight budget without touching production defaults, and so the values appear in one reviewable place.

# fallback_contract.py
from __future__ import annotations

import time
import uuid
from dataclasses import dataclass, field


@dataclass(frozen=True)
class Budget:
    soft_deadline_s: float = 1.5    # local seat must answer within this
    hard_deadline_s: float = 6.0    # remote seat gets no longer than this
    circuit_window_s: float = 60.0
    failure_threshold: int = 3


@dataclass
class Turn:
    prompt: str
    idempotency_key: str = field(default_factory=lambda: uuid.uuid4().hex)
    provenance: str = "unset"
    degraded: bool = False


def run_turn(turn: Turn, *, local, remote, budget: Budget, clock=time.monotonic) -> str:
    start = clock()
    try:
        text = remote.send(
            turn.prompt,
            deadline_s=budget.hard_deadline_s,
            idempotency_key=turn.idempotency_key,
        )
        turn.provenance = "remote"
        return text
    except (ConnectionError, TimeoutError):
        turn.degraded = True

    elapsed = clock() - start
    remaining = max(0.0, budget.soft_deadline_s - elapsed)
    text = local.send(turn.prompt, deadline_s=remaining, idempotency_key=None)
    turn.provenance = "local"
    return text

The remaining-budget arithmetic matters more than it looks. If the remote attempt already burned two seconds, handing the local seat a fresh 1.5 seconds silently violates the user's patience limit. Clamping to the remainder keeps the total turn inside the advertised envelope.

Step 3: Inject the failures instead of waiting for them

Waiting for a real partition is not a test strategy. A handful of fake transports reproduces every failure class in milliseconds, and they run in CI without network access at all.

# fake_seats.py
import time


class HealthySeat:
    def send(self, prompt, *, deadline_s, idempotency_key):
        return "remote answer"


class PartitionedSeat:
    def send(self, prompt, *, deadline_s, idempotency_key):
        raise ConnectionRefusedError("no route to seat")


class SlowSeat:
    def send(self, prompt, *, deadline_s, idempotency_key):
        time.sleep(deadline_s + 0.5)
        raise TimeoutError("deadline exceeded")


class LocalSeat:
    def __init__(self) -> None:
        self.calls = 0

    def send(self, prompt, *, deadline_s, idempotency_key):
        self.calls += 1
        return "local answer"

Each fake returns the same shape as the real transport, so the production call site stays unchanged. That constraint is worth enforcing: the moment a fake needs a special branch in run_turn, the harness has stopped testing the code that ships.

Step 4: Assert on provenance, not just output

The most valuable assertions are the ones about what the user is told. A turn that quietly falls back and reports nothing is a failing test, even when the text it returns is perfectly reasonable.

# test_fallback_contract.py
import pytest
from fallback_contract import Budget, Turn, run_turn
from fake_seats import HealthySeat, LocalSeat, PartitionedSeat, SlowSeat


@pytest.mark.parametrize("seat", [PartitionedSeat(), SlowSeat()])
def test_local_takes_over_and_says_so(seat):
    local = LocalSeat()
    turn = Turn(prompt="refactor auth.py")
    out = run_turn(turn, local=local, remote=seat, budget=Budget())
    assert out == "local answer"
    assert turn.provenance == "local"
    assert turn.degraded is True
    assert local.calls == 1


def test_healthy_seat_keeps_provenance_remote():
    turn = Turn(prompt="summarise diff")
    out = run_turn(turn, local=LocalSeat(), remote=HealthySeat(), budget=Budget())
    assert out == "remote answer"
    assert turn.provenance == "remote"
    assert turn.degraded is False


def test_idempotency_key_is_stable_across_fallback():
    turn = Turn(prompt="migrate schema")
    key = turn.idempotency_key
    run_turn(turn, local=LocalSeat(), remote=PartitionedSeat(), budget=Budget())
    assert turn.idempotency_key == key

Run the suite with duration reporting so the injected latency is visible rather than hidden:

python -m pytest test_fallback_contract.py -q --durations=5

The slow-seat case dominates the runtime, which is itself the diagnostic. If that entry vanishes from the duration report, the fake is no longer sleeping and the timeout path has stopped being exercised.

Where a free server genuinely wins, and where it does not

Local-first is not a moral position, and three tradeoffs decide it. Latency comes first: a small local model has predictable time-to-first-token but may lose badly on long generations, while a remote seat has variable network overhead and much stronger throughput once it starts. Secrets come second: credentials and private source should stay on the machine, with only a reduced payload crossing the network, so the remote path inherits whatever boundary the local redaction step enforces. Offline capability comes third, and it is the only one a local seat wins unconditionally.

This is the narrow place where MonkeyCode's free model access and free server option are relevant to the workflow rather than bolted onto it. A free remote seat makes the contract test cheap to run against real transport behavior instead of only against fakes, which is exactly the observation the harness above cannot provide. Availability, model selection, and any usage limits should be confirmed in the product documentation at the time of reading, since they change. Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Limitations and who should skip this

The harness proves fallback behavior, not performance. Every number in the budget table is a placeholder chosen to make the tests readable, and no latency or cost comparison here should be quoted as a measurement. The synchronous example also cannot cancel in-flight remote work, so duplicate suppression via the idempotency key is mandatory rather than optional.

Teams with no local inference path should not adopt this pattern; they have one seat and the correct design is a queue with visible retry state. Teams under a strict data-residency rule should invert the contract entirely and treat the remote seat as unavailable by design, which makes the local path the only path and the provenance label redundant. Everyone else can start with two fake transports, one parametrized test, and a provenance field, then grow the failure table as real incidents arrive.

If you are mapping your own failure classes, running the same harness against a real free seat for an afternoon is a cheap way to replace guessed budgets with observed ones.

A Three-Way Residency Trial: Local-Only, Server-Only, and Split Fallback Agent Runs

Emery Li — Sat, 17 Oct 2026 06:01:05 +0000

Most residency arguments are settled with a single-arm benchmark, and that is why they keep getting re-litigated. A developer runs a task suite on a local model, then runs the same suite against a hosted free model, and compares two numbers that were never measuring the same thing. The local arm fails on truncation, the hosted arm fails on connect timeouts during a commute, and the comparison collapses into anecdote.

The scene that motivates a third arm

An illustrative composite: a small team keeps an agent on a laptop because the repository contains customer fixtures. Someone proposes moving the batch to a free server seat to escape fan noise and short tool loops. The move fixes throughput and breaks a review rule, so the work oscillates between two arms for weeks. Neither arm was ever tested with a fallback path, which is the configuration the team would actually ship.

MonkeyCode, an open-source coding agent project, is relevant here only because it exposes both a free model path and a free server option as operator-stated availability; the trial below measures whichever backends the reader supplies. Disclosure: This article was prepared as part of MonkeyCode's product outreach. Confirm current free-tier terms on the project's own pages, since quotas and availability change without notice.

Why two arms produce unstable conclusions

A two-arm comparison answers "which was faster," while a shipping decision needs "which fails, and how." Failure class matters more than median latency because transport errors are retryable, schema errors are cheap to validate, and wrong answers cost human review minutes that never appear in a latency chart. The third arm, split with fallback, exposes which of those classes the fallback actually absorbs.

Step 1: Fix a small task suite with gradeable outputs

Twelve to twenty tasks is enough to start, provided every task has a validator that returns pass or fail without human judgement. Include at least four task shapes: a pure edit, a multi-file refactor, a tool-loop task such as running tests, and a task requiring a file the process must read from disk. The suite should stay frozen for the duration of the trial, because a shifting suite converts the measurement into a moving target.

Step 2: Instrument the transport, not the model

The harness must not care whether the backend is a local runtime or a remote seat. Wrap both behind one callable that returns an artifact plus a status, then record every attempt as one JSONL line. Recording at the attempt level, rather than the task level, is what later makes retries visible.

# trial.py — runnable against any adapter implementing run(artifact_spec) -> (text, status)
import argparse, json, pathlib, time, uuid
from dataclasses import dataclass, asdict

FAILURE_CLASSES = ("none", "transport", "schema", "tool_loop", "truncation", "wrong_answer")

@dataclass
class Attempt:
    run_id: str
    arm: str          # local | server | split
    task_id: str
    attempt: int
    backend: str      # resolved backend for this attempt
    wall_ms: int
    status: str       # ok | error | timeout
    tool_calls: int
    egress_bytes: int # payload size handed to a remote backend, 0 for local
    failure_class: str

def classify(status, validator_error, tool_calls, egress_bytes):
    if status == "timeout":
        return "transport"
    if status == "error":
        return "tool_loop" if tool_calls else "transport"
    if validator_error is None:
        return "none"
    if validator_error.startswith("truncated"):
        return "truncation"
    if validator_error.startswith("schema"):
        return "schema"
    return "wrong_answer"

def run_arm(arm, tasks, adapters, out_path):
    run_id = uuid.uuid4().hex[:12]
    with open(out_path, "a", encoding="utf-8") as sink:
        for task in tasks:
            order = adapters.order_for(arm, task)   # local, server, or both
            for i, backend in enumerate(order):
                started = time.monotonic()
                text, status, tool_calls, egress = adapters.call(backend, task)
                wall_ms = int((time.monotonic() - started) * 1000)
                err = task.validate(text)
                rec = Attempt(run_id, arm, task.id, i, backend, wall_ms,
                              status, tool_calls, egress,
                              classify(status, err, tool_calls, egress))
                sink.write(json.dumps(asdict(rec)) + "\n")
                if rec.failure_class == "none" or arm != "split":
                    break

if __name__ == "__main__":
    p = argparse.ArgumentParser()
    p.add_argument("--arm", choices=("local", "server", "split"), required=True)
    p.add_argument("--suite", default="tasks/")
    p.add_argument("--out", default="runs.jsonl")
    a = p.parse_args()
    run_arm(a.arm, load_suite(a.suite), load_adapters(), a.out)

The ordering helper is the only arm-specific logic, and it stays deliberately boring: local-first for the local arm, server-first for the server arm, and local-then-server for split. Because the fallback in split is triggered by any non-pass validator result, the harness over-triggers on purpose; a production router would retry transport errors only.

Step 3: Run the three arms on the same machine

python trial.py --arm local  --out runs/local.jsonl
python trial.py --arm server --out runs/server.jsonl
python trial.py --arm split  --out runs/split.jsonl
python report.py runs/*.jsonl --markdown

Run all three from the same host and the same network conditions, and note the time of day next to each run. A server arm measured on office wifi and a local arm measured on tethered mobile data will produce a difference that reflects the network, not the residency.

Step 4: Read the failure distribution before the latency

The report should print, per arm, the share of each failure class and the median wall time for attempts that passed. The useful comparison is not local versus server; it is split versus the better of the two single arms. If split does not reduce the dominant failure class, the fallback is decorating the pipeline rather than protecting it.

Step 5: Convert the output into a routing table

Signal in the trial output	Default residency	Reason
Dominant class is `transport`	local	Every remote attempt pays a network it does not need
Dominant class is `schema`, local validator cheap	local	Retry loops stay off the wire and off the quota
Dominant class is `tool_loop` on local	server	Longer tool budgets usually live where the host does not sleep
`egress_bytes` above the review threshold	local or redacted	Residency follows the payload, not the benchmark
Offline requirement in the task spec	local	No fallback can recover an absent link
Long unattended batch, no secrets	server	Host sleep and lid close stop mattering

The table is a starting hypothesis, not a verdict, and it should be regenerated whenever the backend or the suite changes.

Limitations and who should skip this

A twelve-task suite supports a routing decision, not a statistical claim, and a single week of runs will not survive a model update. Teams whose artifacts cannot be graded automatically should not adopt the harness, because manual grading reintroduces the bias the trial was meant to remove. Regulated payloads need an egress review before any server arm is measured at all, and anyone expecting the free tier to remain static should treat the numbers as expiring on the day they are printed.

Closing note

Isolation and measurement beat enthusiasm in both directions: the local machine is not automatically safer, and the free server is not automatically faster. The project's repository documents its current free model access and free server option, and both are worth confirming against live terms before either is wired into a pipeline.

The Remote Break-Even: Measuring When a Free Server Seat Beats a Local Agent

Emery Li — Thu, 15 Oct 2026 11:14:35 +0000

The Remote Break-Even: Measuring When a Free Server Seat Beats a Local Agent

A refactor begins at 09:40 on a laptop that is already warm from a test run, and the first streamed token arrives about 1.2 seconds later. Six hundred tokens of patch follow at roughly 18 tokens per second, so the turn closes in about half a minute with nothing leaving the disk. Twenty minutes later the same loop is handed an 8 MB context bundle, and the same lane needs several minutes to answer. The interesting question is not whether local execution is virtuous, but whether a particular turn crosses a break-even point that can be measured before it runs.

Client-side inference keeps resurfacing as a topic, and current developer chatter about AI fatigue pushes the discussion toward ideology rather than arithmetic. The practical decision has three inputs: measured latency on the machine in front of you, the size of whatever must move across the network, and whether the payload is allowed to move at all. This article builds a small harness around those inputs, using a free remote seat as the second lane.

Note: the snippets below are illustrative and were not executed for this article. Every placeholder must be replaced with output from your own probe before any verdict is trusted.

1. Model each lane as a sum of additive costs

The local lane is short: prefill time plus output tokens multiplied by per-token decode time. The remote lane is longer and much easier to underestimate, because it contains four terms that never appear in a vendor's headline throughput number. Those terms are round-trip and queue delay, upload time for the prompt payload, prompt prefill on the remote side, and download time for the response.

A fifth term matters more than the other four in agent workloads, and it is the handoff tax. Handoff includes serializing the current state, re-authenticating, rebuilding session context, and re-establishing whatever tool handles the loop had open. Teams routinely measure remote decode speed and then quietly pay a multi-second tax on every turn, which is why the arithmetic below treats handoff as a first-class cost rather than an implementation detail.

2. Measure the local lane before comparing anything

Run the probe on the machine that would actually host the local lane, because thermal state and memory pressure change these numbers by a wide margin. Take the median across at least five trials and keep the tail, since a single 4-second prefill spikes changes the verdict for short turns.

# probe_local.py — measure prefill and decode on the hosting machine.
import statistics, time

def measure_local(call_local, prompt_tokens: int, out_tokens: int, trials: int = 5):
    """call_local(prompt_tokens, out_tokens) returns an iterator of token chunks."""
    prefill_ms, per_token_ms = [], []
    for _ in range(trials):
        t0 = time.perf_counter()
        stream = call_local(prompt_tokens, out_tokens)
        first = next(stream)                 # time to first token == prefill cost
        t1 = time.perf_counter()
        rest = list(stream)
        t2 = time.perf_counter()
        prefill_ms.append((t1 - t0) * 1000)
        per_token_ms.append(((t2 - t1) / max(1, len(rest))) * 1000)
    return {
        "prefill_s": statistics.median(prefill_ms) / 1000,
        "per_token_s": statistics.median(per_token_ms) / 1000,
        "prefill_p95_s": sorted(prefill_ms)[int(trials * 0.95) - 1] / 1000,
    }

Record the numbers as a local profile file and commit it next to the harness. A profile is only valid for one machine class and one prompt shape, and treating it as portable between laptops is how the break-even calculation quietly becomes fiction. Re-measure whenever the runtime, quantization, or available RAM changes.

3. Compute the break-even with arithmetic, not intuition

With one local profile and one remote profile, the lane decision becomes subtraction. The calculator below takes placeholder values so the shape of the result is visible; substitute probe output and real network measurements before use.

# breakeven.py — decide the lane for one turn. Pure arithmetic, no network calls.
from dataclasses import dataclass

@dataclass
class LocalProfile:
    prefill_s: float          # from measure_local()
    per_token_s: float        # seconds per generated token

@dataclass
class RemoteProfile:
    rtt_s: float              # round trip plus queueing
    prefill_tokens_per_s: float
    decode_tokens_per_s: float
    uplink_bytes_per_s: float
    downlink_bytes_per_s: float
    handoff_s: float          # serialize, re-auth, rebuild session

def estimate_local(p: LocalProfile, out_tokens: int) -> float:
    return p.prefill_s + out_tokens * p.per_token_s

def estimate_remote(r: RemoteProfile, prompt_tokens: int, out_tokens: int,
                    upload_bytes: int, download_bytes: int) -> float:
    return (r.rtt_s
            + upload_bytes / r.uplink_bytes_per_s
            + prompt_tokens / r.prefill_tokens_per_s
            + out_tokens / r.decode_tokens_per_s
            + download_bytes / r.downlink_bytes_per_s
            + r.handoff_s)

def verdict(local_s: float, remote_s: float, margin: float = 0.15) -> str:
    if remote_s < local_s * (1 - margin):
        return "remote"       # clear win, worth paying the handoff
    if local_s < remote_s * (1 - margin):
        return "local"
    return "either"           # inside the noise band; prefer local for secrets

Three scenarios show how sharply the answer moves when payload size changes. All numbers below are placeholders, not benchmarks: local at 1.2 s prefill and 18 tokens/second, remote at 55 decode tokens/second with 3.3 s of prefill for a 3,000-token prompt.

Scenario	Local estimate	Remote estimate	Verdict
600-token patch, 24 KB upload, 2 MB/s up	34.5 s	15.8 s	remote
Same turn, 8 MB context, 1 Mbit/s up	34.5 s	82.9 s	local
Same turn, 20 s handoff instead of 1.5 s	34.5 s	34.3 s	either

The middle row is the one that surprises people, because the upload of 8 MB at 1 Mbit/s costs about 67 seconds by itself. The third row shows why handoff deserves its own term: a 20-second state rebuild erases a 19-second remote advantage almost exactly. When a verdict lands in the noise band, the tie-break should come from data sensitivity rather than from a coin flip.

4. Gate secrets before the verdict gets a vote

A latency verdict must never be able to move secret-bearing material across a network boundary. The check below runs first, and it is deliberately blunt: any field classified as secret pins the turn to the local lane regardless of what the arithmetic says.

# lane_gate.py — policy check that runs before any remote call is attempted.
RANK = {"public": 0, "internal": 1, "secret": 2}

def allowed_lane(fields, connectivity: bool, verdict_str: str) -> str:
    worst = max(RANK[f["class"]] for f in fields)
    if worst >= RANK["secret"]:
        return "local"          # no remote seat, whatever the latency math says
    if not connectivity:
        return "local"          # offline is a lane, not a failure
    return verdict_str

assert allowed_lane([{"class": "secret"}], True, "remote") == "local"
assert allowed_lane([{"class": "public"}], False, "remote") == "local"
assert allowed_lane([{"class": "internal"}], True, "remote") == "remote"

Classification is the hard part, and it is a review problem rather than a code problem. Marking a field as internal when it contains a customer identifier turns the gate into decoration, so the field list should be reviewed whenever a new tool is added to the loop. Redaction can downgrade a payload, but redaction has to happen before the gate reads the classes, not after the request is built.

5. Turn the rule into a decision table

A table keeps the policy reviewable by people who will never read the Python. The rows below are the ones that came up most often when auditing a mixed local and remote agent loop.

Task class	Data class	Connectivity	Lane
Short edit, secret-adjacent	secret	any	local only
600-token refactor	internal	uplink over 2 MB/s	remote
8 MB context bundle	internal	uplink under 1 Mbit/s	local
Bulk generation	public	any	remote while allowance lasts
Anything	any	offline	local

The rule of thumb that falls out of the table: remote wins on small payloads and long outputs, and local wins on large payloads and short outputs. Agent turns tend toward large payloads and short outputs, which is why a local-first default survives scrutiny more often than it fails it.

6. Where a free remote seat actually earns its place

A remote lane is most useful before you own any remote lane, because that is when the probe has nothing to measure against. MonkeyCode, an open-source coding agent project, is described by its operator as offering free model access with a 10 million token allowance and a free server option. Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Treat both statements as operator-stated availability claims rather than benchmarks, and confirm the current terms on the project's own page before wiring anything automated to them. Under the placeholder arithmetic above, a turn consuming roughly 3,600 tokens works out to about 2,700 turns across a 10,000,000-token allowance, which is enough runway to run the probe and observe a real break-even rather than an assumed one. Free model access also makes the sensitivity gate cheap to test, since pinning a turn to local costs nothing but wall-clock time.

Two limits are worth stating plainly. The arithmetic in this article does not compare output quality, and a lane that returns a malformed schema is not faster at any latency. Quota and availability are variables in someone else's roadmap, so a pipeline that fails hard when the allowance changes has a dependency problem, not a capacity problem.

7. Limitations and who should skip this

Do not use this approach if a data-residency obligation forbids remote inference for every class you handle, because the gate will pin everything to local and the calculator becomes dead code.
Do not use it if your workload is dominated by large uploads, since the decision is made by the uplink term before any model quality enters the picture.
Do not use it if you cannot measure on the actual host, because placeholder figures produce confident and wrong verdicts.
Do not let a CI pipeline depend on a free allowance without a local fallback path, since an unavailable remote lane should degrade rather than fail.

Latency figures vary by machine, network, and time of day, and the handoff tax varies by how much state your loop carries. The method survives that variance because it measures locally and recomputes; the specific numbers in this article do not transfer to your laptop.

8. Log every decision so it can be audited later

A verdict that is never recorded cannot be reviewed when behavior changes. Append one JSON line per turn with the measured inputs, the chosen lane, and the gate result, then sample those lines weekly to see whether the break-even has drifted. The log is also the fastest way to discover that a previously internal field has quietly become secret-bearing.

{"turn": 412, "out_tokens": 600, "upload_bytes": 24000, "local_est_s": 34.5,
 "remote_est_s": 15.8, "worst_class": "internal", "lane": "remote"}

The instrument is the useful artifact here, not any single verdict. A laptop answers short turns offline and hands large bundles to a free seat when the upload cost is small, and the only way to know which case you are in is to measure it.

A Redaction Gate for Agent Turns That Outlive the Tunnel

Emery Li — Tue, 13 Oct 2026 00:23:20 +0000

A coding agent on a commuter laptop had already opened three files when the train entered a tunnel and the link dropped. The working tree still held a half-written migration, a failing test log, and a snippet copied from a local environment file. The next turn could stay on the laptop, wait for the signal to return, or borrow a remote seat after the tunnel. That choice is a placement problem rather than a branding choice, and it should be scored before any bytes leave the machine.

Recent public threads circle model quality, yet local-first work still fails first on secrets, reachability, and where the files already live. A remote seat can still win when the laptop is thermally busy or the local model is still cold from a fresh boot. The same remote seat can also help when the remaining work is a long batch that contains no private spans at all. The gate in this article does not select a vendor, and it only records why a turn stayed home or traveled after redaction.

What the gate measures

Four pressures decide the seat, and each pressure can veto a remote hop without waiting for the other three. Latency matters when a developer is waiting on the next edit, because a round trip can cost more than a small local completion. Secrets matter more than latency, because a leaked key remains useful long after the tunnel ends and the commute finishes. Offline state matters when the interface is down, since a remote call that cannot connect should remain a queued turn rather than a failed job.

A free remote seat wins only after those three checks pass and the remaining work is large enough to justify leaving the laptop. The pressures stay independent, so a fast network does not excuse a secret and a free seat does not excuse a missing redaction. Teams that collapse the four pressures into one vague cloud switch will ship private spans on busy afternoons. The checklist below keeps each pressure visible in the run log instead of hiding it inside a single boolean.

Five steps before any hop

The workflow assumes a local queue already stores each agent turn as plain text plus a small metadata record beside it. No step requires a particular model name, a published quota, or a specific hardware shape on either side of the hop. Operators should read the current product documentation before they treat any free seat as available, because availability language changes. A large free-token figure should stay out of this runbook until current documentation confirms both the number and the expiry.

Capture the turn on disk before any network call is attempted, including the user text, the tool excerpts, and a timestamp. Write that record under a local directory that is already gitignored, and refuse to continue if the write fails for any reason. A turn that cannot be stored locally should not be exported later under commute pressure or a flaky reconnect.

mkdir -p .agent-queue
git check-ignore -q .agent-queue || echo 'add .agent-queue/ to .gitignore before continuing'
printf '%s\n' "$TURN_TEXT" > .agent-queue/turn.txt
test -s .agent-queue/turn.txt && echo 'local capture ok'

Scan the stored text for secret-shaped spans, including environment assignments, private-key headers, and long token-like strings. Mark each hit with a byte offset and a label, then replace the span with a stable placeholder before any later copy. Keep the original record on the laptop, and never send the unredacted file as a convenience attachment for debugging.
Probe connectivity with a short timeout, and record only reachable or unreachable together with the measured delay. Do not treat a successful probe as permission to upload the repository, the shell history, or the full conversation transcript. A probe that fails means the turn stays in the local queue until a later pass finds the interface up again.

python - <<'PY'
import os, socket, time
host = os.environ.get('SEAT_PROBE_HOST', 'example.com')
port = int(os.environ.get('SEAT_PROBE_PORT', '443'))
start = time.perf_counter()
try:
    socket.create_connection((host, port), timeout=1.5).close()
    ok, err = True, ''
except OSError as exc:
    ok, err = False, type(exc).__name__
elapsed = int((time.perf_counter() - start) * 1000)
print('probe_ok=%s probe_ms=%s err=%s' % (ok, elapsed, err))
PY

Score the seat with the function in the next section, using secret hits, the probe result, payload size, and whether the developer is waiting. Stay local when any secret remains, when the probe failed, or when the payload is only a short interactive edit. Allow a remote seat only when the payload is clean, the probe succeeded, and the job is a batch rather than a live edit.
Log the decision, the redaction count, and the seat name beside the turn identifier before any client is invoked. Review that log after the commute, and delete remote copies that are no longer required for the batch that just finished. A missing log line is a failed run, even when the model returned a patch that appears to compile on the first try.

An example log line, not a captured production event, can use a single fixed field order for every turn. The fields are a timestamp, a turn identifier, a seat name, a redaction count, and the probe pair. A reviewer should be able to rebuild the table row from that line without opening the original prompt. If the seat is remote_allowed, the same line should also name the redacted file that was actually sent.

2026-10-13T08:14:02Z turn=8f3a seat=local_only redactions=1 probe_ok=true probe_ms=42 file=-

Decision table the score must honor

The table below is the contract the scoring function must honor, and a later code change should not soften a row without a new test. A free server can help a long batch, yet that same server is the wrong seat while a turn still contains a key. Cost and novelty do not appear in the table, because neither one repairs a leak or restores a missing network. When several rows match, the most restrictive seat wins, and a secret hit outranks both the queue and the remote seat.

Condition	Seat	Reason
Probe failed or interface down	local_queue	Offline work cannot borrow a remote seat
Any secret hit remains	local_only	Private material does not travel to buy speed
Developer waiting and payload is small	local_model	Round-trip delay dominates a short edit
Interactive probe slower than 800 ms	local_model	A sluggish hop is worse than a local completion
Clean payload, network up, long batch	remote_allowed	Laptop stays responsive while the batch runs
Scanner crashed or output missing	local_only	Unknown secret state counts as a hit

Runnable gate and checks

The module below is a proposal for the gate, not a record of a measured production deployment or a vendor benchmark. It uses only the Python standard library so a reader can run it on a laptop without installing a vendor SDK. The thresholds are examples, and each team should retune them after measuring its own tunnels, editors, and local model startup. Nothing in the sample invents a model name or a token quota, because those facts belong in current product documentation.

from dataclasses import dataclass
import re

SECRET_PATTERNS = (
    re.compile(r'(?i)\b(api[_-]?key|secret|token|password)\b\s*[:=]\s*\S+'),
    re.compile(r'-----BEGIN [A-Z ]*PRIVATE KEY-----'),
    re.compile(r'\bsk-[A-Za-z0-9]{20,}\b'),
)

@dataclass(frozen=True)
class TurnFacts:
    text: str
    probe_ok: bool
    probe_ms: int
    interactive: bool
    batch_chars: int = 4000

def secret_hits(text: str) -> int:
    return sum(len(pattern.findall(text)) for pattern in SECRET_PATTERNS)

def redact(text: str) -> str:
    cleaned = text
    for pattern in SECRET_PATTERNS:
        cleaned = pattern.sub('[REDACTED]', cleaned)
    return cleaned

def choose_seat(facts: TurnFacts) -> str:
    if secret_hits(facts.text):
        return 'local_only'
    if not facts.probe_ok:
        return 'local_queue'
    if facts.interactive and len(facts.text) < facts.batch_chars:
        return 'local_model'
    if facts.interactive and facts.probe_ms > 800:
        return 'local_model'
    return 'remote_allowed'

def seat_or_hold(facts: TurnFacts) -> str:
    # Proposal only: production code should catch a narrow scanner error.
    try:
        return choose_seat(facts)
    except Exception:
        return 'local_only'

A companion check can live beside the module and run under an ordinary test runner on the same laptop. The cases lock the veto order so a later edit cannot quietly prefer the remote seat when a secret is present. Offline turns stay queued, short interactive edits stay local, and only a clean batch is allowed to leave. Those expectations are the acceptance bar for this artifact, and they do not require a network connection to evaluate.

from span_gate import TurnFacts, choose_seat, redact, seat_or_hold

def test_secret_vetoes_remote_seat():
    facts = TurnFacts(
        'token = sk-live-abcdefghijklmnopqrstuvwxyz',
        True,
        40,
        False,
    )
    assert choose_seat(facts) == 'local_only'
    assert 'sk-live' not in redact(facts.text)

def test_offline_stays_queued():
    facts = TurnFacts('refactor the parser', False, 0, False)
    assert choose_seat(facts) == 'local_queue'

def test_short_interactive_stays_local():
    facts = TurnFacts('rename this helper', True, 30, True)
    assert choose_seat(facts) == 'local_model'

def test_clean_batch_may_leave():
    facts = TurnFacts('x' * 5000, True, 120, False)
    assert choose_seat(facts) == 'remote_allowed'

def test_scanner_failure_fails_closed(monkeypatch):
    def boom(_text):
        raise RuntimeError('scanner down')
    monkeypatch.setattr('span_gate.secret_hits', boom)
    facts = TurnFacts('plain batch text ' * 400, True, 30, False)
    assert seat_or_hold(facts) == 'local_only'

Save the module as span_gate.py and the checks as test_span_gate.py, then run both commands from the repository root. The test command should fail the build if a secret-bearing turn can still select the remote_allowed seat. The second command should print local_only and a redacted line where the password value has been replaced by a placeholder. If either result differs, the gate is not ready for a reconnect, regardless of how idle the remote seat looks.

python -m pytest test_span_gate.py -q
python - <<'PY'
from span_gate import TurnFacts, choose_seat, redact
sample = 'password = hunter2\nplease summarize the diff'
print(choose_seat(TurnFacts(sample, True, 50, False)))
print(redact(sample))
PY

Wiring a free remote seat

Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode is relevant here only as one operator-supplied option that currently describes free model access and a free server seat. This article does not treat a token allotment, a model name, a hardware shape, or a duration as a verified current fact. Readers who need those details should check the project documentation on the day they run the gate, then record the terms they observed.

If that free server is the chosen remote seat, the same function still applies, and the product name does not change the veto order. A clean batch may use the free seat so the laptop fan stays quieter and the editor remains responsive during the run. A turn with a key, a failed probe, or a short interactive edit should not move, even when the remote option is free. Free capacity is a scheduling convenience, not a reason to widen the set of bytes that are allowed to leave the machine.

Operators can wire the seat name into a client they already trust, provided the client receives the redacted text rather than the original file. A practical check is to diff the outbound body against the redacted string before the request is signed or sent. If that diff is not empty, the client should abort and leave the turn marked local_only until a person reviews the span. That abort path matters more than the brand of the seat, because the leak happens at the moment of copy.

Limits and poor fits

The patterns catch common assignment shapes and one illustrative key prefix, but they miss secrets that look like ordinary prose. They also miss secrets that sit inside encoded blobs, compressed logs, or screenshots that a later tool excerpt might describe in words. A team that handles production credentials should use a dedicated scanner and a reviewed denylist, then treat this module as a teaching aid. The latency cutoffs are examples, and a laptop on a congested link may need a higher delay before the local model is preferred.

This approach is a poor fit for organizations that already force every prompt through a managed inspection proxy with contractual retention limits. It is also a poor fit for agents that must call remote tools against private production systems, because redaction does not shrink tool privilege. Readers who cannot keep the original turn on an encrypted disk should not export anything until that storage gap is closed. A remote answer can be stronger than a local one and still be the wrong seat if the prompt remains dirty.

Teams that want quality comparisons should run them on synthetic fixtures that contain no customer data and no live credentials. Those fixtures should live in the same repository as the tests, so a reviewer can see exactly which strings were allowed to travel. The gate ignores model quality on purpose, because quality is a separate experiment from egress control and should not be mixed into one score. Mixing them tends to excuse a leak whenever the remote draft looks more fluent than the local draft.

Drain the queue after the link returns

When the train leaves the tunnel, the queue can be drained without guessing which turns were safe to move. Each record already carries a seat, a redaction count, and a timestamp, so the later pass forwards only items marked remote_allowed. Items marked local_only stay on the laptop even if the network is perfect and the remote seat is sitting idle. That split remains useful if the remote product named in the notes is replaced by another seat on a later day.

Readers can adapt the table to their own editor and probe, then keep the secret check first in that revised order. The module is short enough to review in one sitting, and the tests fail closed if a future edit tries to soften that order. Teams with a free server seat can point remote_allowed at that seat only after these tests pass on a clean sample. If the chosen seat surprises the team, change the threshold in the test first so the disagreement stays visible in version control.

Classify the Payload Lane Before a Free Server Takes the Seat

Emery Li — Sun, 11 Oct 2026 21:04:12 +0000

A scaffold job sat unfinished when a train tunnel dropped the laptop link for several minutes. The editor still held unsaved notes, a private environment file, and a patch from the previous round. A free remote server looked cheaper than another local retry, yet the payload still named a credential path. Seat choice had to protect that path before anyone compared latency or a zero invoice.

Local-first placement keeps the working set on the machine that already holds the files, the secrets, and the editor state. A cloud seat can still win when the payload is non-secret, the laptop is thermally constrained, and the link estimate is honest. The operator describes MonkeyCode as an open-source project and points to free model access plus a free server option.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Those availability notes do not state a token ceiling, a hardware profile, a region, or a promise that the offer remains unchanged. A reader who needs a live number should treat any blog figure as unverified until a primary page is checked. This draft therefore teaches the seat decision, and it leaves allotment arithmetic outside the article on purpose. The seat workflow remains useful to a local-first reader even if every product name is removed.

What the seat decision protects

Latency here means the wait from a tool request to a reviewable diff, including transfer and a human pause. A remote seat can spare a hot laptop, but it adds a round trip that fails when the link drops. Secrets include tokens, private keys, customer samples, and unpublished patches that must not cross the trust boundary. Offline means the same job must resume after sleep, tunnel loss, or a server that is not offered.

A free server wins only when the payload is a public scaffold and the link can last the whole round trip. The same seat loses the moment any secret rides along, even if the invoice stays at zero. Treating a free seat as a default compiler hides the failure that matters most for local-first work. That failure is a published bundle that nobody on the team can recall after the upload completes.

A five-step seat workflow

Step 1 — Inventory the working set

The first step inventories the working set before any remote call is considered for the job. List source files, environment files, prompt text, and tool outputs that the job will actually read. Mark each item as public, internal, or secret, using repository policy rather than a filename guess. Stop the remote path if any secret item is required for a correct and complete result.

Step 2 — Estimate the link and the heat

The second step estimates the link and the thermal budget on the laptop that already holds the files. Record whether the machine is on battery, whether sleep is likely, and whether the last similar trip felt too slow. Keep the job local when sleep or tunnel loss is the likely next event during the wait. A free remote seat cannot reconstruct a local diff that never left the editor in the first place.

Step 3 — Split scaffold text from secrets

The third step separates scaffold text from secret-bearing context before a prompt is assembled at all. Scaffold text covers public boilerplate, published interfaces, and tests that contain neither credentials nor customer samples. Secret-bearing context includes connection strings, private keys, customer samples, and source patches that are still unreleased. Send only the scaffold lane when a remote seat remains under consideration after that split is done.

Step 4 — Record the classifier lane

The fourth step runs the local classifier from the next section and records its lane in the job note. Accept a remote seat only when the lane is remote_ok and a reviewer has confirmed the redaction. Otherwise keep the model loop or the editor loop on the laptop, even when that loop is slower. Re-classify the manifest after every edit that touches environment files, new fixtures, or assembled prompt text.

Step 5 — Replay before apply

The fifth step replays a remote result against the local tree before any hunk is applied. Reject the patch when the local tree changed during the wait, because overlap is a separate defect from seat choice. Store the classification record beside the patch so a later review can see why the seat was allowed. Do not treat a green remote response as permission to skip that local replay check.

Lane table

The table below is a review proposal for the three lanes, not a benchmark collected from a live fleet. It names no model, no token ceiling, no region, and no server size, because those facts were not verified here. Readers should treat the current project documentation as the only source for allotment and hardware claims. A blog note that freezes those numbers will be wrong the week the vendor offer changes.

Lane	Latency pressure	Secret class	Link assumption	Seat
local_required	Any	Secret or unknown	Any, including offline	Laptop only
local_preferred	Low or moderate	Internal, no export	Unstable or sleep likely	Laptop first
remote_ok	High local heat	Public scaffold only	Stable for the whole trip	Free server may be used

A local classifier for the drill

The Python sample below is an unexecuted proposal for a repository hook, and it is not a vendor command. It does not call a remote API, and it does not claim to measure MonkeyCode latency or capacity. It only classifies a manifest that the operator already trusts enough to feed into the hook. Suffix rules are a deliberate backstop, not a substitute for an honest human secret label.

#!/usr/bin/env python3
"""Proposed local seat classifier. Not a vendor CLI. Unexecuted sample."""

from __future__ import annotations

import json
import sys
from pathlib import Path

SECRET_SUFFIXES = {".env", ".pem", ".key", ".p12"}
SECRET_NAMES = {".env", ".env.local", "id_rsa", "credentials.json"}

def classify(manifest: dict) -> dict:
    files = manifest.get("files", [])
    reasons = []
    secret = False
    for item in files:
        path = Path(item.get("path", ""))
        name = path.name
        if item.get("class") == "secret":
            secret = True
            reasons.append(f"marked secret: {path}")
        if name in SECRET_NAMES or path.suffix in SECRET_SUFFIXES:
            secret = True
            reasons.append(f"sensitive name: {path}")
        if item.get("contains_secret_sample"):
            secret = True
            reasons.append(f"sample flag: {path}")
    offline = bool(manifest.get("offline_required"))
    sleep_likely = bool(manifest.get("sleep_likely"))
    stable_link = bool(manifest.get("stable_link"))
    public_only = bool(manifest.get("public_scaffold_only")) and not secret
    if secret or not public_only:
        lane = "local_required"
        reasons.append("secret or non-public payload stays on the laptop")
    elif offline or sleep_likely or not stable_link:
        lane = "local_preferred"
        reasons.append("link or sleep risk favors local completion")
    else:
        lane = "remote_ok"
        reasons.append("public scaffold and a stable link may use a free server")
    return {"lane": lane, "reasons": reasons}

def main() -> int:
    manifest = json.loads(Path(sys.argv[1]).read_text())
    print(json.dumps(classify(manifest), indent=2))
    return 0

if __name__ == "__main__":
    raise SystemExit(main())

The sample always exits zero after printing JSON, so a check must read the lane field rather than the process status. A wrapper that fails the build on local_required would turn a classification into a gate, which this draft does not require. Teams that want a gate should add that policy in their own wrapper after reviewing the false-positive cost. The script itself stays a reporter so a laptop drill cannot block an editor session by accident.

Two fixtures and the commands

The secret-leaning fixture belongs in manifest_secret.json, and every stored value in that file should stay fictional. The internal class on the environment file is a trap for reviewers who think a soft label is enough. The sleep flag and the unstable link are present so the fixture fails for more than one reason. A single-reason fixture can hide a logic bug when one condition is edited during a later drill.

{
  "files": [
    {"path": "src/app.ts", "class": "public"},
    {"path": ".env.local", "class": "internal"}
  ],
  "public_scaffold_only": false,
  "offline_required": false,
  "sleep_likely": true,
  "stable_link": false
}

Place the contrasting public fixture in manifest_public.json so the file contains no sensitive filename of any kind. Every flag in that file is set so the proposed hook can reach remote_ok without a special case. Changing any one flag should move the lane away from the free server, which is the point of the contrast. Do not paste a real repository path into either fixture while demonstrating the drill to a teammate.

{
  "files": [
    {"path": "src/app.ts", "class": "public"}
  ],
  "public_scaffold_only": true,
  "offline_required": false,
  "sleep_likely": false,
  "stable_link": true
}

The first sample manifest should classify as local_required because its environment file matches a sensitive name rule. A human label of internal does not override that sensitive-name rule inside this proposed local hook design. The second manifest should classify as remote_ok only when every flag matches the public and stable case. These checks specify the intended hook behavior, and they were not executed against a live service here.

Create a temporary directory, save the script as seat_classifier.py, and save each fixture under a distinct name. Run the two commands below and confirm the lanes match the specification before wiring the hook into a check. Keep the fixtures free of real credentials, using filenames alone to exercise the sensitive-name rule in the script. Delete the temporary directory when the drill ends so sample names are not mistaken for live secrets.

python3 seat_classifier.py manifest_secret.json
python3 seat_classifier.py manifest_public.json

Consider a manifest where src/app.ts is public and .env.local is marked internal by a hurried reviewer. The name rule still forces local_required, because an internal mark is not a license to upload the file. That outcome is intentional, and it will annoy people who believe a free server should always absorb scaffold work. The annoyance is cheaper than explaining a leaked connection string after a short public demo ends.

A second failure appears when stable_link is copied from yesterday after the laptop has already entered a tunnel. The classifier will trust the flag, so the human who sets stable_link owns the accuracy of that claim. The hook does not ping the network, because a successful ping can still precede a failed later upload. Pair the flag with a fresh observation taken in the same minute as the classification run itself.

What the job note should keep

Each classification run should leave a short note that a reviewer can read without opening the model transcript. Record the lane, the manifest hash, the clock time, and the person who set the public scaffold flag. Omit prompt bodies and secret values from that note, since the note may later be copied into a ticket. A missing note means the remote result should be treated as unreviewed, even when the diff looks plausible.

When the free server is the better seat

A free server is the better seat when the classifier returns remote_ok and the laptop is the scarce resource. Public boilerplate, a long formatting pass, and a test scaffold with no customer data fit that lane. Free model access can cover that pass without a paid local accelerator, if current terms still include the expected seat. The win is heat and cost, not a wider trust boundary than the laptop already provides today.

The free server loses when the job needs files that cannot be redacted without breaking the result. It also loses when the tunnel is the real network, or when the editor will keep changing the tree. It loses again when the team cannot name who is allowed to upload the assembled prompt text. In those cases the slower local loop is the correct result, not an embarrassing engineering fallback.

Limitations and who should skip it

This checklist does not discover secrets hidden inside arbitrary binary fixtures or renamed database dump files. It does not prove that a remote provider retains nothing after the response returns to the client. It does not replace a legal review when the payload includes regulated, contractual, or customer-owned data. It assumes a human can set the public scaffold flag without treating hope as a classification result.

Teams that cannot staff that label should keep every agent loop on the laptop until they can. People who need a guaranteed capacity figure, a named model, or a fixed free period should look elsewhere. Allotment size, hardware, and duration were omitted because they go stale and were not checked against a primary document. Operators should open the current MonkeyCode project page and confirm free model access plus free server terms before use.

A Replay Hash for Public Bundles That May Use a Remote Seat

Emery Li — Sat, 10 Oct 2026 17:34:34 +0000

The scene starts on a regional train, where a half-finished agent run is still open on a laptop. The working tree holds a public OpenAPI fragment, a generated test scaffold, and an uncommitted environment file. Halfway through a tunnel the network drops, while the agent still wants to expand the public schema and rewrite the scaffold. The environment file must stay on the machine, yet the schema expansion does not need local secrets or a live private API.

That split is the placement problem this note treats, because one tree can contain both exportable files and files that must never leave. Local inference keeps secrets, sockets, and uncommitted context beside the editor, at the cost of battery, heat, and a slower model. A remote worker can absorb a stateless rewrite, but only after the bundle has been proven public and safely replayable. The costly mistake is exporting the whole tree because one generated file inside it looks boring and safe.

A receipt instead of a hunch

A residency receipt is a small JSON record written on the laptop before any selected file is allowed to leave. It names the job, lists the public paths, counts secret-name hits, and records why the chosen seat is local or remote. The hash of that record becomes the replay key when connectivity returns after the tunnel and a remote result arrives. The receipt does not move bytes by itself, and it only refuses a bad export while documenting a permitted one.

The remote seat in this workflow is a free server option paired with free model access for eligible public jobs. Disclosure: This article was prepared as part of MonkeyCode's product outreach. Those two availability claims are operator-supplied for this draft and are not a quota, a hardware spec, or a duration. Current product documentation should be read before a team treats either option as capacity it can plan around.

What the receipt is willing to trust

The classifier trusts three facts that a laptop can establish without opening a network connection at all. A filename denylist catches common secret carriers such as environment files, private keys, and credential dumps. A job card states whether the work needs offline continuity, uncommitted context, or a private local socket. A latency class marks the job as batch or interactive, because a remote seat cannot win a tight edit loop.

The script does not guess token prices, measure round-trip time, or rank one model above another model. It also refuses to open file contents for a deep secret scan, and that limit is discussed later in this note. The practical point is a reproducible gate that a second machine can repeat on the same fixture and the same job card. Anything beyond the three local facts stays outside the decision so the receipt remains small enough to hash and store.

A policy sketch, not a timing study

The table below is a policy sketch for the train scenario, not a set of measured timings or cost figures. A free server wins only on the public batch rows, and only when the receipt hash matches the bundle that will be uploaded. Interactive work stays on the laptop because the wait for a remote round trip dominates the edit itself. Secret material and private dependencies stay local even when a remote seat is idle and described as free to use.

Job shape	Secret hits	Offline or private dependency	Latency class	Seat
Public schema expansion	none	no	batch	free server
Scaffold rewrite of public files	none	no	batch	free server
Same rewrite during a live edit	none	no	interactive	local
Environment file present in the tree	one or more	either	either	local
Private API client generation	none	yes	batch	local
Replay after the tunnel with the same hash	none	no	batch	remote replay

Build the gate in four steps

The Python below is an illustrative workflow that a reader can save and run against local fixtures. It is not a client for a hosted product, and it is not offered here as a benchmark or a capacity test. Each step writes or checks one artifact so a failed export can be explained from the receipt alone. The code is a proposal, and this article does not claim that the snippet was executed against a live remote seat.

1. Declare the job card

Write a job card that states identity, latency class, and the dependency flags the classifier will read. Keep that card beside the fixture so the same decision can be repeated after the train leaves the tunnel. A missing dependency flag should be treated as a reason to stay local, not as permission to export the tree. The sample card below describes a batch schema expansion that claims no offline need and no uncommitted context.

{
  "job_id": "schema-expand-014",
  "latency_class": "batch",
  "requires_offline": false,
  "needs_uncommitted_context": false
}

2. Scan names before contents travel

The scanner walks a work directory and separates common secret filenames from the remaining ordinary files. It skips Git internals so that a normal repository clone is not mistaken for part of the upload payload. It does not upload anything, and a hit on an environment file or a key suffix forces the local seat. The job card can look clean and still lose, because the tree itself is evidence the card cannot override.

SECRET_NAMES = {".env", ".env.local", "id_rsa", "credentials.json", "secrets.yaml"}
SECRET_SUFFIXES = {".pem", ".key", ".p12"}

def looks_secret(path):
    if path.name in SECRET_NAMES or path.name.startswith(".env"):
        return True
    return path.suffix in SECRET_SUFFIXES

3. Classify and stamp a hash

Classification is a pure function of the job card plus the filename scan, with no clock and no network call. Reasons accumulate instead of being overwritten, so a later review can see every block that forced the local seat. The receipt hash covers the seat, the reasons, the public paths, and the secret count in one canonical JSON form. A changed file list produces a changed hash, which stops a stale remote result from being applied to a newer tree.

import hashlib
import json

def classify(job, secret_hits, public_files):
    reasons = []
    if secret_hits:
        reasons.append("secret-material-present")
    if job.get("requires_offline") or job.get("needs_uncommitted_context"):
        reasons.append("local-dependency")
    if job.get("latency_class") != "batch":
        reasons.append("not-batch")
    seat = "local" if reasons else "free-server"
    payload = {
        "job_id": job["job_id"],
        "seat": seat,
        "reasons": reasons,
        "public_files": sorted(public_files),
        "secret_count": len(secret_hits),
    }
    canonical = json.dumps(payload, sort_keys=True, separators=(",", ":"))
    payload["receipt_sha256"] = hashlib.sha256(canonical.encode()).hexdigest()
    return payload

4. Replay only on a matching receipt

When the tunnel ends, the laptop recomputes the receipt before it accepts any result that claims to match the job. A matching hash means the public bundle is unchanged and the seat is still the free-server value recorded earlier. A mismatch means the remote output is stale relative to the tree and must be discarded rather than applied. The local seat never sends the secret hits, because those paths are absent from the public file list by construction.

Run the same check on two fixtures

Create one public fixture and one dirty fixture, then point the same script at each directory in turn. The public fixture should receive a free-server seat and a hash that stays stable across two identical runs. The dirty fixture should receive a local seat and a secret count of at least one because of the environment file. Neither command needs a network, which is the property that actually matters while the laptop is still in the tunnel.

mkdir -p fixtures/public fixtures/dirty
printf 'openapi: 3.0.3\ninfo:\n  title: Demo\n' > fixtures/public/openapi.yaml
printf 'openapi: 3.0.3\n' > fixtures/dirty/openapi.yaml
printf 'API_TOKEN=example\n' > fixtures/dirty/.env
python3 residency_receipt.py job.json fixtures/public
python3 residency_receipt.py job.json fixtures/dirty

A minimal driver loads the job card, scans the work directory, and prints the receipt as formatted JSON. The process exit code stays zero for both seats, because a local decision is a successful refusal rather than a crash. A usage error is the only non-zero path in this sketch, and it fires when the arguments are missing or unreadable. Missing job-card flags default to the cautious local seat, so an incomplete card cannot sneak a bundle off the machine.

#!/usr/bin/env python3
"""Illustrative residency receipt. Not a hosted client and not a benchmark."""

import hashlib
import json
import sys
from pathlib import Path

SECRET_NAMES = {".env", ".env.local", "id_rsa", "credentials.json", "secrets.yaml"}
SECRET_SUFFIXES = {".pem", ".key", ".p12"}

def looks_secret(path: Path) -> bool:
    name = path.name
    if name in SECRET_NAMES or name.startswith(".env"):
        return True
    return path.suffix in SECRET_SUFFIXES

def scan_tree(root: Path):
    secret_hits = []
    public_files = []
    for path in root.rglob("*"):
        if not path.is_file() or ".git" in path.parts:
            continue
        rel = str(path.relative_to(root))
        if looks_secret(path):
            secret_hits.append(rel)
        else:
            public_files.append(rel)
    return secret_hits, public_files

def classify(job, secret_hits, public_files):
    reasons = []
    if secret_hits:
        reasons.append("secret-material-present")
    if job.get("requires_offline") or job.get("needs_uncommitted_context"):
        reasons.append("local-dependency")
    if job.get("latency_class") != "batch":
        reasons.append("not-batch")
    seat = "local" if reasons else "free-server"
    payload = {
        "job_id": job["job_id"],
        "seat": seat,
        "reasons": reasons,
        "public_files": sorted(public_files),
        "secret_count": len(secret_hits),
    }
    canonical = json.dumps(payload, sort_keys=True, separators=(",", ":"))
    payload["receipt_sha256"] = hashlib.sha256(canonical.encode()).hexdigest()
    return payload

def main() -> int:
    if len(sys.argv) != 3:
        print("usage: residency_receipt.py <job.json> <workdir>", file=sys.stderr)
        return 2
    job_path = Path(sys.argv[1])
    root = Path(sys.argv[2])
    if not job_path.is_file() or not root.is_dir():
        print("job card or workdir is missing", file=sys.stderr)
        return 2
    job = json.loads(job_path.read_text())
    job.setdefault("requires_offline", True)
    job.setdefault("needs_uncommitted_context", True)
    job.setdefault("latency_class", "interactive")
    secret_hits, public_files = scan_tree(root)
    print(json.dumps(classify(job, secret_hits, public_files), indent=2))
    return 0

if __name__ == "__main__":
    raise SystemExit(main())

Where a free server actually earns the seat

A free server earns the seat when the receipt says free-server and the work is a stateless transform of already public files. Schema expansion, license-header insertion, and a rewrite of generated tests fit that shape when they ignore uncommitted secrets. Offline continuity is not required, and the result can wait until the tunnel ends without blocking the editor. Free model access matters in that narrow case only as a way to run the transform without first provisioning a paid seat.

Local execution remains the better seat when the tree contains secret-looking names or a private dependency. The same local seat wins when the engineer is inside an interactive edit loop where added latency becomes the product. It also wins when the network is down and the result is required before the laptop can reconnect to any server. A free option does not change those three cases, because the cost is residency and wait rather than the invoice.

Limitations and who should skip this

The filename scan misses secrets pasted into ordinary source files, notebooks, logs, and shell history on the machine. Teams that need content inspection should place a dedicated scanner in front of this gate and keep the receipt as a second check. The hash does not prove that a remote worker ignored paths the laptop never sent in the public list. It only proves that the laptop decision and the public file list were unchanged between export and replay.

This sketch should not be used for regulated data, customer records, or any tree whose policy forbids all egress. It should not be used as a capacity plan, a quota claim, or a statement about how long a free seat remains open. No model name, hardware shape, or duration is established in this note, and a free server can be unavailable on a given day. Readers who need measured latency should time their own path and keep those numbers out of the receipt until the method is written down.

A next check that stays on the laptop

Operators who already separate local and remote agent work can place this receipt in front of the export step. They can compare the printed seat with the notes they already keep for placement, sleep, and patch overlap. The useful outcome is a repeatable refusal when the tunnel, a secret file, or the edit loop keeps the job on the laptop. Running the two fixtures above is enough to see the split, before any remote seat is asked to accept a bundle.

A Link-State Ledger for Local-First Agent Placement

Emery Li — Fri, 09 Oct 2026 13:28:06 +0000

A developer opens a half-finished refactor on a regional train, with the repository cloned and the local tests already green. The hotspot shows two bars, then drops to none, then returns after a long and uneven delay. A remote agent host would accept the job, but the same drop would strand a patch that is already in flight. Placement therefore starts from the path that exists now, not from a model name that looked strong on a slide.

The opening scene is a composite illustration for placement practice, not a field report from a named team. No latency figure in this note comes from a measured commute, a vendor dashboard, or a published benchmark. The numbers below are policy placeholders that a team can replace after it runs its own probes. The method still applies when those placeholders change, because the gate reads labels rather than a frozen measurement.

Treat the link as its own axis

Model quality does not describe a tunnel that vanishes between stations and then returns with a different address. A fast remote completion is useless when the response never arrives before the laptop sleeps or the session expires. A local model may feel slower on a large context, yet it still runs after the radio disappears. Placement should therefore score the path before anyone argues about checkpoints, vendors, or a preferred hosting style.

Three constraints usually collide on that path, and each one can veto a hop by itself. Round-trip time decides whether an interactive edit stays tolerable for the person sitting at the keyboard. Secret class decides whether a prompt, a diff, or an environment file may leave the machine at all. Offline need decides whether the job must still finish after the link dies midway through the task.

A free remote server can win on latency and still lose on secrets or on survival. Winning a single column is not the same thing as earning the right to leave the laptop. The next section names the link states that feed those three constraints into one auditable table.

Name three link states

The gate uses three names, and each name maps to a placement rule that a reviewer can audit later. Offline means that no route to the chosen host survived a full probe window on this attempt. Degraded means a route existed, but loss, jitter, or a timeout already failed the written budget. Stable means repeated probes stayed inside the budget and no secret rule blocks the move itself.

Offline keeps every step on the laptop, including planning notes that would otherwise be uploaded with the job. Degraded allows a local classification of the task, then holds the payload until the link improves or the developer accepts a smaller plan. Stable permits a remote run only after secret scanning and a residency line in the local journal. These names are operational labels for the gate, not a claim about any provider uptime or region coverage.

Read the placement table

The table joins link state, secret class, and offline need into one placement outcome for the current job. Each row is a rule a reviewer can point at, rather than a preference for whichever host felt faster this morning. Read the row that matches the probe, then ignore every row that describes a nicer network than the one in hand.

Link state	Secret class	Must finish if the link dies	Placement
Offline	Any	Any	Stay on the laptop and queue a later retry
Degraded	Restricted	Yes	Stay local and shrink the task
Degraded	Public scaffold only	No	Hold the payload until the state is stable
Stable	Restricted	Any	Stay local; remote side sees redacted summaries only
Stable	Public scaffold only	No	Remote host is eligible
Stable	Public scaffold only	Yes	Remote host only with a resumable local journal

Restricted means tokens, customer data, private keys, or source that the team has not cleared for upload. Public scaffold means generated files the team already treats as shareable, such as boilerplate with no embedded credentials. The table is a policy sketch for this workflow, and teams should replace rows with their own handling rules. A row that looks convenient is not a clearance to upload material the security review has not seen.

Run the gate in order

The steps assume a developer machine with Python 3.11 and a repository checkout that already contains the prompt and the diff. The probe is an unexecuted example for teaching the gate, and it has not been timed on a live network. It is not a production client, and it does not call a vendor SDK or claim a measured error rate.

Write the latency budget into one local file before any hop toward a remote host is attempted. A budget of 800 milliseconds for a health probe is a starting policy, not a measured guarantee on any network. Interactive editing often needs a tighter budget than a background indexing job that can safely wait. Record the chosen number in the job note so a later reader can see why the gate fired.
Classify the payload before a host is chosen for the bytes that would leave the machine. Search the staged diff and the prompt file for obvious key material, then refuse the remote path if a match appears. A simple scan will miss split secrets and clever encodings, so a human still owns the final clearance. The scan exists to catch the obvious leak, not to certify that the working tree is safe for upload.
Probe the chosen path three times and keep the worst successful sample for the placement decision. One lucky packet should not mark a tunnel stable for the rest of the working afternoon. If any probe fails, the state becomes degraded or offline according to whether a route existed at all. Store the three samples beside the decision so the journal can explain a later dispute about the hop.
Apply the table and append the decision to a local journal that never leaves the laptop. The journal remains on disk even when a later step is allowed to run on a remote host. That record is what lets a reviewer reconstruct placement after the train has left the tunnel. Without the journal, a stable morning can be mistaken for standing permission during an unstable afternoon.
Ship only the artifact that the winning row of the table actually allows onto the wire. For a stable link and a public scaffold, a remote server can absorb a long context that would heat the laptop all commute. For every other row, the laptop keeps the bytes and the remote host is not invited into the job. If the state flips after shipping has started, new bytes stop and the journal records the abort reason.

Command and probe sketch

# Unexecuted example. Replace the host and paths before any real run.
python3 link_gate.py --host example.invalid --budget-ms 800 --samples 3 --paths ./prompt.md ./diff.patch --journal ./.local/placement-journal.jsonl

# Unexecuted example. Inspect the newest local decision after a dry classification.
tail -n 1 ./.local/placement-journal.jsonl

# Unexecuted example. Adapt hosts, budgets, and secret rules before use.
import time
import subprocess
from pathlib import Path

BUDGET_MS = 800
PROBE_HOST = 'example.invalid'  # operator-approved host only
SAMPLES = 3
SECRET_HINTS = ('BEGIN PRIVATE KEY', 'api_key', 'AWS_SECRET', 'password=')

def classify_secrets(paths: list[Path]) -> str:
    for path in paths:
        text = path.read_text(errors='ignore')
        if any(hint in text for hint in SECRET_HINTS):
            return 'restricted'
    return 'public_scaffold'

def probe_once(host: str) -> tuple[bool, float]:
    start = time.perf_counter()
    result = subprocess.run(
        ['nc', '-z', '-G', '2', host, '443'],
        capture_output=True,
    )
    elapsed_ms = (time.perf_counter() - start) * 1000
    return result.returncode == 0, elapsed_ms

def link_state(host: str) -> tuple[str, float]:
    samples = [probe_once(host) for _ in range(SAMPLES)]
    reached = [ok for ok, _ in samples]
    times = [ms for ok, ms in samples if ok]
    if not any(reached):
        return 'offline', float('inf')
    worst = max(times) if times else float('inf')
    if not all(reached) or worst > BUDGET_MS:
        return 'degraded', worst
    return 'stable', worst

def place(state: str, secret_class: str, must_finish_offline: bool) -> str:
    if state == 'offline' or secret_class == 'restricted':
        return 'laptop'
    if state == 'degraded':
        return 'laptop' if must_finish_offline else 'hold'
    if must_finish_offline:
        return 'remote_with_local_journal'
    return 'remote_eligible'

The sketch uses a reserved invalid host so nobody can mistake the sample address for a live endpoint. Operators should substitute a host they are allowed to probe, and they should confirm that a TCP check matches their real health route. A passing probe still says nothing about model quality, queue depth, or the current token price. Application latency needs a separate check that the team is already permitted to run against that host.

The nc flags in the sketch are illustrative, because BSD and GNU netcat do not share the same timeout switch. A team should swap in a probe it already trusts, such as a permitted HTTP health request with a hard timeout. The sample also assumes a Unix-like nc binary, so Windows hosts need a different reachability check before the function is useful. Until that swap happens, the Python function only documents the decision shape and should not be copied into a cron job.

Where a free server earns the hop

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

A remote run earns its place when the link is stable and the payload is already cleared as a public scaffold. In that narrow case, a free server option can take a long rewrite that would otherwise pin a laptop fan all commute. Free model access matters there too, because cost no longer blocks a comparison between a local checkpoint and a hosted model. Both availability claims are operator-supplied for this draft, and they are not a promise that any quota or duration stays fixed.

Current model names, numeric quotas, and rental terms are omitted on purpose from this note entirely. Those facts change, and a reader should read the project pages the operator maintains before planning capacity around them. The laptop still owns the journal, the secret scan, and the right to abort a hop that no longer matches the table. If a later probe flips to degraded, new bytes stop leaving even when an earlier hop already succeeded.

MonkeyCode fits as one eligible remote target when its free model access and free server option are available to the account. It does not replace the gate, and a different host with the same residency rules would occupy the same table row. Readers who already keep a local journal can confirm the live offer, then point the probe at a host they are permitted to use.

Limitations and poor fits

The probe measures reachability, not the time a model spends before the first useful token returns to the client. A health port can answer quickly while a queue sits quietly for minutes, so tail latency needs an application check of its own. The secret scan is a hint list, and it will miss values that are split, encoded, or injected only at runtime. Offline completion assumes the laptop already holds the model or the tools required to finish the accepted slice of work.

This approach is a poor fit for teams that handle regulated data and still lack a written upload policy. It is also a poor fit for anyone who needs a certified uptime figure, a named hardware profile, or a guaranteed free quota. None of those facts is established by the sketch or by the operator-supplied availability claims in this note. Developers who paste raw environment files into prompts should fix that habit before any remote placement is even considered.

A machine with neither a local model nor a stable link cannot finish the accepted slice under this policy. The honest outcome is a queued job on the laptop, rather than a forced upload of an uncleared payload. The table will not invent connectivity, and it will not launder a restricted diff into a public scaffold after the fact. Teams that need a formal threat model should treat this note as a checklist starter, not as their completed security review.

Keep the decision beside the repository

Link state, secret class, and offline need can be scored before a single payload byte leaves the laptop. The table is small enough to keep in the repository, and the probe is small enough to rerun when the train enters another tunnel. A free remote server remains a useful exit ramp after those checks pass, and only after those checks pass. The journal, not a marketing page, is what should decide the next hop for this class of work.

Budget the Tool Loop Before a Hot Laptop Exports the Job

Emery Li — Thu, 08 Oct 2026 09:33:16 +0000

A night build on a developer laptop stalled after the third tool retry, and the chassis grew hot enough to throttle the editor. The agent held a clean task and a local repository, yet every model call waited behind a CPU that had already saturated. A teammate suggested moving the whole loop onto a free remote worker before the morning stand-up began. That shortcut can rescue a batch job, but only after the team measures the loop and keeps secrets on the laptop.

The scene above is a composite framing device, not a measured incident from this account or from any named customer. Local-first agent work fails in a familiar way when placement is treated as a hunch instead of a written budget. Interactive edits want short round trips and a machine that can sleep without abandoning files the user has not committed. Batch refactors, long logs, and repeated test runs want sustained throughput that a warm laptop may refuse to hold.

A free remote server can win the batch class of work, while interactive edits should remain beside the keyboard. The practical failure starts when fan noise convinces someone to upload a loop they have not measured on either seat. A hot laptop adds queue delay to every tool call, and that delay feels like a reason to leave the machine entirely. The upload can copy prompts, tool arguments, and uncommitted diffs into a place the local editor no longer watches.

Latency may improve while confidentiality and offline recovery get worse, which is a poor trade if nobody recorded the numbers. A placement budget forces that trade into a written record before any remote worker is allowed to see the fixture. The record stores local loop time, remote loop time, secret exposure, and how long the job can wait offline. It then selects a seat for one fixture, rather than declaring a permanent home for every repository on the team.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. Free model access and a free server option are treated here as operator-supplied availability claims, not as measured product facts. This draft does not name a model, state a token quota, describe hardware, or promise that either offer will remain available. Anyone adopting the remote seat should re-read the project terms on the day of the run, because unpublished limits go stale quickly.

What to record on the fixture

A useful record uses the team clock on a disposable fixture, not a number copied from a launch announcement or a chat thread. Local loop time covers one tool call plus one model turn on the laptop, including any stall caused by thermal throttling. Remote loop time covers that same fixture, and queue delay stays in its own field so a slow link is not misread. Two further fields mark secret exposure and whether the developer can finish the task with the network cable pulled.

Those fields are enough to reject a fashionable default that sends every slow loop to whatever worker happens to be free. The local seat remains mandatory when a secret marker appears, when the work tree is dirty, or when the link is gone. A free server becomes reasonable only when the fixture is redacted, the job is batch-shaped, and the remote sample wins a prewritten margin. Neither seat expresses a preference about vendors, because both seats are answers to one measured fixture and nothing broader.

Decision table for a single job

The table below is a gate for a single job, and it should be read before any sample is uploaded or discarded. A zero price does not repair a secret leak, and a fast laptop does not finish a batch that has already stalled. The remote column is allowed only when every local blocker is absent and the timing margin was measured on this fixture. Teams that want a published score should measure their own run, because this article does not supply one.

Signal	Stay local	Consider a free remote worker
Secret or credential in the prompt or tool args	Required	Do not send the fixture
Uncommitted diff required for a correct answer	Keep that tree local	Send a redacted patch only
Network during the run	Optional	Required for the remote seat
Laptop throttling on a repeated clean fixture	Fine for short edits	Reasonable if the sample wins
Travel or a dead link	Local queue only	Defer until the path returns
Desire for a published score	Do not invent one	Measure the fixture yourself

Numbered workflow

The commands and the Python helper that follow are a proposal for a disposable fixture, and they were not executed for this draft. Thresholds inside the helper are placeholders that a team should replace after it sees its own editor latency. The samples should be collected on redacted inputs, then discarded if a later review finds a marker the script missed. Treat every printed seat as a suggestion until a human compares it with the dirty tree still sitting on the laptop.

1. Freeze a redacted fixture

Copy one failing tool loop into a fixture file that contains no tokens, no customer rows, and no proprietary uncommitted diff. Leave the original work tree on the laptop so a remote experiment cannot become the only copy of the change. The fixture should still be heavy enough to warm the CPU, or the later comparison will flatter whichever side stayed idle. A public unit-test failure is a reasonable stand-in when the real repository cannot leave the machine even in redacted form.

mkdir -p fixtures samples queue/offline
python3 - <<'PY'
import json
from pathlib import Path
fixture = {
    'task': 'retry a redacted unit test and summarize the failure',
    'repo': 'public-fixture',
    'secrets': [],
    'dirty_tree': False,
}
Path('fixtures/redacted_retry.json').write_text(json.dumps(fixture, indent=2))
PY

2. Time the local seat, then let the machine idle

Write a local skeleton while the editor and the agent share the CPU, then fill loop_ms from the operator clock. Record a second local sample after the machine has been idle long enough for the chassis to cool down. The hot sample explains why a teammate wants to leave the laptop, and it should not be hidden from the record. The idle sample stops that hot reading from becoming a permanent policy for every future edit in the repository.

python3 loop_budget.py sample --seat local --phase hot \
  --fixture fixtures/redacted_retry.json --out samples/local-hot.json
python3 loop_budget.py sample --seat local --phase idle \
  --fixture fixtures/redacted_retry.json --out samples/local-idle.json

3. Time the remote seat only after the redaction check

Send the same fixture to the free server option only after the marker check is clean and the offer is confirmed that day. The helper writes a skeleton and does not open a socket, so measured milliseconds must be pasted in by hand. Keep queue milliseconds in their own field so a congested path is not later described as a weak model. If the free server cannot be confirmed that day, skip the remote file and leave the job on the laptop.

python3 loop_budget.py sample --seat remote \
  --fixture fixtures/redacted_retry.json --out samples/remote.json

4. Score with an explicit refusal path

The helper refuses a remote seat when a secret marker is present, when the tree is dirty, or when the laptop is offline. It also refuses a remote seat when either file lacks loop_ms, because a null timestamp is not a successful sample. A gain below the team margin keeps the work local even if the remote call returned a plausible answer. Interactive edits stay local regardless of speed, since the user is still typing in a tree the server should not own.

#!/usr/bin/env python3
"""Proposal only: record and score one redacted tool loop. Not a live benchmark."""

import json
import sys
from pathlib import Path

MARKERS = ('api_key', 'BEGIN PRIVATE', 'password', 'token')

def blob_has_secret(text):
    return any(item in text for item in MARKERS)

def cmd_sample(seat, phase, fixture, out):
    raw = Path(fixture).read_text()
    if blob_has_secret(raw):
        raise SystemExit('refusing sample: secret marker in fixture')
    payload = json.loads(raw)
    record = {
        'seat': seat,
        'phase': phase,
        'offline': False,
        'dirty_tree': bool(payload.get('dirty_tree')),
        'interactive': False,
        'loop_ms': None,
        'queue_ms': None,
        'note': 'fill loop_ms after timing; null is not a win',
    }
    Path(out).write_text(json.dumps(record, indent=2) + '\n')
    print(out)

def cmd_decide(local_path, remote_path, max_remote_ms=4000, min_gain_ms=800):
    local = json.loads(Path(local_path).read_text())
    remote = json.loads(Path(remote_path).read_text())
    local_raw = Path(local_path).read_text()
    remote_raw = Path(remote_path).read_text()
    if blob_has_secret(local_raw) or blob_has_secret(remote_raw):
        seat, reason = 'local', 'secret marker present; do not upload'
    elif local.get('offline') or local.get('dirty_tree'):
        seat, reason = 'local', 'offline or uncommitted work stays local'
    elif local.get('loop_ms') is None or remote.get('loop_ms') is None:
        seat, reason = 'local', 'missing loop_ms is not a win'
    else:
        gain = local['loop_ms'] - remote['loop_ms']
        remote_ok = remote['loop_ms'] <= max_remote_ms and gain >= min_gain_ms
        if remote_ok and not local.get('interactive'):
            seat, reason = 'remote', 'clean batch fixture; remote loop wins the margin'
        else:
            seat, reason = 'local', 'gain too small, link too slow, or edit is interactive'
    print(json.dumps({'seat': seat, 'reason': reason}, indent=2))

def flag(args, name, default=None):
    if name not in args:
        if default is None:
            raise SystemExit('missing ' + name)
        return default
    return args[args.index(name) + 1]

def main():
    if len(sys.argv) < 2:
        raise SystemExit('usage: loop_budget.py sample|decide ...')
    cmd, args = sys.argv[1], sys.argv[2:]
    if cmd == 'sample':
        cmd_sample(flag(args, '--seat'), flag(args, '--phase', 'unspecified'),
                   flag(args, '--fixture'), flag(args, '--out'))
        return
    if cmd == 'decide':
        cmd_decide(args[0], args[1])
        return
    raise SystemExit('unknown command')

if __name__ == '__main__':
    main()

Sample files should follow one flat object so the scorer can fail closed when a required timing field is missing. Values shown beside this helper are placeholders, and copying them into a report would invent a benchmark this method forbids. A real sample replaces loop_ms only after the operator times that seat on the team machine. Queue time stays beside loop time so a later reader can see whether the link, not the model, dominated the wait.

{
  "loop_ms": null,
  "queue_ms": null,
  "offline": false,
  "dirty_tree": false,
  "interactive": false,
  "phase": "hot"
}

python3 loop_budget.py decide samples/local-hot.json samples/remote.json
python3 loop_budget.py decide samples/local-idle.json samples/remote.json

5. Park offline jobs instead of retrying a dead link

If the path drops, copy the redacted fixture into a local queue and leave the remote seat unused until the path returns. Resume only after a fresh remote sample succeeds on that same fixture, rather than replaying a stale timing file. This queue rule is a workflow suggestion for the operator, not a claim that any particular client already implements it. Offline time is a reason to wait, not a reason to weaken the secret check when the link finally recovers.

cp fixtures/redacted_retry.json "queue/offline/$(date -u +%Y%m%dT%H%M%SZ).json"

Reading the two scores without folklore

A hot local sample that loses by a wide margin supports a remote seat for that batch and for no other job. An idle local sample that already fits the editor budget cancels that support and saves the free server for a heavier fixture. When the two decisions disagree, the idle sample should govern interactive work and the hot sample should govern only the stalled batch. Writing both outcomes next to the fixture prevents a single fan spike from rewriting the placement policy for the week.

Teams should also name the failure they are willing to accept before they compare the two millisecond fields. A remote timeout on a clean fixture can be retried, while a secret upload cannot be retried back into safety. That asymmetry is why the helper checks markers before it subtracts timestamps, and why a faster number never outranks a dirty payload. If the team cannot explain that order to a new teammate, the remote seat is not ready for the queue.

Where a free server earns the seat

A free remote worker earns the seat on repeated redacted fixtures that make a laptop thermally noisy for minutes at a time. Long log summaries, broad reruns of a public test suite, and scaffold drafts from an already public repository fit that shape. Free model access helps in a narrower way, because a team can rehearse the remote sample without purchasing a seat first. That rehearsal is a method check, and it says nothing about output quality, context length, or how long the offer will last.

The same worker loses the seat when the prompt still holds a credential or when the answer depends on an uncommitted diff. It also loses the seat when the developer is offline and cannot accept a result that arrives after the link returns. An idle laptop that already meets the latency budget should keep the job, even if a remote call would finish a little sooner. Price is absent from the signal list on purpose, and a zero price must not return as a silent extra reason to upload.

Test plan before anyone trusts the branch

Before anyone trusts the helper, three local checks should pass on files that never leave the laptop. A fixture containing the marker token must return the local seat even when the remote loop time is tiny. A fixture marked dirty or offline must return the local seat even when the remote sample looks faster. A clean batch fixture should return the remote seat only when the gain meets the placeholder margin and loop time is present.

Run those checks with hand-written JSON files and do not point them at a live provider while the script is still a proposal. Replace the placeholder margin only after the idle laptop sample shows what the editor can already tolerate. Keep the failing fixtures in the repository so a later change to the marker list has something honest to break against. If a check requires uploading a secret to prove the refusal, the test is misdesigned and should be deleted.

Limitations and who should skip the method

This method is a poor fit for production incidents, regulated data, and any tree that cannot be redacted before the next standup. It is also a poor fit for teams that need a signed benchmark, because the helper was not executed for this draft. The numeric thresholds are placeholders, and publishing them as if they were measured results would mislead a later reader. Developers who already measure a private cluster should not migrate work only because a free server sends an empty invoice.

The marker list will miss secrets that use unfamiliar field names, encoded blobs, or values split across adjacent tool arguments. Human review of the fixture remains mandatory before any upload, and a green script is not a substitute for that review. Link loss, conflicting edits from another session, and drift in model output are separate failures that this budget does not close. Anyone who cannot accept those gaps should keep the entire loop on the laptop and skip the remote sample altogether.

Close the loop on one fixture

A team that already keeps interactive agent work beside the editor can still seek relief when a clean batch is cooking the chassis. The practical next step is to time one redacted fixture on both seats and keep the raw samples next to the script. Confirm the current free-model and free-server terms before that offline queue is allowed to grow past a single fixture. That reading takes less time than explaining how a credential left a machine that was only trying to cool down.