Weigh the Payload Before the Hop

Remote models win on small, clean payloads. Local work wins on secrets and bulk. Mean latency is the wrong meter for that choice.

A working tree is not a prompt. It is a warehouse. You do not ship the warehouse to think. The hop fails in three distinct ways. Secrets leak. The payload bloats. The tail of the link spikes.

Any one of those should keep the job on disk. This article is a packing workflow. It is not a model review. The scripts below are proposals. Run them on a throwaway repo first.

Gate one is the secret class. Scan names, not contents, to start. A file named .env is already a veto. A *.pem is a veto. An id_rsa is a veto. Name checks miss secrets in code. They still catch the loud cases. Treat a hit as LOCAL_ONLY. Do not negotiate with the match.

Gate two is payload mass. Count the bytes you would send. Estimate tokens with a blunt heuristic. Character count divided by four is a guess. Label it as a guess in the log. A two-hundred kilobyte tree is not a chat. It is a cargo ship. Local summarizers can cut that ship into a barge. The barge is a unified diff plus a file list.

Gate three is tail latency. Ping is not enough. ICMP lies on many networks. Time a tiny HTTPS fetch several times. Sort the samples. Read p95, not the mean. A free remote looks cheap at the median. Noisy neighbors live in the tail. Your editor feels the tail.

When all three gates pass, a remote burst can help. Interactive secret work should stay local. Overnight non-secret refactors can drain a queue. Offline laptops should enqueue, not freeze. The queue is the local-first spine. The remote is overflow, not home.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode offers free model access and a free server option. Those two facts are the only product claims here. No quota, hardware, or model name is implied. Use that remote as an overflow drain. Do not point it at production secrets. Pack first. Then hop.

Start with a name veto. Keep it boring and fast. Fast vetoes beat clever parsers on a dirty tree.

# proposal: name-only secret veto, not a scanner product
from pathlib import Path

VETO_NAMES = {".env", ".env.local", "id_rsa", "id_ed25519"}
VETO_SUFFIX = {".pem", ".p12", ".key", ".kdbx"}
VETO_DIRS = {".git", "node_modules", ".venv", "__pycache__"}

def secret_hits(root: Path) -> list[str]:
    hits = []
    for p in root.rglob("*"):
        if any(part in VETO_DIRS for part in p.parts):
            continue
        if not p.is_file():
            continue
        if p.name in VETO_NAMES or p.suffix in VETO_SUFFIX:
            hits.append(str(p.relative_to(root)))
    return hits

A hit means the job never leaves disk. Absence of a hit is not clearance. It is only gate one. Move to packing only after this function returns empty.

Packing is the real work. Do not concatenate the repo. Ask git for the staged story. Untracked noise is not context. It is gravel in the gearbox.

# proposal: pack a barge, not a warehouse
git diff --cached --unified=3 > /tmp/barge.diff
git diff --cached --name-only > /tmp/barge.files
wc -c /tmp/barge.diff /tmp/barge.files

If the index is empty, refuse the hop. Empty packs look cheap and teach nothing. Force a local read of git status instead. The model should see intent, not your entire history.

Weigh the barge in the same process. Bytes are facts. Tokens are not facts here. The divide-by-four rule is a ruler drawn on a napkin.

# proposal: napkin token weight, labeled as estimate
from pathlib import Path

MAX_BYTES = 48_000          # policy you choose, not a product limit
MAX_EST_TOKENS = 12_000     # napkin cap, not a billed quota

def weigh(path: Path) -> dict:
    raw = path.read_bytes()
    text = raw.decode("utf-8", errors="replace")
    est = max(1, len(text) // 4)
    return {
        "bytes": len(raw),
        "est_tokens_napkin": est,
        "over_bytes": len(raw) > MAX_BYTES,
        "over_tokens": est > MAX_EST_TOKENS,
    }

CJK and minified JS break this ruler. Treat overflow as LOCAL_ONLY or recut the diff. Recut beats a blind hop. A barge that still sinks should stay in harbor.

Tail latency needs repeated samples. Seven is a start, not a lab. Use HTTPS to a host you already trust. Do not invent a health myth from one lucky fetch.

# proposal: p95 of tiny HTTPS fetches
import time, ssl, statistics
from urllib.request import urlopen, Request

def fetch_ms(url: str, timeout: float = 2.5) -> float:
    req = Request(url, method="GET")
    t0 = time.perf_counter()
    with urlopen(req, timeout=timeout, context=ssl.create_default_context()) as r:
        r.read(256)
    return (time.perf_counter() - t0) * 1000

def p95_ms(url: str, n: int = 7) -> dict:
    samples = []
    errors = 0
    for _ in range(n):
        try:
            samples.append(fetch_ms(url))
        except Exception:
            errors += 1
    if len(samples) < 5:
        return {"ok": False, "errors": errors, "samples": samples}
    samples.sort()
    idx = min(len(samples) - 1, int(round(0.95 * (len(samples) - 1))))
    return {
        "ok": True,
        "p50": statistics.median(samples),
        "p95": samples[idx],
        "errors": errors,
        "n": len(samples),
    }

Policy lives in one function. Keep it readable on a bad train network. Offline is a result, not an exception. Queue the job and walk away.

# proposal: three-gate decision, no hidden I/O
P95_BUDGET_MS = 350  # your budget, not a vendor SLA

def decide(hits, weight, probe) -> str:
    if hits:
        return "LOCAL_ONLY_SECRETS"
    if weight["over_bytes"] or weight["over_tokens"]:
        return "LOCAL_ONLY_PAYLOAD"
    if not probe.get("ok"):
        return "QUEUE_OFFLINE"
    if probe["p95"] > P95_BUDGET_MS:
        return "QUEUE_TAIL"
    return "REMOTE_OK"

REMOTE_OK still does not mean paste the tree. It means the packed barge may leave. Log the decision with hashes. Tomorrow you will not remember the p95.

Persist the queue in SQLite. Files vanish. Rows do not, if you sync. Store the diff path, the napkin weight, and the verdict.

# proposal: local queue, remote is a drain target
import sqlite3, hashlib, time
from pathlib import Path

DDL = """CREATE TABLE IF NOT EXISTS hops (
  id INTEGER PRIMARY KEY,
  created REAL NOT NULL,
  diff_sha TEXT NOT NULL,
  verdict TEXT NOT NULL,
  bytes INTEGER NOT NULL,
  est_tokens INTEGER NOT NULL,
  drained INTEGER NOT NULL DEFAULT 0
);"""

def enqueue(db: Path, diff: Path, verdict: str, weight: dict) -> None:
    sha = hashlib.sha256(diff.read_bytes()).hexdigest()
    con = sqlite3.connect(db)
    con.execute(DDL)
    con.execute(
        "INSERT INTO hops(created, diff_sha, verdict, bytes, est_tokens) VALUES (?,?,?,?,?)",
        (time.time(), sha, verdict, weight["bytes"], weight["est_tokens_napkin"]),
    )
    con.commit()
    con.close()

Drain only REMOTE_OK rows. Leave QUEUE_TAIL for a quieter window. Leave LOCAL_ONLY_SECRETS forever. A free server does not wash a secret. It multiplies the copies.

Think of the link as a loading dock. The dock has a scale and a clock. Overloaded docks queue trucks. They do not flatten the warehouse onto one trailer. Your prompt pipeline should act the same.

A local-first loop still needs a clock. Wall time on the laptop is the scarce good. Battery and heat are cousins of that clock. If the fan climbs, overflow non-secret work. If the lid is shut, do not pretend the GPU is free.

# proposal: optional thermal glance on Linux, skip if absent
if [ -r /sys/class/thermal/thermal_zone0/temp ]; then
  awk '{print $1/1000}' /sys/class/thermal/thermal_zone0/temp
fi

Missing thermal files mean skip the gate. Do not fake a number. Unknown heat is not a remote excuse. It is a missing sensor.

Test the workflow like a brake check. You want refusal, not eloquence. Seed a repo with a fake .env and a tiny staged diff. The verdict must be LOCAL_ONLY_SECRETS. Remove the file. Oversized the diff with a minified vendor blob. The verdict must be LOCAL_ONLY_PAYLOAD. Unplug the network. The verdict must be QUEUE_OFFLINE.

# proposal: brake-check the packer
set -e
mkdir -p /tmp/barge-demo && cd /tmp/barge-demo
git init -q
echo 'secret=nope' > .env
echo 'print("ok")' > app.py
git add app.py
# run secret_hits() against $PWD; expect a hit on .env
# then: rm .env && python -c 'print("x"*80000)' > vendor.js && git add vendor.js
# weigh vendor.js; expect over_bytes or over_tokens

If any brake fails, stop shipping. A packer that cannot refuse is a firehose. Firehoses feel productive until the leak report arrives.

Limitations sit in the open. Name vetoes miss inline keys. Napkin tokens drift on Asian source and on bundled JS. Seven HTTPS samples are a weather vane, not a climate model. SQLite on a laptop is not a cluster queue. Free remote capacity can vanish without notice. Packed diffs hide unread files the model still needs. This workflow will under-inform a large architectural change.

Who should not use it is equally plain. Do not use it on regulated patient or card data. Do not use it as a production SLA. Do not use it when the model version must be pinned by contract. Do not use it to launder secrets through a “temporary” host. Do not use it as a substitute for code review.

Local-first is a failure-domain choice. Your disk and your link fail on different days. A packed barge respects both. Average RTT stories hide that split. Payload mass does not.

Keep the tree. Ship the slice. Log the napkin weight. Read p95. If a free remote is handy and the row says REMOTE_OK, drain one non-secret job and compare the log to your editor’s feel. That comparison is the whole experiment.