Agent loops fail at the first unplanned hop. The failure is architectural, not model quality. Default remote calls treat every thought as cargo.
That default burns latency on work that never left disk. It also ships secret-shaped text that needed no network. A hop score reverses the order before any client opens a socket.
Think of each agent step as a package at a loading dock. Some packages never leave the building. Some wait for a clean truck and a clear road. The dock does not ship first and audit later.
Three signals decide the dock on every step. Latency budget is the first signal. Secret contact is the second. Offline need is the third. A free remote server wins only when all three stay green.
The assumption that breaks agent loops
Public agent write-ups keep repeating one quiet pattern. The loop assumes a reachable model for every thought. It also assumes tools can wait on that round trip. Both assumptions collapse on a laptop with flaky wifi.
A planner that waits on the WAN feels clever in a demo. The same wait feels broken inside a tight tool cycle. File edits, test runs, and env reads are local events. They should not queue behind a remote token stream.
Treat the rest of this article as a lab proposal. No production traffic was measured for these notes. Copy the scorer, then record timings on your own box.
Inner loops and outer loops need different patience. Tool steps live in the inner loop. Planning steps live in the outer loop. Mixing those budgets is how agents start assuming the network is free.
A kitchen analogy holds better than a cloud slide. The stove still cooks when the delivery van is stuck. Local tools are the stove. Remote plans are the van. You do not halt dinner because a van might be faster on Sundays.
A hop score, not a vibe
Give every step an integer score that starts at three. Subtract one point when a check fails. A score of three may hop. Any lower score stays on the machine.
The three checks map to the three signals above. Probe TCP latency against a budget you own. Scan the payload for secret-shaped tokens. Confirm the process still has a network path. Missing any check keeps the package inside.
Save this as hop_score.py. It is a router, not a benchmark.
#!/usr/bin/env python3
"""Hop scorer for local-first agent loops. Lab proposal, not a benchmark."""
from __future__ import annotations
import argparse
import json
import os
import re
import socket
import time
from dataclasses import asdict, dataclass, field
SECRET_RE = re.compile(
r"(?i)(api[_-]?key|authorization:|bearer\s+\S+|-----BEGIN|"
r"aws_secret|x-api-key|password\s*=)"
)
PLAN_STEPS = {"plan", "decompose", "summarize_public"}
LOCAL_STEPS = {"apply_patch", "run_tests", "read_env", "git_status"}
@dataclass
class HopDecision:
score: int
route: str
reasons: list[str] = field(default_factory=list)
probe_ms: float | None = None
step: str = ""
def probe_latency(host: str, port: int, timeout: float) -> float | None:
start = time.perf_counter()
try:
with socket.create_connection((host, port), timeout=timeout):
return (time.perf_counter() - start) * 1000.0
except OSError:
return None
def score_payload(payload: str, host: str, port: int, budget_ms: float) -> HopDecision:
score = 3
reasons: list[str] = []
probe_ms = probe_latency(host, port, timeout=0.4)
if probe_ms is None:
score -= 1
reasons.append("offline_or_unreachable")
elif probe_ms > budget_ms:
score -= 1
reasons.append(f"latency_{probe_ms:.0f}ms_over_{budget_ms:.0f}ms")
if SECRET_RE.search(payload):
score -= 1
reasons.append("secret_shaped_payload")
if os.getenv("FORCE_LOCAL_HOPS") == "1":
score = 0
reasons.append("forced_local")
route = "remote" if score == 3 else "local"
return HopDecision(score=score, route=route, reasons=reasons, probe_ms=probe_ms)
def route_step(kind: str, decision: HopDecision) -> HopDecision:
if kind in LOCAL_STEPS:
decision.route = "local"
decision.reasons.append("step_pinned_local")
elif kind not in PLAN_STEPS:
decision.route = "local"
decision.reasons.append("step_not_in_plan_allowlist")
decision.step = kind
return decision
def main() -> None:
parser = argparse.ArgumentParser(description="Score one agent hop")
parser.add_argument("--host", required=True)
parser.add_argument("--port", type=int, required=True)
parser.add_argument("--budget-ms", type=float, default=150.0)
parser.add_argument("--payload-file", required=True)
parser.add_argument("--step", default="plan")
args = parser.parse_args()
payload = open(args.payload_file, encoding="utf-8").read()
decision = score_payload(payload, args.host, args.port, args.budget_ms)
decision = route_step(args.step, decision)
print(json.dumps(asdict(decision), sort_keys=True))
if __name__ == "__main__":
main()
The regex is a tripwire, not a vault. Treat a match as stay local and stop. Do not redact and forward in the same breath. Redaction bugs are how secrets still leave the dock.
Remote is allowed only at score three. That is stricter than a soft threshold. One failed check is enough to keep the step local. Local-first means the truck stays parked on doubt.
Commands that make the score honest
A score without a probe is a slogan. Run a cheap TCP check before any HTTP client wakes. Keep the timeout under half a second so a dead hop fails fast.
printf '%s\n' 'summarize public test logs' > /tmp/public_plan.txt
printf '%s\n' 'Authorization: Bearer sk-demo' > /tmp/secret_step.txt
python3 hop_score.py --host 127.0.0.1 --port 8080 --budget-ms 150 \
--step plan --payload-file /tmp/public_plan.txt
python3 hop_score.py --host 127.0.0.1 --port 8080 --budget-ms 150 \
--step read_env --payload-file /tmp/secret_step.txt
FORCE_LOCAL_HOPS=1 python3 hop_score.py --host 127.0.0.1 --port 8080 \
--budget-ms 150 --step plan --payload-file /tmp/public_plan.txt
Point --host at whatever remote hop you already run. Do not hard-code a vendor hostname into the scorer. The router should not know the truck's paint color.
Log every decision as one JSON line. A week of lines beats a single anecdote. You will see which steps never deserved a hop in the first place.
python3 hop_score.py --host 127.0.0.1 --port 8080 --budget-ms 150 \
--step plan --payload-file /tmp/public_plan.txt >> hops.jsonl
Choose budgets by loop, not by hope. Inner tool hops can use 150 ms as a lab default. Outer plan hops can use 800 ms without pretending that is science. Change the numbers after you watch probe_ms for a day.
Four steps, four dock decisions
Walk a short agent through the scorer before wiring a framework. Step one plans over a public README. Step two reads .env. Step three applies a patch. Step four summarizes a public test log.
Step one may hop if the probe is green. Step two stays local because the payload is secret-shaped. Step three stays local because patching is a stove job. Step four may hop again if the log has no credentials.
That split is the architecture. Planning tokens can ride the van. Tool execution stays in the kitchen. Retry and critique stay local if the last probe drifted past budget.
The allow-list in PLAN_STEPS is small on purpose. Agent frameworks grow flags faster than they grow taste. A short list keeps the dock boring, which is the point.
Unknown step kinds fall local. That is not a bug in the router. It is the cost of refusing to assume a network. New step names must earn a place on the van.
When a free server actually wins
Local-first is not local-only. Some steps are large, slow, or idle on a laptop CPU. Planning over a public repository summary is one case. Summarizing a public test log is another case.
Those steps still need a green score. No secret-shaped text in the payload. A stable path. A latency budget that matches the outer loop, not the demo.
Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode offers free model access and a free server option. Those two facts are the only product claims used here. Put the scorer in front of that server the same way you would front any other remote hop.
The server is a truck at the dock. It does not get the package until the score says the road is clear. Free does not change that order. It only changes whether a clean plan step has a place to go.
Offline still wins when the probe returns nothing. A local model process can keep tools moving. The van can wait. The stove should not.
A tiny reproducibility check
Do not trust the scorer without a failing case. Save tests next to the script and run them on every change.
# test_hop_score.py
# run: python3 -m pytest test_hop_score.py -q
from hop_score import route_step, score_payload
def test_secret_payload_stays_local(monkeypatch):
monkeypatch.setattr("hop_score.probe_latency", lambda *a, **k: 40.0)
d = score_payload("Authorization: Bearer sk-demo", "example.test", 443, 150)
assert d.route == "local"
assert "secret_shaped_payload" in d.reasons
def test_slow_probe_stays_local(monkeypatch):
monkeypatch.setattr("hop_score.probe_latency", lambda *a, **k: 400.0)
d = score_payload("summarize public logs", "example.test", 443, 150)
assert d.route == "local"
assert d.score == 2
def test_offline_probe_stays_local(monkeypatch):
monkeypatch.setattr("hop_score.probe_latency", lambda *a, **k: None)
d = score_payload("summarize public logs", "example.test", 443, 150)
assert d.route == "local"
assert "offline_or_unreachable" in d.reasons
def test_clean_plan_may_hop(monkeypatch):
monkeypatch.setattr("hop_score.probe_latency", lambda *a, **k: 55.0)
d = score_payload("summarize public logs", "example.test", 443, 150)
d = route_step("plan", d)
assert d.route == "remote"
assert d.score == 3
def test_apply_patch_never_hops(monkeypatch):
monkeypatch.setattr("hop_score.probe_latency", lambda *a, **k: 20.0)
d = score_payload("apply the patch in memory", "example.test", 443, 150)
d = route_step("apply_patch", d)
assert d.route == "local"
assert "step_pinned_local" in d.reasons
If the secret test ever goes remote, stop the agent. The dock is broken. If apply_patch ever goes remote, the allow-list is broken. Those two asserts are the architecture review in code form.
Limitations
The scorer does not understand your threat model. Regex misses vault paths and custom header names. A TCP probe ignores TLS, auth, and queue time on the far side. A green score can still be a bad idea.
Remote hops still copy tokens off the box. Free does not mean private. Do not send customer source, credentials, or regulated records through a clean score. Offline mode is only as strong as the local process you actually run.
This workflow also adds a decision point on the hot path. That is extra code and extra logs. Teams with one always-on remote model may find the branch noisy. Measure the branch or delete it.
Payload size is out of scope here. A huge public context can still be a bad hop on a metered link. Add a byte cap in your fork if upload cost matters. Do not pretend the current scorer sees that cost.
Who should not use this
Skip the hop score if policy already forbids remote models. Skip it for hard real-time loops under a few milliseconds. Skip it if you cannot run the fixture tests above.
Air-gapped builds should not keep a remote host in the config. Delete the route instead of scoring it. A dead truck at the dock is still a truck, and someone will start it later.
Skip it for agents that must touch production secrets on every step. Those agents need a local-only path with no remote branch. A score that can return remote is the wrong shape.
What to log for a week
Record step kind, score, probe_ms, and route. Count how often remote would have fired on secret-shaped text. Count how often the probe failed while the agent kept talking to itself.
Those two counts are the review. They beat a slide about using models more. If remote never wins a clean plan step, drop the hop. If local never wins a tool step, the split is fiction.
A free server earns its keep on the remaining clean plans. Everything else stays in the building. If you already front a free server, drop this scorer in and keep a week of JSON lines.
Top comments (0)