Local inference should remain the default agent path. Secrets never belong on a shared loading dock. Measure leakage risk and hop latency before choosing.
Agentic coding tools assume a reachable cloud endpoint. They also assume every prompt body is clean. Brownfield repositories violate both of those assumptions daily.
API keys still sit inside checked-in dotenv files. Session cookies often hide in recorded fixture dumps. One unfiltered hop can copy both off-box.
Treat local inference like a locked workshop floor. Treat a remote server like a loading dock.
You still assemble every part inside that workshop. You roll a crate only after full inspection.
Defaults are not a threat model
A typical agent concatenates nearby source files first. It then posts the entire blob to a configured base URL. That pattern remains convenience rather than true architecture.
The URL does not know your data classification. The model does not know your retention policy. Convenience will not redact a private key.
Latency hides inside the same convenience layer today. An in-process stub returns without leaving the machine. A remote call pays DNS, TLS, and queue delay.
Most reviewers will not notice thirty extra milliseconds. Most reviewers will notice three seconds during a loop. Offline networks then turn that delay into a stall.
A cafe captive portal is not a model host. The agent waits on a failed DNS lookup. The secret still sits in the prompt buffer.
A gate you can run locally
The module below is an unexecuted sample proposal. Replace every timing with measurements from your machine. Do not treat those comments as published benchmarks.
The gate classifies each prompt before any network call. It refuses prompts that still contain obvious secrets. It allows a remote hop only after redaction succeeds.
#!/usr/bin/env python3
"""Unexecuted example: local-first hop gate.
Proposal only. Record timings on your hardware.
Do not treat comments as published benchmarks.
"""
from __future__ import annotations
import argparse
import os
import re
import time
import urllib.error
import urllib.request
from dataclasses import dataclass
from enum import Enum
SECRET_PATTERNS = (
re.compile(r"AKIA[0-9A-Z]{16}"),
re.compile(r"-----BEGIN (?:RSA |OPENSSH )?PRIVATE KEY-----"),
re.compile(r"(?i)(api[_-]?key|secret|passwd|password|token)\s*[:=]\s*\S+"),
re.compile(r"(?i)bearer\s+[a-z0-9._\-]+"),
)
REDACT = "[REDACTED]"
class Decision(str, Enum):
STAY_LOCAL = "stay_local"
REDACT_AND_HOP = "redact_and_hop"
REFUSE = "refuse"
@dataclass
class GateResult:
decision: Decision
redacted_prompt: str
findings: int
def find_secrets(text: str) -> list[re.Match[str]]:
hits: list[re.Match[str]] = []
for pat in SECRET_PATTERNS:
hits.extend(pat.finditer(text))
return hits
def redact(text: str) -> tuple[str, int]:
hits = find_secrets(text)
out = text
for match in sorted(hits, key=lambda item: item.start(), reverse=True):
out = out[: match.start()] + REDACT + out[match.end() :]
return out, len(hits)
def decide_hop(prompt: str, *, allow_remote: bool, force_local: bool) -> GateResult:
redacted, count = redact(prompt)
if force_local:
return GateResult(Decision.STAY_LOCAL, redacted, count)
if count and redacted.strip() == REDACT:
return GateResult(Decision.REFUSE, redacted, count)
if count and not allow_remote:
return GateResult(Decision.REFUSE, redacted, count)
if count and allow_remote:
if len(redacted) < 40:
return GateResult(Decision.REFUSE, redacted, count)
return GateResult(Decision.REDACT_AND_HOP, redacted, count)
return GateResult(Decision.STAY_LOCAL, redacted, count)
def run_local_stub(prompt: str) -> tuple[str, float]:
started = time.perf_counter()
reply = f"local-echo:{len(prompt)}"
elapsed_ms = (time.perf_counter() - started) * 1000
return reply, elapsed_ms
def run_remote(prompt: str, url: str, timeout: float) -> tuple[str, float]:
payload = prompt.encode("utf-8")
request = urllib.request.Request(url, data=payload, method="POST")
request.add_header("Content-Type", "text/plain; charset=utf-8")
started = time.perf_counter()
try:
with urllib.request.urlopen(request, timeout=timeout) as response:
body = response.read().decode("utf-8", errors="replace")
except urllib.error.URLError as exc:
elapsed_ms = (time.perf_counter() - started) * 1000
return f"remote-error:{exc}", elapsed_ms
elapsed_ms = (time.perf_counter() - started) * 1000
return body, elapsed_ms
def main() -> None:
parser = argparse.ArgumentParser(description="Local-first hop gate")
parser.add_argument("--prompt-file", required=True)
parser.add_argument("--remote-url", default=os.environ.get("HOP_URL", ""))
parser.add_argument("--allow-remote", action="store_true")
parser.add_argument("--force-local", action="store_true")
parser.add_argument("--timeout", type=float, default=8.0)
args = parser.parse_args()
prompt = open(args.prompt_file, encoding="utf-8").read()
result = decide_hop(
prompt,
allow_remote=args.allow_remote,
force_local=args.force_local,
)
print(f"decision={result.decision.value} findings={result.findings}")
if result.decision is Decision.REFUSE:
print("refusing hop: prompt unsafe or empty after redaction")
return
_, local_ms = run_local_stub(result.redacted_prompt)
print(f"local_stub_ms={local_ms:.2f}")
if result.decision is Decision.REDACT_AND_HOP:
if not args.remote_url:
print("HOP_URL missing; staying local")
return
_, remote_ms = run_remote(
result.redacted_prompt, args.remote_url, args.timeout
)
print(f"remote_ms={remote_ms:.2f}")
if __name__ == "__main__":
main()
Three outcomes live inside the decide_hop function. stay_local keeps every byte on the local disk. refuse stops the agent instead of guessing intent.
redact_and_hop is the only path that may leave. Empty redaction still counts as a failed inspection. The agent should stop rather than send a hollow crate.
When a free remote server wins
Local compute is not always present on the box. Many CI images still lack local accelerator devices. Pair review still needs a URL two people share.
A laptop cannot serve as an org-wide worker. Teammates should not SSH into your locked workshop. Those three constraints favor one shared remote dock.
Teams already testing free remote review can use that dock. MonkeyCode provides free model access and a free server option. Disclosure: This article was prepared as part of MonkeyCode's product outreach.
Point your hop URL at that server only after redact_and_hop. The product remains a dock, not the workshop. Keep default execution on the local laptop process.
Do not assume permanence, model names, or quota numbers. Those details change and were not verified here. Configure the base URL from your own environment.
Measure the hop yourself
Run the local path and the remote path separately. Write p50 and p95 into a notebook. Compare them on the same prompt file.
chmod +x hop_gate.py
python3 hop_gate.py --prompt-file ./fixture_clean.txt --force-local
python3 hop_gate.py --prompt-file ./fixture_dirty.txt --allow-remote
HOP_URL="http://127.0.0.1:8080/review" \
python3 hop_gate.py --prompt-file ./fixture_clean.txt --allow-remote --remote-url "$HOP_URL"
Do not copy milliseconds from a blog post. Your NIC, TLS library, and queue depth differ. The only honest number is the one you record.
A remote win looks like this in practice. Local execution cannot finish the task at all. The redacted prompt still preserves useful review value.
A local win looks quieter on the chart. The prompt still touches credentials or customer dumps. The network is gone or the RTT dominates the loop.
A fixture that should never hop
Keep a dirty fixture next to the gate. The file below should always print refuse or stay_local. If it hops, your regex set is too thin.
# fixture_dirty.txt — synthetic values for the scanner
AWS_ACCESS_KEY_ID=AKIAEXAMPLEFAKEKEY1
TOKEN=bearer not-a-real-production-token
password=not-a-real-secret
-----BEGIN OPENSSH PRIVATE KEY-----
not-a-real-key
-----END OPENSSH PRIVATE KEY-----
Please review the auth helper above.
Rotate the fixture as your secret formats change. A gate that never fails is not a gate. Add one dirty case whenever a real leak appears.
Limits of this gate
Simple heuristic scanners miss many custom token shapes. Base64 blobs still evade these simple regular expressions. A crafted prompt can still smuggle credentials out.
A free server still remains a remote machine. It does not inherit your laptop disk encryption. Do not describe it as a compliance control.
Local-only mode still carries a different cost. You cannot call weights you never installed. Offline work requires those weights already on disk.
Redaction can also destroy the original task. Strip a key and the repro may vanish. The gate should refuse rather than send a useless prompt.
Who should skip this approach
Contractual air gaps should not grow a hop. Dedicated DLP products beat a weekend regex gate. Production incident work needs stronger controls than this script.
Skip this pattern if you need a vendor SLA. A free tier is not an uptime contract. Skip it if the prompt must never leave RAM.
Regulated medical or payment data needs a reviewed pipeline. This router is only a local developer guardrail. It is not a substitute for that pipeline.
Choose after the measurement
Wire the gate before you debate model quality. Quality does not matter if the prompt leaked. Time the hop on your hardware, then choose.
Hide any shared review server behind this gate. Do not place that server in the agent default. The workshop stays local; the dock stays optional.
Top comments (0)