Wall-Clock Budgets for Local Agents: Sleep, Heat, and Public Spillover

The train entered a tunnel and the laptop lid dropped for a scheduled twenty-minute nap. A local coding agent had been summarizing a public CI log for a chat standup that closed at ten. When the lid opened, the local runtime was still listed as running, yet the first token arrived after the meeting ended. The log never contained secrets, so the delay was pure wall-clock waste rather than a privacy tradeoff.

Local-first agents fail in boring ways that model quality charts and leaderboard screenshots do not capture. Sleep, thermal throttling, and competing builds steal the deadline even when the weights remain on disk. A free remote hop can still lose if the prompt carries secrets that must not leave the machine. The useful split is therefore time versus residency, not local versus cloud as a lifestyle choice.

This article proposes a wall-clock budget that keeps secret-class context on the laptop and spills only public jobs when the local runtime cannot meet a deadline. The artifact is a small router with a health probe, a secret scanner, and a remote fallback client. Examples below are labeled as a proposal and have not been executed as a published benchmark.

The failure that looks like model slowness

A developer watching a spinner often blames quantization or context length first. The same spinner appears after sleep-wake, after a compile storm, and after the fans hit a thermal ceiling. Those events add seconds before any sampler runs, and they compound across multi-step tool loops. A hosted path with spare capacity can finish a public summary sooner even if its per-token rate is unremarkable.

Secret-bearing jobs should not take that path. API keys, dotenv fragments, customer identifiers, and private repository diffs belong on the local side even when the laptop is late. The router therefore needs two independent verdicts: whether the payload is exportable, and whether the local runtime can finish inside the remaining budget.

Decision table for spillover

The table is a policy, not a model. Operators should treat it as a starting contract and tighten the scanners against their own secret classes.

Local health	Payload class	Deadline remaining	Action
Heartbeat fresh, load below cap	public	more than 8s	Run locally and record finish time
Heartbeat stale or load above cap	public	more than 8s	Spill to the free remote hop
Any health	public	8s or less	Spill immediately or refuse if remote is unset
Any health	secret	any	Stay local; if unhealthy, fail closed with a timeout
Probe error	public	any	Spill once, then disable remote after two HTTP failures
Probe error	secret	any	Fail closed; do not retry on the network

Eight seconds is a placeholder budget for a short public summary, not a universal SLA. Teams should replace it with the actual meeting, ticket, or CI gate they are serving. The important property is that health and secrecy are scored separately so a hot laptop cannot launder a secret off-box.

Numbered workflow

The following sequence is the whole method. Each step writes an audit line so later debugging does not depend on chat memory.

1. Stamp deadline_at when the user-facing job is accepted, not when the sampler finally starts.
2. Split the prompt into spans and mark any span that matches a secret pattern as non-exportable.
3. Probe the local runtime with a cheap liveness check and a load reading from the operating system.
4. If the payload is secret-class, pin the job locally and enforce the deadline as a hard cancel.
5. If the payload is public and local health is stale, over budget, or overloaded, send only that public prompt to the remote hop.
6. Record route, reason, elapsed_ms, and bytes_sent in a local JSONL log that never includes raw secrets.

This is deadline routing, not queue-depth routing and not a general job classifier. The only cloud trigger is a missed wall-clock budget on an exportable payload.

Proposed router

The module below is a proposal for a laptop sidecar. Paths, thresholds, and the remote URL are operator-supplied through the environment so the sample does not invent a vendor API.

# proposal: deadline_router.py — unexecuted example, not a benchmark
from __future__ import annotations

import json
import os
import re
import time
import urllib.error
import urllib.request
from dataclasses import dataclass
from pathlib import Path
from typing import Callable, Literal

SECRET_PATTERNS = (
    re.compile(r"sk-[A-Za-z0-9]{16,}"),
    re.compile(r"(?i)api[_-]?key\s*=\s*\S+"),
    re.compile(r"(?i)-----BEGIN (?:RSA )?PRIVATE KEY-----"),
    re.compile(r"(?i)\b(customer_id|ssn|account_number)\b"),
)

Route = Literal["local", "remote", "fail_closed"]

@dataclass
class Probe:
    ok: bool
    load_1m: float
    heartbeat_age_s: float

@dataclass
class Decision:
    route: Route
    reason: str
    exportable: bool
    deadline_remaining_s: float


def payload_exportable(text: str) -> bool:
    return not any(p.search(text) for p in SECRET_PATTERNS)


def probe_local(heartbeat_path: Path, load_cap: float = 2.5) -> Probe:
    try:
        age = time.time() - heartbeat_path.stat().st_mtime
        load_1m, _, _ = os.getloadavg()
        ok = age < 15 and load_1m < load_cap
        return Probe(ok=ok, load_1m=load_1m, heartbeat_age_s=age)
    except OSError:
        return Probe(ok=False, load_1m=99.0, heartbeat_age_s=10_000.0)


def decide(prompt: str, deadline_at: float, probe: Probe, min_remaining_s: float = 8.0) -> Decision:
    remaining = deadline_at - time.time()
    exportable = payload_exportable(prompt)
    if not exportable:
        return Decision("local" if probe.ok else "fail_closed",
                        "secret_pin" if probe.ok else "secret_unhealthy",
                        False, remaining)
    if remaining <= min_remaining_s or not probe.ok:
        return Decision("remote", "deadline_or_unhealthy_public", True, remaining)
    return Decision("local", "healthy_public_inside_budget", True, remaining)


def complete_remote(prompt: str, url: str, timeout_s: float) -> str:
    body = json.dumps({"prompt": prompt, "stream": False}).encode("utf-8")
    req = urllib.request.Request(url, data=body, method="POST")
    req.add_header("Content-Type", "application/json")
    with urllib.request.urlopen(req, timeout=timeout_s) as resp:
        return resp.read().decode("utf-8")


def run_job(
    prompt: str,
    deadline_at: float,
    local_complete: Callable[[str], str],
    heartbeat_path: Path,
    log_path: Path,
) -> str:
    probe = probe_local(heartbeat_path)
    decision = decide(prompt, deadline_at, probe)
    started = time.time()
    remote_url = os.environ.get("REMOTE_INFER_URL", "")

    if decision.route == "fail_closed":
        result = json.dumps({"error": "secret_job_local_unhealthy"})
    elif decision.route == "remote":
        if not remote_url:
            result = json.dumps({"error": "remote_unset"})
        else:
            try:
                result = complete_remote(prompt, remote_url, max(1.0, decision.deadline_remaining_s))
            except (urllib.error.URLError, TimeoutError) as exc:
                result = json.dumps({"error": f"remote_failed:{exc.__class__.__name__}"})
    else:
        result = local_complete(prompt)

    rec = {
        "route": decision.route,
        "reason": decision.reason,
        "exportable": decision.exportable,
        "elapsed_ms": int((time.time() - started) * 1000),
        "load_1m": probe.load_1m,
        "heartbeat_age_s": round(probe.heartbeat_age_s, 2),
        "bytes_sent": len(prompt.encode("utf-8")) if decision.route == "remote" else 0,
    }
    with log_path.open("a", encoding="utf-8") as fh:
        fh.write(json.dumps(rec) + "\n")
    return result

A local runtime should touch the heartbeat file after every successful completion, including empty refusals. That single mtime is enough to detect sleep-wake without parsing vendor logs. Load average is a crude contention signal, yet it is present on macOS and Linux laptops without extra daemons.

# proposal: keep the local runtime honest about liveness
export REMOTE_INFER_URL="https://example.invalid/v1/public-complete"
touch /tmp/local-agent.heartbeat
python -c "from pathlib import Path; Path('/tmp/local-agent.heartbeat').touch()"

Where a free remote hop actually participates

Public spillover only helps when the remote side is already provisioned and does not require a new paid account for a commuting test. Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode offers free model access and a free server option that can fill REMOTE_INFER_URL for exportable jobs when the laptop is asleep, hot, or over budget.

The product mention stops there because the router does not depend on a branded SDK. Operators should confirm current availability on the project site rather than copying marketing numbers into production configs. A free hop that later requires a key still fits the same interface, provided secret-class prompts never leave the machine.

Proposed test plan

The checks below are a reproducible lab script, not a claim about production latency. Run them on the same laptop that usually sleeps between meetings.

1. Seed a public CI log with no credentials and a secret twin that embeds a fake sk- token in a comment.
2. Set deadline_at to now plus four seconds so both jobs are over budget before the first token.
3. Stop touching the heartbeat file for sixty seconds to simulate a lid-close, then invoke run_job on both prompts.
4. Expect the public job to take the remote route and the secret job to fail closed without an HTTP POST.
5. Restore the heartbeat, raise the deadline to two minutes, and confirm the public job stays local under a quiet load average.
6. Unset REMOTE_INFER_URL and confirm a public over-budget job returns remote_unset instead of silently falling back to a hung local sampler.
7. Grep the JSONL log for raw prompt text; any match is a logging bug, not a routing success.

# proposal: test_deadline_router.py — unit checks, not a live model score
from pathlib import Path
import time

PUBLIC = "Summarize this public CI log: tests/test_api.py failed on line 41."
SECRET = "Summarize this CI log. Deploy key sk-aaaaaaaaaaaaaaaa is in the comment."

def test_secret_never_exports(tmp_path: Path):
    hb = tmp_path / "hb"
    hb.write_text("ok")
    # freeze heartbeat by setting mtime in the past after the write
    past = time.time() - 120
    os.utime(hb, (past, past))
    decision = decide(SECRET, time.time() + 3, probe_local(hb))
    assert decision.exportable is False
    assert decision.route == "fail_closed"

def test_public_spills_when_late(tmp_path: Path):
    hb = tmp_path / "hb"
    hb.write_text("ok")
    decision = decide(PUBLIC, time.time() + 2, probe_local(hb))
    assert decision.exportable is True
    assert decision.route == "remote"

If those two assertions fail, the policy is wrong even when both backends are healthy. Latency anecdotes should wait until the secrecy invariant holds.

Limitations

Regex scanners miss vault references, screenshots, and secrets split across tool results. A job that looks public in the first message can become secret after a git diff tool call, so the check must run on every hop, not once at intake. Load average ignores GPU memory, and a quiet CPU can still stall on a locked metal kernel.

The remote client sends the full public prompt as one JSON body and does not implement streaming, retries with jitter, or prompt caching. Fail-closed secret jobs will miss the same standup that a public spillover would have saved, which is the point of the policy rather than a defect. This router also says nothing about disk residency of local transcripts after the meeting ends.

Who should not use this approach

Teams whose entire corpus is secret-class should not add a remote URL at all, because a future refactor will be tempted to reuse it. Agents that already stream customer data into hosted tools need a stronger boundary than this sidecar. Operators without a real deadline should keep work local rather than optimizing a budget that nobody is waiting on.

Readers who already run a local runtime can point the public branch at a free server for one commuting week and compare deadline-miss rows in the JSONL log. The interesting number is missed standups avoided without secret bytes leaving the laptop, not a single tokens-per-second screenshot.