Local Inference Is Not Secret Isolation: A Disk-Residency Audit for Agent Context

A product team moved a coding agent onto laptops so customer tokens would never cross a datacenter boundary. The model stayed local, the tools stayed local, and the security review treated that topology as the whole control. A week later a shared Time Machine volume and an unsynced chat transcript still held live API keys in plaintext, because the runner had written every prompt into a JSONL cache under the user profile. Local inference had never been the leak. The laptop disk had.

This article treats that failure as a residency problem rather than a model-quality problem. It walks through a concrete audit of where agent context actually lands, then a decision table for keeping a prompt on-box versus sending it to a free remote inference path. The workflow is useful even if the remote path is a generic HTTPS endpoint. Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode's free model access and free server option appear later only as one remote path that avoids writing those prompts onto the developer laptop.

The scene that the local-first slogan hides

Local-first agent stacks usually mix three surfaces that security reviews collapse into one word. The weights may sit in a GGUF file. The runtime may bind a loopback port. The conversation store, KV-cache dumps, tool traces, and editor logs still behave like ordinary user documents. Backups, MDM agents, and ~/Library indexers copy those documents without caring that a token once lived only in RAM.

A second confusion is offline behavior. Offline is a latency and availability property. It is not an isolation property. A machine that never reaches the public internet can still replicate secrets through USB disks, shared home directories, and CI artifacts checked in by accident.

A third confusion is tool output. Agents that read .env, cloud credential files, or git-credential helpers often splice those values into the next model turn. Once spliced, the secret is no longer a file permission problem. It is a prompt-log problem.

What this audit is for

The audit answers one operational question: which classes of context may touch laptop disk, and which classes must never be serialized by the local runner. It does not claim that remote inference is safer in every case. Sending a production secret to any third-party server is still a residency choice, just a different one.

Teams that already ban local model caches, or that run agents only inside ephemeral VMs with no home-directory persistence, will find little new here. Teams that treat "the model is local" as a substitute for a secret policy will find the gap immediately.

Artifact: a disk-residency inventory

The following script is a labeled example, not a production scanner. It looks for common local-agent residue: JSONL transcripts, llama.cpp-style logs, .env copies inside prompt dumps, and strings that resemble cloud keys. Operators should point ROOTS at the real cache directories used by their runner.

#!/usr/bin/env python3
"""secret_residency_audit.py — example inventory, not a CVE finder."""
from __future__ import annotations

import argparse
import json
import re
from pathlib import Path

PATTERNS = {
    "aws_access_key": re.compile(r"AKIA[0-9A-Z]{16}"),
    "pem_header": re.compile(r"-----BEGIN (?:RSA |EC |OPENSSH )?PRIVATE KEY-----"),
    "generic_bearer": re.compile(r"(?i)bearer\s+[A-Za-z0-9._\-]{20,}"),
    "env_assignment": re.compile(r"(?m)^(AWS|AZURE|GCP|OPENAI|GITHUB|STRIPE)_[A-Z0-9_]+=\S+"),
}

SKIP_DIRS = {".git", "node_modules", ".venv", "venv", "models", "gguf"}
TEXT_SUFFIX = {".json", ".jsonl", ".log", ".txt", ".md", ".yml", ".yaml", ".toml"}

def iter_files(root: Path):
    for path in root.rglob("*"):
        if not path.is_file():
            continue
        if any(part in SKIP_DIRS for part in path.parts):
            continue
        if path.suffix.lower() not in TEXT_SUFFIX:
            continue
        if path.stat().st_size > 8_000_000:
            continue
        yield path

def scan_file(path: Path) -> list[dict]:
    try:
        text = path.read_text(encoding="utf-8", errors="ignore")
    except OSError:
        return []
    hits = []
    for name, pattern in PATTERNS.items():
        for match in pattern.finditer(text):
            hits.append({
                "path": str(path),
                "kind": name,
                "preview": match.group(0)[:24] + "…",
            })
    return hits

def main() -> None:
    parser = argparse.ArgumentParser()
    parser.add_argument("roots", nargs="+", type=Path)
    parser.add_argument("-o", "--out", type=Path, default=Path("residency-hits.json"))
    args = parser.parse_args()
    findings: list[dict] = []
    for root in args.roots:
        for path in iter_files(root.expanduser()):
            findings.extend(scan_file(path))
    args.out.write_text(json.dumps(findings, indent=2), encoding="utf-8")
    print(f"wrote {len(findings)} hits to {args.out}")

if __name__ == "__main__":
    main()

A typical first pass on a developer workstation looks like the commands below. Paths are examples. Replace them with the transcript directory the agent actually uses.

chmod +x secret_residency_audit.py
python3 secret_residency_audit.py \
  ~/.cache/agent-runner \
  ~/Library/Application\ Support/local-llm \
  ./tmp/tool-traces \
  -o residency-hits.json
python3 -c "import json; h=json.load(open('residency-hits.json')); print(len(h), {x['kind'] for x in h})"

The output is an inventory, not a verdict. Each hit still needs a classification: live secret, already-revoked secret, or a documentation example that only looks like a key.

Numbered workflow: classify, then choose a path

1. Label the secret class before the first model turn

Treat every tool that can read credentials as a context source. Split values into three buckets before the agent starts. Bucket A never leaves the process: short-lived cloud keys that the tool needs, but the model does not. Bucket B may sit in a local prompt cache: public identifiers, repo names, non-sensitive stack traces. Bucket C must not be serialized on a laptop that is backed up or shared: customer tokens, production private keys, and anything covered by a data-processing addendum.

A small policy file keeps that split out of tribal knowledge.

# residency-policy.example.yaml — proposal, not an enforced control
version: 1
bucket_a_process_only:
  - GITHUB_TOKEN
  - AWS_SECRET_ACCESS_KEY
  - CUSTOMER_DB_URL
bucket_b_local_disk_ok:
  - REPO_SLUG
  - TICKET_ID
  - PUBLIC_ERROR_CLASS
bucket_c_no_laptop_prompt_log:
  - CUSTOMER_PII_SNIPPET
  - PROD_PRIVATE_KEY_PEM
redact_before_prompt:
  - pattern: "AKIA[0-9A-Z]{16}"
    replace: "[REDACTED_AWS_ID]"

2. Stop the runner from echoing Bucket A into the next prompt

Most local agents fail here because tool results are concatenated verbatim. The fix is mechanical. The tool executor returns a redacted view to the model and keeps the live secret in a side channel the model never sees. The example below is pseudocode for that split.

# proposal: split tool results before the next completion call
LIVE = {}

def call_tool(name: str, args: dict) -> dict:
    live_result = TOOLS[name](**args)
    LIVE[name] = live_result
    return redact(live_result, policy="residency-policy.example.yaml")

def complete(messages: list[dict], route: str) -> str:
    # route is "local-disk" or "remote-no-local-log"
    return ROUTES[route](messages)

Without that split, every later choice between local and remote inference is theater. The secret is already in the transcript.

3. Decide the inference route with a residency table, not a latency table

Latency still matters, but it is the second column. The first column is whether the prompt, once written, will be copied by something that is not the model runtime.

Prompt class	Laptop backup / shared home	Offline required	Preferred route	Why a free remote server can win
Bucket B, no secrets	Yes or no	Yes	Local runner	Remote adds nothing and can fail offline
Bucket B, no secrets	Yes or no	No	Either	Pick by warm-up cost and payload size
Bucket A, process-only	Irrelevant	Either	No model route	The model should never receive the live value
Bucket C, must not hit laptop logs	Yes	No	Remote inference, local log disabled	The laptop is the untrusted store
Bucket C	Yes	Yes	Do not run the agent	Offline plus no-disk-log is a process-only job, not a chat log
Bucket C	No, ephemeral VM	No	Local inside the VM	Disk dies with the VM; remote expands the trust set

The surprising row is Bucket C on a backed-up laptop. Engineers reach for a local model to keep secrets close, then the backup software publishes those secrets more widely than a short-lived remote completion would have. A free remote server wins in that row only when three conditions hold: the prompt must not be written under the home directory, the secret is already destined for a vendor the team trusts for that class of data, and offline operation is not required.

MonkeyCode's free model access and free server option fit that row as a remote completion path when the operator has already redacted Bucket A and has accepted that Bucket C will leave the laptop. They do not fit the offline row, and they do not replace process-only handling for live keys.

4. Prove the local log stayed empty

After a remote turn, rerun the inventory and assert that new files under the local cache did not grow. The snippet below is a regression check for CI or a pre-commit hook on an internal agent repo.

# test_residency_regression.py — example assertion
import json
from pathlib import Path

BEFORE = Path("fixtures/residency-before.json")
AFTER = Path("residency-hits.json")

def test_no_new_secret_kinds_on_disk():
    before = {(h["path"], h["kind"]) for h in json.loads(BEFORE.read_text())}
    after = {(h["path"], h["kind"]) for h in json.loads(AFTER.read_text())}
    new_hits = after - before
    assert not new_hits, sorted(new_hits)[:20]

Pair that test with a file-size check on the transcript path. A completion that "went remote" but still appended a full prompt locally has not changed residency at all.

stat -f%z ~/.cache/agent-runner/transcripts.jsonl 2>/dev/null || stat -c%s ~/.cache/agent-runner/transcripts.jsonl
# run one Bucket C completion with local logging disabled
stat -f%z ~/.cache/agent-runner/transcripts.jsonl 2>/dev/null || stat -c%s ~/.cache/agent-runner/transcripts.jsonl

If the byte count moves, the route flag is lying. Fix the logger before debating models.

5. Record the trust set in the same place as the route

A remote free server shrinks laptop disk exposure and grows the set of operators who can see the prompt. Write that trade in the runbook next to the command, not in a slide deck. The record can be a four-line header on every agent session.

session: 2026-09-03T15:12Z
route: remote-no-local-log
secret_buckets_in_prompt: C
local_transcript: disabled
offline_ok: false

That header is the whole policy, made inspectable. Future incidents then start from a file, not from a memory of which laptop was "local-first."

Limitations

The regex inventory misses well-formed secrets that do not match the sample patterns, and it flags documentation examples that were never live. It also skips binary KV-cache files, which can still contain prompt fragments. Teams with regulated data should add an allow-list of directories and a real DLP scanner rather than this script.

Remote inference does not erase provider logs, legal process, or misconfiguration on the server side. A free server is still a server. The win described here is only relative to a laptop that is backed up, shared, or enrolled in file indexing. It is not a claim about encryption, retention, uptime, model identity, or capacity.

This approach is the wrong default for air-gapped work, for prompts that contain secrets the remote operator is not allowed to see, and for agents that must function during network partitions. It is also the wrong default when the local runner already uses an encrypted, non-backed-up, single-user ephemeral disk and the threat model is network exfiltration rather than backup leakage.

Who should skip this workflow

Skip it if the agent never interpolates credentials into prompts. Skip it if device policy already blocks home-directory backups and crash dumps. Skip it if the compliance rule is "no third-party inference," because a free remote path cannot satisfy that rule. In those shops the useful work is still redaction of tool results, not a route table.

For everyone else, the practical order is stable. Redact live keys out of the model view. Inventory the disk the runner actually writes. Route Bucket C away from backed-up laptops when offline is not required. Then, if a free remote completion path is acceptable for that bucket, use it as a residency control rather than as a marketing substitute for local-first discipline.

Operators who want to compare a laptop-log-disabled remote path against their current runner can try MonkeyCode's free model access and free server option on a Bucket C fixture that has already been redacted, then rerun the inventory to confirm the local transcript did not grow.