Let the Secret Decide Where the Prompt Runs

A pasted config file should never reach a remote model. Not because remote is slow. Because the payload carries keys.

Latency arguments come later. Secret class comes first, and it is a policy call. Decide on what the text contains, then on how fast it must return.

The rule nobody enforces

You paste forty lines of .env into a chat box. The answer is good. The request is also gone.

Most teams handle this with a guideline. Do not paste secrets. Guidelines without a gate decay in about a week.

So build the gate. Three checks, one decision, fail closed. The gate runs before the client library, not inside your head.

The gate is three checks

Check one is pattern matching for known credential shapes. Check two is Shannon entropy over long tokens. Check three is the budget: offline, or local p50 above your limit.

#!/usr/bin/env python3
# egress_gate.py - decide local vs remote before a prompt leaves the machine.
import argparse, json, math, re, sys
from dataclasses import dataclass, asdict

PATTERNS = {
    "aws_key": re.compile(r"AKIA[0-9A-Z]{16}"),
    "github_token": re.compile(r"gh[pousr]_[A-Za-z0-9]{36,}"),
    "private_key": re.compile(r"-----BEGIN [A-Z ]*PRIVATE KEY-----"),
    "slack_token": re.compile(r"xox[baprs]-[A-Za-z0-9-]{10,}"),
    "db_url_with_password": re.compile(
        r"(?i)\b(postgres|mysql|redis|mongodb)://[^\s:@/]+:[^\s:@/]+@"
    ),
}

def shannon(token: str) -> float:
    counts = {}
    for ch in token:
        counts[ch] = counts.get(ch, 0) + 1
    n = len(token)
    return -sum((c / n) * math.log2(c / n) for c in counts.values())

def entropy_hits(text: str, min_len: int = 20, threshold: float = 3.6):
    hits = []
    for tok in re.split(r'''[\s"'`,;(){}\[\]<>]+''', text):
        if len(tok) >= min_len and shannon(tok) >= threshold:
            hits.append(tok)
    return hits

@dataclass
class Decision:
    route: str      # "local" or "remote"
    reason: str
    findings: list

def gate(payload: str, offline: bool, local_p50_ms, budget_ms: float) -> Decision:
    findings = [name for name, pat in PATTERNS.items() if pat.search(payload)]
    hits = entropy_hits(payload)
    if hits:
        findings.append(f"entropy_hits={len(hits)}")
    if findings:
        return Decision("local", "secret_class", findings)
    if offline:
        return Decision("local", "offline")
    if local_p50_ms is None or local_p50_ms > budget_ms:
        return Decision("remote", "over_budget")
    return Decision("local", "fits_budget")

if __name__ == "__main__":
    ap = argparse.ArgumentParser()
    ap.add_argument("payload")
    ap.add_argument("--offline", action="store_true")
    ap.add_argument("--local-p50-ms", type=float, default=None)
    ap.add_argument("--budget-ms", type=float, default=400.0)
    a = ap.parse_args()
    text = open(a.payload, encoding="utf-8").read()
    d = gate(text, a.offline, a.local_p50_ms, a.budget_ms)
    print(json.dumps(asdict(d)))
    sys.exit(0 if d.route == "local" else 10)

The exit code is the policy. Zero means keep it on the machine. Ten means egress is permitted. Wire that number into a pre-commit hook or a client wrapper, and the guideline becomes code.

Entropy is the noisy check. Hashes, UUIDs, and base64 assets all look random. That noise is the price of catching keys you never wrote a regex for.

Measure the false positive rate first

Do not trust the scanner on faith. Run it across a real repository and count what it flags.

mkdir -p fixtures
printf 'AKIAABCDEFGHIJKLMNOP' > fixtures/aws.txt
python egress_gate.py fixtures/aws.txt; echo "exit=$?"

# scan every source file, one JSON line per file
find ~/src/myrepo -type f -name '*.js' -o -name '*.py' -o -name '*.tf' \
  | while read -r f; do
      python egress_gate.py "$f" | sed "s|^{|{\"file\":\"$f\",|"
    done > flagged.jsonl

python - <<'PY'
import json, collections
rows = [json.loads(l) for l in open("flagged.jsonl")]
print(collections.Counter(r["reason"] for r in rows).most_common())
PY

Sort the flagged files by entropy hit count and read the top fifty yourself. Some will be lockfiles and minified bundles. Those belong on a path allowlist, not in the gate's blind spot.

Set your own tolerance. I will not quote a false positive percentage I have not measured on your codebase. The number is repository-specific and moves with every dependency you add.

Latency uses your machine, not mine

A budget check needs a real baseline. Measure warm p50 over twenty runs, then measure cold start separately.

for i in $(seq 1 20); do
  /usr/bin/time -f '%e' python -c "
import your_client, pathlib
your_client.generate(pathlib.Path('fixtures/clean.txt').read_text())" 2>&1 | tail -1
done | sort -n | awk '{a[NR]=$1} END{print "p50_seconds", a[int(NR/2)+1]}'

Warm and cold differ by seconds, not milliseconds, on a laptop. Cold start belongs in the budget or it will surprise you on the first request of the morning.

Pass the measured p50 into --local-p50-ms. Now the gate compares your hardware to your deadline, not a vendor chart.

The decision table

Payload class	Local p50 vs budget	Link	Route
Credential or high entropy	any	any	local
Credential or high entropy	none available	any	local
Clean text	over budget	online	remote
Clean text	within budget	online	local
Clean text	any	offline	local

Read the table top to bottom. The first matching row wins. Secrets never fall through to a performance argument.

When a free server wins

The gate answers a policy question. It does not answer a hardware one. When the gate returns ten, you still need somewhere to send the request.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

MonkeyCode is an open source project that offers free model access and a free server option. The operator states the free tier includes ten million tokens. Treat that as a starting budget, not a permanent guarantee, and confirm current limits on the project page before you depend on them.

That fits one narrow case well. A solo developer, a clean payload, a tight deadline, and no GPU on the desk. The free server removes the hardware question for the first day of experiments.

It fits poorly when the payload is dirty. A free tier does not make an unsafe request safe. Run the gate before you spend a token.

Where this breaks

Encoded secrets slip through. A base64 blob of a private key has high entropy sometimes, and sometimes not. Decode before you scan if your payloads carry encoded files.

Entropy flags minified JavaScript and lockfiles. Expect allowlist maintenance, and keep the allowlist in review.

The gate is a tripwire, not a DLP system. It catches shapes you thought about. It will miss a password pasted as plain English.

Free tokens are finite by construction. A retry loop with no cap burns a budget in an afternoon. Log the decision JSON next to your spend.

Who should not use this

Regulated data should not take this route at all. If your payload needs an audit trail, a regex gate is not the control you are looking for.

Production traffic needs a policy engine, not a script. This gate is for interactive work on a developer machine, where the risk is a paste, not a pipeline.

Teams with a shared secret store already solved the problem upstream. Gate the store, and let this script retire.

Run it on your repo first

Start with the scan step, not the model call. Your repository is the honest test set, and the flagged list is your first deliverable.

Keep the exit code in version control. A gate that lives in a wiki is a suggestion. A gate that returns ten is a route.

If the gate allows egress and your laptop p50 misses the deadline, the free server option is the shortest path to a first measurement. Measure, then decide what stays local.