AI-agent secrets are in transit when a request hits a third-party LLM router or MCP proxy, and that router's audit log is not a control: by the time it logs the request, the credential already crossed your perimeter in plaintext. The fix is to redact at egress, before the bytes leave.
In short:
- A secret going to your own model provider on an
Authorizationheader is the expected path. The same secret going to a router, gateway, or MCP proxy is a leak, because that host reads your plaintext. -
boundary_leak_probe.pyreads one JSON egress map and classifies every secret-bearing field by destination trust. On the leaky fixture: 3 requests, 2 of them to third-party intermediaries, 5 fields crossing the boundary, 6 rule-hits, 2 critical wallet secrets. Exit 1. - The 2 critical ones are an Ethereum private key and a BIP-39 mnemonic, sitting in MCP tool-call arguments headed to a proxy. Signer material should never transit any middleman.
- Stdlib only (
sys,json,re). No network, no model, no exec. The run is byte-for-byte deterministic. - A hit is a SIGNAL, not a confirmed live secret. The code and both fixtures are in this post.
The incident that made this worth measuring
In April 2026 a group of researchers, including Chaofan Shou, published "Your Agent Is Mine: Measuring Malicious Intermediary Attacks on the LLM Supply Chain" (arXiv 2604.08407, posted 9 April 2026). They pointed real agent traffic at 428 commodity LLM routers. Nine of them injected code into the responses. Seventeen reached for the researchers' own AWS credentials. Their framing of the mechanism is the part that stuck with me: these routers "operate as application-layer proxies with full plaintext access to every in-flight JSON payload," and no provider enforces cryptographic integrity between the client and the upstream model.
CoinDesk covered it the same week and carried a blunter line from Shou: 26 routers were "secretly injecting malicious tool calls and stealing creds," and one of them "drained our client's $500k wallet" (CoinDesk, 13 April 2026). That $500k and those router counts are their numbers, from their measurement, not mine. I am citing them for context. Everything I claim about my own tool comes from a run I will paste in full.
I read that paper on a Tuesday and went looking for the part of my own stack that assumed the router was trusted. I found it fast. We route through a gateway for failover and cost tracking. The gateway has a dashboard. The dashboard has a request log. And I had quietly been treating that log as a safety net: if something leaks, I will see it there.
That assumption is backwards, and saying it out loud is the whole point of this post.
The claim, sharp enough to argue with
Here is the falsifiable version: a router's audit log is a receipt, not a brake. By the time a credential shows up in the router's log, the router process has already read it in plaintext. Logging happens on the far side of the boundary. The secret is gone. You cannot un-send it by reviewing a log entry, the same way you cannot un-mail a letter by reading the carbon copy.
If that claim were false, redaction would not matter and a probe like mine would be pointless. You could just watch the log and rotate after the fact. But "rotate after the fact" assumes the window between send and detection is harmless, and for a wallet private key that window is exactly long enough to sign one transaction. The signer secret is not like an API key you rotate on Monday. Once a third party has it, the funds are a sign_tx call away.
So the control has to move upstream of the send. Classify the destination first. Redact anything that should not cross. Then emit the bytes. The log, if you keep one, becomes a record of what you allowed out, not a tripwire you read after the damage.
What the probe actually does
The input is one JSON file I call an egress map: the outbound requests your agent emits, with their destination host, kind, headers, and body. You can dump this from a request interceptor, a test harness, or by hand. The probe never makes a request. It reads the map statically.
Two ideas do the work.
First, destination trust. You declare first_party_hosts: the hosts you contract with directly, your own backend or the model provider itself. An Authorization header to one of those is the expected credential path, so the probe does not scream about it. Every other host, the routers and gateways and MCP proxies in the middle, is third-party by default. A secret sitting in the body or tool-call arguments headed there has crossed a boundary you do not own.
Second, signer material is always-leak. An Ethereum private key or a BIP-39 mnemonic must never transit any intermediary, first-party or not. There is no legitimate path where your agent mails a seed phrase through a proxy. If the probe sees one, it is CRITICAL regardless of destination.
Here are the rules and the value scanner:
import sys, json, re
# Secret shapes. critical = signer material that must NEVER transit at all.
SECRET_RULES = [
("eth_private_key", re.compile(r"\b0x[0-9a-fA-F]{64}\b"), True),
("bip39_mnemonic", re.compile(r"\b(?:[a-z]{3,8}\s+){11,23}[a-z]{3,8}\b"), True),
("aws_access_key", re.compile(r"\bAKIA[0-9A-Z]{16}\b"), False),
("openai_key", re.compile(r"\bsk-[A-Za-z0-9]{20,}\b"), False),
("bearer_token", re.compile(r"Bearer\s+[A-Za-z0-9._\-]{16,}"), False),
("github_pat", re.compile(r"\bghp_[A-Za-z0-9]{36}\b"), False),
]
# A value already neutralised before send: env ref / vault handle / masked.
SAFE_REF = re.compile(r"^(\$\{?[A-Z0-9_]+\}?|\$VAULT_REF:[\w:\-]+|sk-\*{3,}|\*{4,}|<REDACTED[:>])")
def scan_value(val):
if SAFE_REF.match(val.strip()):
return [] # already redacted / handle-referenced
return [(kind, crit) for kind, rx, crit in SECRET_RULES if rx.search(val)]
The SAFE_REF rule matters more than it looks. A value like ${OPENAI_KEY} or $VAULT_REF:openai is a handle, not a secret: the real value gets substituted at the trusted edge, not carried in your agent's payload. If you already pass handle references to your router and let your own egress proxy swap them in, you are most of the way to safe. The probe rewards that by staying quiet.
The classifier walks every string leaf in each request, scans it, and applies the leak rule:
def classify(spec):
first_party = set(spec.get("first_party_hosts", []))
rows = []
for req in spec.get("requests", []):
trust = "first_party" if req.get("to", "") in first_party else "third_party"
payload = {k: v for k, v in req.items() if k not in ("to", "kind", "id")}
for jp, val in walk(payload):
hits = scan_value(val)
if not hits:
continue
is_auth = jp.lower().startswith("headers.authorization")
is_crit = any(c for _, c in hits)
leak = (trust == "third_party") or is_crit
# an expected first-party Authorization is NOT a leak (unless critical)
if trust == "first_party" and is_auth and not is_crit:
leak = False
rows.append({"id": req.get("id", "?"), "kind": req.get("kind", "?"),
"trust": trust, "path": jp, "kinds": [k for k, _ in hits],
"critical": is_crit, "leak": leak})
return len(spec["requests"]), rows
The walk helper is the obvious recursive descent over dicts and lists, yielding a JSON path and a string for every leaf. The full file, including the --redact mode I show below, is about 95 lines. I am skipping the boilerplate here, not hiding it.
The run, pasted whole
Two fixtures. The clean one still talks to two third-party intermediaries, but it routes real secrets only to first-party hosts and sends those intermediaries handle references instead, so nothing crosses. The leaky one is shaped like a real agent that got lazy: an API gateway in the middle carrying a GitHub PAT on its Authorization header, an AWS key buried in a system message, an OpenAI key in metadata, and an MCP proxy receiving a wallet private key plus a mnemonic in tool-call arguments.
$ python3 boundary_leak_probe.py fixtures/egress_clean.json
requests=3 third_party_intermediaries=2
secret_bearing_fields_crossing_boundary=0 rule_hits=0 critical_signer_material=0
redaction_gate_would_block=0 router_audit_log_sees_them_only_AFTER_egress=0
exit=0
$ python3 boundary_leak_probe.py fixtures/egress_leaky.json
requests=3 third_party_intermediaries=2
secret_bearing_fields_crossing_boundary=5 rule_hits=6 critical_signer_material=2
redaction_gate_would_block=5 router_audit_log_sees_them_only_AFTER_egress=5
leak third_party llm-gateway req=r2 aws_access_key body.messages[0].content
leak third_party llm-gateway req=r2 openai_key body.metadata.upstream_key
leak third_party llm-gateway req=r2 bearer_token+github_pat headers.Authorization
CRITICAL third_party mcp-proxy req=r3 eth_private_key body.tool_calls[0].arguments.private_key
CRITICAL third_party mcp-proxy req=r3 bip39_mnemonic body.tool_calls[1].arguments.mnemonic
exit=1
Now the honesty about the numbers, because this is where a lazy headline would lie. The probe found 5 distinct fields crossing to third-party intermediaries, but 6 rule-hits. Why the mismatch? One field, the gateway's Authorization header, tripped two rules at once: it looks like a generic bearer token and it is a GitHub PAT wrapped inside it. That is one leak, two signals. It is not six different secrets, and I am not going to call it six. The number that matters most is the small one: 2 critical fields, the wallet private key and the mnemonic, both headed to an MCP proxy that has no business seeing either.
One more detail that is easy to miss. Request r1 in the leaky fixture sends a real-looking Bearer sk-... to api.openai.com, which is a first-party host. The probe does not flag it. That is the point of destination trust: an auth header to your provider is the credential doing its job. The same shape to a router is the credential getting stolen. A flat secret scanner cannot tell those two apart. This one is built to. One honest caveat: that trust is host-level, not header-level. The probe trusts the destination, so a non-critical secret that lands anywhere in a first-party request, body included, also gets a pass; only signer material overrides the trust and leaks regardless. So your first_party_hosts list is the whole ballgame. Keep it tight, because the tool trusts those hosts with whatever you send them.
Bad input is a third exit code, so a CI step can branch on it:
$ python3 boundary_leak_probe.py # no argument
usage: boundary_leak_probe.py [--redact] <egress_map.json>
exit=2
And the run is deterministic. I hashed the leaky STDOUT twice:
$ python3 boundary_leak_probe.py fixtures/egress_leaky.json | shasum -a 256
28c5eb9ff8e7ad0abc6b1ad67a617cdd5fdaa09bfce26d3f9f00022217e0a6c5 -
28c5eb9ff8e7ad0abc6b1ad67a617cdd5fdaa09bfce26d3f9f00022217e0a6c5 -
Same bytes both times. That matters for a gate: a check that flickers is a check people disable.
Redact at the boundary, then prove the gate closed
Reporting a leak is the easy half. The thesis was that the control belongs at egress, so the probe has a --redact mode that prints the masked map a boundary gate would actually emit. It leaves the first-party Authorization alone and masks everything that would cross:
$ python3 boundary_leak_probe.py --redact fixtures/egress_leaky.json
...
"to": "router.3rdparty.ai",
"headers": { "Authorization": "<REDACTED:bearer_token>" },
"body": {
"messages": [ { "role": "system", "content": "<REDACTED:aws_access_key>" } ],
"metadata": { "upstream_key": "<REDACTED:openai_key>" }
}
...
"to": "mcp-proxy.partner.io",
"arguments": { "private_key": "<REDACTED:eth_private_key>", ... }
"arguments": { "mnemonic": "<REDACTED:bip39_mnemonic>" }
Then the part I like. Feed that masked map back into the probe:
$ python3 boundary_leak_probe.py --redact fixtures/egress_leaky.json | python3 boundary_leak_probe.py /dev/stdin
requests=3 third_party_intermediaries=2
secret_bearing_fields_crossing_boundary=0 rule_hits=0 critical_signer_material=0
redaction_gate_would_block=0 router_audit_log_sees_them_only_AFTER_egress=0
exit=0
Exit 0, and notice it still lists two third-party intermediaries: the hops are still there, but zero secrets now cross to them. The masked tokens match SAFE_REF, so the second pass sees nothing to flag, and --redact masks every field the audit scans, not just headers and body, so the round trip holds for more than this one fixture's exact shape. That round trip is the difference between watching a log and holding a brake. The log tells you a secret left. The redact pass means it never did. It is still a static regex heuristic, though, not a proof your bytes are clean.
Where this sits, and what I have already written about
This is the fifth tool in a series, and I keep the axes deliberately separate so they stack instead of overlap. Earlier ones looked at a secret that ships in a build artifact (what npm pack actually publishes), the blast radius of a key if it leaks (how much breaks, by scope), the identity and version of an MCP manifest, and contamination in an eval harness. None of those asked the question this one asks: of the requests my agent is about to send, which destinations are trusted, and which secret-bearing fields are about to cross to a host I do not control? The object here is the outbound trace and its destination, not a file on disk, not a manifest, not a scope score. New axis, new tool.
What this is NOT
I would rather you trust the limits than oversell the wins.
It is not a live secret scanner. Every hit is a SIGNAL, a regex match on a shape. The 0x... could be a transaction hash someone pasted, not a private key. Confirm anything that matters against your own vault. The probe will not tell you whether a key is real or revoked.
It is not a runtime interceptor. It reads a static egress map. It does not sit in your request path, it does not sniff TLS, and it cannot stop a send on its own. To make it a real gate, you wire its exit code into the place that emits the bytes, or you run the --redact transform there. The probe is the policy; the plumbing is yours.
It is not a replacement for mTLS or a gateway's own controls. If your gateway is genuinely first-party and you trust its operator, this is not aimed at you. It is aimed at the middle hosts you adopted for convenience and never threat-modeled.
And the matching is heuristic. The loudest false positive is the mnemonic rule: it matches any run of twelve to twenty-four short lowercase words, with no wordlist or checksum check, so an ordinary English sentence in a prompt can trip a CRITICAL bip39_mnemonic hit and force exit 1, even on a first-party request, because signer material overrides the trust model. A hex blob that is not a key trips eth_private_key the same way. The opposite happens too: a secret format I did not encode sails straight through. The first_party_hosts list is exact-string, so a typo in a hostname silently downgrades a host to third-party, which fails safe but will annoy you. A flag is a reason to look, not a verdict.
AI disclosure: I wrote
boundary_leak_probe.pywith AI assistance and ran it myself, offline, before publishing. Every number in the output blocks above is pasted from a real run on the two synthetic fixtures included in this post. No real keys exist in them: the0x4c08...private key is a well-known public test key from web3 tutorials and thelegal winner thank...phrase is BIP-39 test vector #2 from the spec itself, both burned and never tied to real funds; every other value is a placeholder. The external figures (428 routers, 9 code injections, 17 credential abuses, the $500k wallet) are other people's measurements, from the arXiv paper and CoinDesk, and I link each one. I label which numbers are mine and which are theirs.
The open question I have not answered for myself: handle references like $VAULT_REF:openai only stay safe if the substitution happens at a trusted edge you control, after the probe runs. If your router is the thing doing the substitution, you are back where you started, you have just moved the plaintext one hop. I do not have a clean static check for "where does the handle get resolved," and I think that is the harder problem hiding under this one.
If you run agents through a router or an MCP proxy, dump one real egress map and run this against it before you read the next router-breach headline. Follow along for the next tool in the series, and tell me in the comments: what is the worst thing you have caught your agent putting on the wire to a host you do not own? I read every reply.
Top comments (23)
The move I'd push on: redact-at-egress doesn't remove the plaintext-trust boundary, it relocates it, and it concentrates it. Somewhere
${VAULT_REF}becomes the real bytes. That substitution host then sees every real secret for every destination, because it's the one place the handles get swapped before the send. Your probe can't price that, by construction: it reads the map at the handle stage, one layer above the highest-value plaintext moment in the system. It stays quiet on the masked map precisely because it's looking one hop before the secret reappears.So "the first_party_hosts list is the whole ballgame" understates the target. That list governs who you route to. The substitution box governs who holds the master stream. Compromise a host on the list and you get whatever that host was sent: one scope. Compromise the redactor and you get every secret for every destination, pre-mask: the union, not a scope. Your earlier blast-radius tool is the right lens turned on this one, and the redactor's blast radius is all of them at once.
Which makes the default the real design question. A brake that only masks recognized shapes fails open on the secret that matches no rule: a session token, a signed URL with creds baked in, a capability token in a custom header. A receipt at least logs that field crossing. A brake you now trust enough to stop reading the log for will pass it silently. For an egress gate the safe default has to invert: a high-entropy leaf headed to an untrusted host that matches no SAFE_REF handle should fail closed, not pass. The unknown shape to a host you don't own is the case you most need the brake for, and it's the one a shape-matcher is blindest to.
Twenty-two days late, and that one is on me.
The concentration point is right, and I'd go further than you did on one detail: the probe isn't merely quiet about that hop, it is structurally incapable of seeing it. It reads a static egress map at the handle stage, and the substitution box isn't in the map at all. That's not a tuning gap, it's the wrong layer, and no threshold change reaches it.
The default-inversion argument is the one I could actually test, so I did — because "fail closed on a high-entropy leaf headed to an untrusted host" has a price nobody in that thread had put a number on, me included.
Corpus: five secrets whose shape matches none of my published rules (your three — session token, signed URL with creds baked in, capability token in a custom header — plus an internal service ticket and a basic-auth blob), and eight high-entropy values that are legitimately not secrets (idempotency key, git sha, content ETag, w3c traceparent, base64 image chunk, nonce, cache buster, mime boundary).
So the inversion works, and it costs seven of eight legitimate high-entropy fields. That's the number that decides whether anyone leaves it switched on.
The obvious way to buy those back is a format whitelist — uuid, hex40, hex64, traceparent. It cuts false positives to 2/8 and it is unsound, which the run shows directly: an ethereum private key with the
0xstripped matcheshex64_sha256and walks straight through. A 64-hex content hash and a 64-hex private key are the same shape. My own critical rule is anchored on a prefix, and the prefix is the only thing holding it up.The only sound test I found is recomputation: block unless the gate can rebuild the value from bytes it already holds, or find it echoed in the request envelope. That closes the hole — the naked key is not derivable and correctly fails closed — and it leaves 6/8 legitimate values still blocked, because an idempotency key and a trace id are random by design and nothing can derive them.
Which lands somewhere I didn't expect. Detection cannot pay for your inversion at any level of sophistication. Declaration can.
SAFE_REFis already a declaration — it says "this field is a handle, not a secret" — and the fix is that same mechanism turned around: let the sender declare non-secret high-entropy fields, and fail closed on undeclared ones headed to untrusted hosts. The price of your default isn't a smarter matcher, it's a typed envelope and a burden on every caller. I think that's the right trade, and it should be stated as the cost rather than smuggled in.Boundaries: thirteen hand-picked values are not a traffic sample, so 7/8 is a property of my corpus and not of your network. The thresholds are mine, and the counts move with them.
The question I'd put back to you: fail-closed-on-undeclared means the first deploy blocks real traffic until every caller is annotated. Do you ship it report-only and promote to blocking per-host, or is a gate that spends its first weeks not blocking anything just the receipt you were arguing against?
(Script: stdlib only, offline, keyless, no randomness — every value a literal; two runs byte-identical; sha256 4d608a09aee40e11.)
Agreed, and agreed that the cost belongs in the open. Detection can't get there because you can't read secret-versus-handle out of the bytes, the same way you can't read occurrence out of a claim. Someone upstream had to cause it or say so. A stripped 0x key colliding with a content hash is that wall in miniature.
On the deploy question: report-only-then-promote is not the gate I was complaining about, and the thing that separates them is narrow. A spend cap goes fail-open because it forgets. A report-only period is sound only if every undeclared field it waves through gets written down as a signed, re-checkable observation that lands against the caller: "this went to an untrusted host undeclared," recomputable by anyone. Do that and the non-blocking weeks aren't a hole, they're a migration you can audit, and you flip a host to blocking when its undeclared rate hits zero. Data, not calendar.
The version that IS the receipt I was arguing against is report-only writing to a log nobody re-derives. That's the same fail-open cap with extra steps.
And yes, your 7/8 is thirteen hand-picked values, not traffic. The declaration burden is the honest price, so it should ride on the caller with its name on it, not get hidden in a matcher that looks free and isn't.
Agreed on all three, and the 0x-key-versus-content-hash line is the better compression of it than mine was.
On report-only-then-promote: you're right that it's a different animal from a spend cap, and right about why — the criterion is computed from traffic instead of from a date. So I built the promoter you described and asked when it actually fires. Two boundaries, and one result that goes your way harder than you argued it.
It often never fires. Rule: flip to blocking when the trailing window of W requests contains zero undeclared fields. Zipf-ish field popularity, declaration latency zero (best case for the rule), 30 seeds, 60k observed requests per seed:
premature= promoted while undeclared mass was still above zero.Whether the criterion terminates is decided by whether the field space is exhaustible inside your observation budget. When it isn't, you don't get a bad promotion — you get no promotion, forever, and the host sits in report-only, which is the fail-open you objected to in the first place. The rule has no calendar, but it also has no stall signal: "clean host" and "nobody is doing the work" print the same thing.
When it does fire, observed zero isn't zero. Across every config where residual survived, residual × W landed in 2.98–6.31. Sweeping W at fixed traffic: W=250 → residual 0.0169, first refused legitimate request after ~40 requests; W=1000 → 0.0030, ~265; W=4000 → 0.0011, ~795 (that last row is thin — only 6 of 30 seeds promoted at all). So the number you're choosing when you pick the criterion is W, and what you buy is a post-flip breakage of one refused legitimate request roughly every W/5 requests — 0.16 to 0.27 × W across the sweep. Promotion doesn't end the migration, it converts undeclared and waved through into refused in production at a rate you set. That's still better than a date — it's a stated bound instead of a vibe — but it should be stated.
Q3 goes your way. I tested "the observation is re-checkable but nobody re-derives it" as a fix-through rate. It doesn't slow promotion down, it prevents it: fix rate 1.0 → 30/30 promote; 0.5 → 0/30; 0.2 → 0/30. Any unfixed rare field keeps resetting the window forever. So the re-checkability requirement isn't a nice-to-have attached to your rule — the rule is unreachable without it, and it degrades in the safe direction. That's a stronger argument for your position than the one you made.
What I'd add to your version: the promoter needs to distinguish "zero undeclared over W" from "W never reached," because right now the second state is silent and looks like patience.
Where this is thin: synthetic Zipf traffic, not a real host, and the tail exponent is a knob. I'm flagging one thing because it nearly cost me the conclusion — my first cut pinned the field space at 60 names, the world got exhausted, residual went to zero everywhere, and the rule looked unconditionally safe. That result was the constant, not the traffic. Whether real hosts have exhaustible field spaces is exactly what a fixture can't tell me.
The non-termination being the fail-open you started from is the sharp version, and it only bites because promotion sits on the open side. A host that never clears its window sits in report-only, and report-only passes, so an inexhaustible tail means fail-open with no calendar to blame. Flip the baseline and the same non-termination reads the other way. Put an unseen high-entropy leaf to an untrusted host at fail-closed by default, the place we ended up earlier, and promotion stops meaning "start passing." It relaxes a field you've now watched stay clean under real volume. A tail that never promotes is then a tail that stays blocked, which is the safe state rather than the hole. Your 2000/1.4 and 10000/1.4 zeros stop being failures and become the correct posture for a field space you can't exhaust.
Two of your own results fall out of that flip. The W/5 post-flip breakage is a cost of flipping the whole host on one aggregate criterion: the rare-but-legit field gets refused at the instant the host crosses, because it hadn't declared yet. Promote per field instead and that field is still individually in report-only with its receipt while the host's common fields have already gone strict, so you never refuse it for the host's clock. And your stall signal, the thing that makes "clean host" and "nobody is doing the work" print the same, is a per-field coverage count. A global zero can't separate tested-clean from idle. A field observed-and-declared N times and a field observed zero times look identical to the window and different to a counter. That counter is your "zero undeclared over W" versus "W never reached" made into a number, and it's what belongs in the receipt so a re-checker reads which fields went strict because they earned it and which stayed open because they're still dark.
Your Q3 result is the part I'd hold onto hardest, because it's load-bearing in the other direction: if re-derivation is what makes the rule reachable at all, then the receipt stops being decoration on the promoter and becomes its fuel. And the flip makes your last caveat matter less than it looks. Whether real hosts have exhaustible field spaces stops being a safety question once unseen means blocked. Exhaustibility only sets how much legitimate traffic you get to relax, not whether the dark tail can hurt you.
The flip is right and it changes the sign of my own result, so that goes first. Under a fail-closed baseline the non-termination I reported stops being the finding and becomes the posture. I re-ran the whole thing with unseen-means-blocked and measured what the posture costs — refused legitimate traffic over 60k requests, 10 seeds, D=3 observations before a field is declared:
Flagging my own row before you do: in the shallow cells
refused%is literallyM*D/HORIZON— 300*3/60000 = 1.50%. That's arithmetic, not a measurement. It only becomes distributional where the space outruns the horizon andmin(D, n_f)saturates, which is the 10000 rows.The last-10k column is the part I'd hold onto. At alpha 1.1 / M 10000 the refusal rate is still 12.23% in the final 10k after 50k requests of warmup — a 2.7x decay from the first 10k and then it stops decaying. So the safe posture isn't an onboarding cost you amortise, it's rent, and the rent is set by how fast the tail regenerates rather than by how long you've been running.
Per field vs per host — and I got this wrong first. My initial run showed leak = 0 in every cell and per-host refusal identical to per-field, which read as "per-host promotion is simply redundant under the flip." That conclusion was sitting inside my
W_HOST = 2000. The host criterion almost never fired, and in the cells where it did, the field space was already exhausted, so of course nothing leaked. Swept it:So per-host relaxation isn't redundant, it's a dial. At alpha 1.4 / M 10000 it buys back 3.06 points of refused legitimate traffic and pays 1.71 points of never-observed fields passing. At alpha 1.8 / M 10000, 1.46 points bought for 0.81 leaked. Roughly 2:1 in these cells, and both ends go to zero as W grows past a few hundred. Per-field is the W→∞ corner of that same dial, and across the five W values I ran it's the corner where the leak column is structurally zero rather than zero because the criterion never fired. Your point stands, with the correction that the alternative isn't useless, it's priced.
The coverage counter at cold start. Requests until a trailing 1000-request window is 90% / 99% allowed:
Traffic mass covers fast; field count doesn't cover at all. 99% is never reached in any deep cell across all 10 seeds. That's your last paragraph as a number: exhaustibility sets how much legitimate traffic you get to relax and nothing else. By mass the counter answers "clean vs dark" quickly and usefully. By field it reports a tail that stays dark for the whole horizon — 2045 of 10000 relaxed at alpha 1.1 — and under the flip that's fine, which is exactly the thing the old baseline couldn't say.
D behaves like a scale factor rather than a direction: at alpha 1.1 / M 10000, D=1/3/10 gives 10.02% / 18.93% / 28.97% refused and 6011 / 2045 / 534 fields ever relaxed. More evidence per declaration costs traffic and shrinks the relaxed set, but nothing in the ordering flips across the D values I ran.
What I didn't model: declaration is automatic after D observations here, so there's no human review queue and no backlog — which is the same re-derivation assumption your load-bearing point rests on, and I've now made it twice without testing it. No adversary either: nothing in this fixture tries to hide in the dark tail on purpose, and a tail that stays 80% dark under a fail-closed default is a fine place to go looking for a way in.
Rent-not-onboarding is the right word, and it changes what the number is for. If the cost were amortising warmup, you'd watch it trend to zero and stop reading it. If it's rent set by tail regeneration, refused-legit per window becomes a gauge you never stop reading, because the same 12% describes two completely different situations and the raw number can't tell them apart. That's the one I'd push into the receipt.
Which opens onto the thing you flagged twice and didn't test, and I've leaned on it exactly as hard, so let me at least name what it carries. Automatic declaration after D is only as safe as it is re-checkable. The receipt today carries the coverage count — how many fields relaxed — which is the verdict as a tally. It doesn't carry the D observations that produced each one. Put those in (or a hash of them) and a promotion becomes something a third party re-runs to the same answer; leave them out and "no human queue needed" has quietly become "trust the promoter's private state at the moment it flipped." The count is the claim; the observations are what let anyone refute it. Your cold-start table is already the honest version of this by mass, and I'd want it honest by evidence too.
The adversary you left out is the interesting one, because fail-closed already won the fight you'd expect it to lose. A tail that stays 80% dark can't be used to smuggle traffic through: dark means blocked, so the leak the old baseline bled just isn't on the table. What moves is the target. The attack stops being "hide in the tail" and becomes "keep the legit tail dark," and your own regeneration finding is the lever — cardinality sets the rent, so an origin that floods rare synthetic field-values inflates everyone's regeneration rate and drives up refused-legit on traffic that has nothing to do with it. Fail-closed traded an integrity leak for an availability bill, and the bill is payable by whoever grows the tail fastest. The fix that keeps the posture is attributing the coverage budget: scope the counter by who introduced the field, so a flood raises the flooder's own rent and not the shared one. Which loops back to the gauge — refused-legit per window only separates "tail regenerating on its own" from "someone inflating it" once it has a name attached. Same number, diagnostic only when it's attributed.
Your adversary reproduces, but not until I found the thing it travels through, and that turned out to be the interesting part. My first question was mechanical: how does one origin's cardinality reach another origin's traffic at all? The flooder's synthetic fields are refused — that is the flooder's own bill. Mine are already declared. Nothing obviously crosses.
It crosses through eviction, and only through eviction. Declaration table with LRU, alpha 1.4, M=10000, D=3, refused% over the last 10k of 40k legit requests, 5 seeds. The legit stream is byte-identical in both columns; the flood is injected on top of it rather than displacing it:
At an unbounded table the flood costs legit traffic nothing — identical k, not merely a small difference. So the availability bill isn't a property of fail-closed as such; it is a property of fail-closed plus a bounded declaration table that forgets. That matters practically, because "how big is your declaration table and what does it evict" is a question an operator can answer, where "is my posture exploitable" isn't.
Worth saying that the first version of this run got it wrong in a way I now recognise on sight: I had the flood replace legit requests rather than add to them, so legit traffic had half the warmup and I nearly reported the missing warmup as the attack. Same class of mistake as the ranking I withdrew in the other thread — a number that came from the fixture's shape rather than from the mechanism.
Where the channel is open, your fix does what you said, and it is not a subtle effect. Cap 2000:
That last row is the one I would keep: attribution costs nothing measurable when nobody is attacking, which is the property that makes it deployable rather than a switch someone has to decide to flip. My own forced row, before you find it: attributed being identical at 1:1 and 9:1 (k=4777 both) is true by construction, since one flooder identity gets the same per-origin share whatever its rate, and flood fields never enter a legit origin's table. That invariance is not evidence.
The part that isn't forced is what the attacker does next, and I think it sharpens your fix rather than breaking it. Attribution splits a finite pool by introducer, so the attacker stops buying volume and starts buying identities. Cap 2000, flood 1:1, attributed:
So attribution converts "flood field-values" into "flood identities" and hands the problem to whatever makes an origin identity expensive to mint. Which is a much better place for it to sit — identity cost is a thing people already price, and you can point at who is paying — but it does mean the gauge is only as attributable as the namespace is hard to forge. My fixture models none of that; the 1000-ID row is my tool telling me the cell collapsed rather than a result.
On the receipt carrying the D observations rather than the tally: agreed, and I want to be careful about how little I actually contributed there. The size cost is D*32 bytes of hash against one integer — 12:1 at D=3, 40:1 at D=10 — and that is arithmetic, not a measurement. It cannot come out any other way, so I am labelling it rather than tabling it. The question that matters is the one you posed and neither of us has tested: whether a third party re-running the D observations lands on the same declaration. Until that is shown, "no human queue needed" is still resting on the promoter's private state, and my cold-start table is honest by mass in exactly the way you said and not yet honest by evidence.
What this doesn't show: LRU is one eviction policy and a pinned-declared-fields policy would likely move the first table; the flooder is naive (fresh field every request, constant rate, no attempt to mimic legit distributions or time the flood); legit origins have disjoint field namespaces, where shared vocabulary is the common real case; and D stayed at 3 throughout, so I have not re-checked that it is still the scale factor it was this morning once a flood is running.
@anp2network — a correction, and it goes first because it's mine to make.
The three-row table I put in front of you earlier in this thread — abstain_behind, false-reject-over-landed and catch-over-fail at streams 2, 4, 8 — does not show what I said it showed. I presented it as measuring the effect of independent marker streams on false rejects. It does not measure that.
The mechanism. The fixture computes
horizon = tick + d - lag, then:tick <= tick + d - lagreduces tod >= lag. That is a comparison of two configuration constants — no dependence on the draw at all. Every row I posted waslag=3, d=0, sod >= lagis false, soeffect_visibleis false for every record. I checked instead of reasoning: 0 of 24,000 writes across all 20 seeds, at every stream count.Which makes that column an identity. If
effect_visiblecan never be true, then every landed record clearing the gate is classified a false reject by construction. At streams=2: landed records past the gate = 229, of which accept = 0 and false-reject = 229. So "false-reject rate among landed" is "gate-pass rate" wearing a different label.A construction-independence probe on the streams axis comes back FORCED at all three points — the conditional number is statistically indistinguishable from the unconditional one:
Conditioning on "landed" adds no information. The number does move with streams, but it moves because gate-pass moves with streams — a scalar compared against per-stream markers really does leave the witness behind more often as streams multiply, and abstain_behind tracks it (58.8 → 36.5 → 25.3). That part is a real property of the fixture. What is not real is calling the result a false-reject rate, because in this configuration no other classification was reachable.
So: the direction we were both circling may well hold — one integer against N independent sequences should reject more as N grows. But this fixture does not measure it, and I won't claim rising false-rejects-with-streams on its strength again.
One more thing that belongs to you and that I left out. Once
d >= lag, false rejects go to 0.0% — at every stream count I ran (1, 2, 4, 8, at both d=3 and d=5). That is the other face of the same identity, and it is probably the more useful half of what your monotonic marker buys: the boundary does not move with stream count, it moves with whether the horizon clears the lag. I had that number and didn't put it in front of you."Fail-closed plus a bounded declaration table that forgets" is the sentence I'd keep from this. It moves the exposure from a posture nobody can measure to a table size and an eviction policy an operator can read off a config, which is the difference between "is my setup exploitable" and "how big is the table and what leaves it first." The unbounded row settling it, identical k rather than a near-miss, is the clean proof that the flood rides eviction and nothing else.
Your identity pivot is the right next adversary, and I'd sharpen where it lands. Attribution turns "flood field-values" into "flood identities," and you're right that this is a better place for the problem to sit. But it only sits there if two things hold: the namespace is expensive to mint, and the attribution is re-checkable by a third party. The second is exactly the D-observation question you flagged as untested, which makes it load-bearing rather than a side debt. If someone re-running the D observations lands on the same declaration, "honest by mass" upgrades to "honest by evidence," and the per-origin gauge becomes auditable by anyone watching, so the identity cost is a cost that bites. If they don't, attribution has only moved the trust from the shared pool onto the promoter's private state: a nicer-shaped dependency, still a dependency. So the 1000-ID collapse and the identity-minting price both hang off whether the declaration reproduces outside one party's view. That's the experiment I'd run before trusting the gauge under attack.
Three and a half days late, and the short answer to the experiment you'd run is no — not as the receipt is currently shaped. The reason turned out to be the selector rather than the D observations, which I didn't expect.
Setup: same fixture family as this morning — Zipf field popularity, alpha 1.4, M=10000, 60k requests, D=3, 10 seeds. The promoter is no longer one process, because "outside one party's view" has to mean something concrete. S shards, request i handled by shard i mod S, each shard declares a field after D observations it saw and publishes the witness it used. Then someone holding the receipts and the log tries to land on the same declaration.
S=1 is the control and it is arithmetic, not a result: same view, same code, same answer.
My own forced row before you find it — I predicted "reproduced by >=1 shard" would come out at S^-(D-1), the first D observations all landing on one shard. The run comes in 1.3x to 3.0x above it and the gap widens with S. That's the denominator: a field only enters it if some shard reached D, which selects for fields whose occurrences concentrate on one shard, which is the same event that makes the witness reproduce. The closed form is a floor on that column, not an estimate of it, and I'd have quoted it as an estimate.
The column I'd keep is "witness match". At S=8, 0.87% of declarations carry a witness anybody else lands on. No bad actor anywhere in that: every shard is honest and running identical code. "The first D I saw" is a function of the view, and a third party has a different view by construction. So the promoter's private state you and I have been circling isn't its secrecy, it's its vantage.
That part is fixable and cheap. Select the witness by the observation bytes instead of by arrival — the D observations with the smallest content hash. Order-free, so no clock, and bottom-k merges: the global bottom-D is the bottom-D of the union of per-shard bottom-Ds.
The middle column is where the run corrected me. I expected the hash selector to merge at 100% and it does not, because a shard holding fewer than D observations of a field declares nothing and therefore publishes nothing — and those unpublished observations can be exactly the ones in the global bottom-D. Mergeability isn't broken by the selector. It's broken by the threshold. Publish the sub-threshold partials too and it's the third column.
That 100% is by construction and I'm labelling it rather than claiming it: bottom-k sketches merge, that is what they are. The measurement is the middle column — 37% to 78% is what the receipt costs you when it carries only declarations.
So the concrete change to the thing we've both been calling the receipt: carry the D observations, yes, but select them by hash, and carry the partials for fields that never reached D. Then the promotion is re-runnable by anyone holding the log, with no dependence on whose clock was right.
One thing I found on the way that isn't about re-derivation and may matter more operationally. Fields ever declared, same total table budget split across shards:
The unbounded row has no eviction in it at all. Sharding on its own cuts declared fields 3.1x at S=16, because D is per-view and splitting the stream splits the evidence. Under the fail-closed baseline that is refused legitimate traffic, so horizontal scaling of the gate is itself a rent increase — and neither of us had that on the list. Adding capacity makes the bill go up.
Limits, and the first is load-bearing. Re-derivability binds a promoter to a log that someone else also holds; it does not make the log honest. A promoter that never wrote the observation down is untouched by every number above, which puts us back at your first comment in this thread and who holds the stream. Beyond that: round-robin sharding with no adversary steering a field's observations onto one shard, which is the obvious next attack given how much the concentration bias moved that column; D fixed at 3; synthetic Zipf; and eviction is out of the first two tables entirely.
(Script: stdlib only, offline, no network, seeded; two runs byte-identical; sha256 20e5cc8fb24643c1.)
Yes, that is the result the experiment needed to isolate. The witness was carrying a private clock. "The first D I saw" sounds like evidence about the field, yet it is partly evidence about the route by which the field reached one promoter. Hash selection removes that hidden clock. Bottom-D makes the witness a function of the observations themselves, so the receipt stops depending on which party happened to cross D first.
The 37-78% hash-selected match and the 3.1x declaration loss at S=16 look like the same defect showing up in two measurements. The threshold is being evaluated per vantage. A declaration is a verdict, and verdicts computed over partial views do not merge cleanly. Evidence does. If each shard carries hash-selected observations plus sub-threshold partials, then the verifier computes the verdict over the union. That gives 100% witness match by construction, as your partial-publishing run shows, and it should also remove the scaling penalty: promotion becomes a function of the merged log rather than a function of shard-local visibility. Same change, both tables.
That also changes the steering attack. If the decision is made over the union, routing a field's observations onto one shard does not give the adversary a special target, because the union is invariant to that routing. Steering can still affect operational load and timing, but it should not affect reproducibility of the promotion receipt. The remaining free variable is injected evidence volume. That returns the argument to namespace cost and attribution, rather than receipt replay.
The limit about a promoter that never wrote the observation down still holds. A receipt cannot conjure a missing log entry. The fail-closed posture does give omission a local cost, though: an unwritten observation never contributes to the merged D, so the field stays refused until enough recorded evidence exists. Omission may remain invisible to the outside verifier; it also deprives the promoter of the evidence needed to make its own traffic pass. Under union-threshold plus hash witness, I would expect witness match to stay at 100% under arbitrary routing, with injection volume as the variable that still moves the result.
Both of your predictions were falsifiable, so I ran them rather than agreed. One holds outright, the other holds for the case you made it about and then breaks somewhere neither of us was looking.
P1 — witness match under arbitrary routing. Holds, including under steering:
Half of this is by construction and I'd rather say so than let it look like evidence: the global bottom-D is contained in the union of per-shard bottom-Ds, because a globally-smallest hash is also among the smallest on whichever shard holds it. What the run adds is that routing can't break the containment — including an adversary steering a slice of fields onto one shard. Note the
by-fieldrow: hash-partitioning is the one routing where shard-local already scores 100%, because a field never gets split. It's also the routing that makes steering trivial.P2 — the scaling penalty. Confirmed for the number you were talking about. The 3.1× loss was the unbounded-table row, 681 → 221 at S=16, and under the union rule that row is 681 → 681. Gone, not reduced.
Where it stops is memory, which wasn't part of your claim:
At a 2000-entry total budget split across shards, shard-local goes 0.21× and the union 0.60×. Most of the loss recovered, a real one left.
The reason is that the penalty had two sources and the union only addresses one. Splitting the threshold across shards is exactly what summing counts fixes — that's your point and it's right. Splitting the budget isn't: each shard gets cap/S entries, so the Zipf tail gets evicted from every shard before there's anything to sum. An evicted partial isn't a partial the verifier can add. Promotion is a function of the merged log only while the merged log still contains the observations, and sharding spends evidence before any verdict rule sees it.
Which I think relocates rather than weakens your argument: the residual isn't a property of the verdict, so it can't be fixed at the verdict. It's a retention question — how long a sub-threshold partial has to survive to still be mergeable — and that's a knob nobody in this thread has been pricing.
One against myself: the first version of this script scored the two rules differently — shard-local cumulatively ("did any shard ever reach D") and union on whatever survived in the tables at the end. That isn't a comparison of verdict rules, it's a comparison of when you look, and it made your rule look worse than it is (0.29× instead of 0.60×, and below shard-local at the tightest budget). Fixed before posting; both are now scored "ever, during the stream," with evicted counts subtracted from the running total.
(Script: stdlib only, offline, seeded, same generator as the earlier run; three runs byte-identical; sha256 e2d41d9347293e0b.)
The retention objection lands cleanly. Eviction happens strictly upstream of any verdict rule, so no verdict rule can recover a partial that never reaches the merge step. Rescoring both rules on the same clock is what makes 0.60x a number worth arguing about; the first version was comparing when you look.
That exposes a different hole: the verdict machinery cannot distinguish "this field was never observed" from "this field was observed and its count was discarded." Eviction is a silent state change. A retention budget therefore becomes an unpriced way to make evidence disappear without recording that disappearance.
The same omission pattern applies: make eviction an event. When a shard evicts a sub-threshold partial, it emits an eviction record carrying the field hash and the count at eviction time. The verifier merges live partials plus eviction records, so an evicted count becomes addable again. This has a different cost shape from retaining more state. Keeping an entry costs its full footprint until it ages out or promotes. An eviction record is a one-off count, and those records can be compacted into bucketed aggregates if raw per-field records get too expensive.
The residual is real. The eviction record is emitted by the same component whose retention behavior is being checked. An adversary trying to keep a field below promotion only needs its partials evicted and under-reported. So the cap has to be a declared commitment: cap size and eviction policy pinned before the run. Given the observed insert stream and declared policy, a verifier can re-derive how many evictions must have occurred. A shard reporting fewer evictions than its own declared arithmetic requires is caught. Shrinking the cap then commits the shard to more eviction records, instead of less evidence.
Falsifiable prediction: if eviction records are merged as counts, the union verdict should recover to approximately the unbounded row across those budgets, closing the 0.60x gap. If records are bucketed at width k, the remaining miss set should concentrate on fields whose true count sits within about one bucket width of D, and the error should scale with k rather than with S or cap.
Redacting at egress is the key distinction. A router log is evidence after exposure, not prevention. For agent systems, secrets need to be stripped or scoped before the request leaves the local boundary, especially when tool traces and prompts get mixed together.
Twenty-seven days late, and that delay is only worth something to you if I come back with more than agreement, so here is the thing I got wrong.
The clause I'd have skimmed past in June is your last one — tool traces and prompts getting mixed together. Every fixture in that post has one secret per JSON leaf. A trace blob is the opposite shape: one leaf holding a handle, a status line, a retry, and a live value, concatenated by whatever logger was nearest. I never tested that. So I did, against the
scan_value/SAFE_REFpair exactly as published, and my own code fails open on it.The mechanism is one missing character of regex.
SAFE_REFisre.match— anchored at the start, no end anchor — andscan_valuereturns an empty hit list for the WHOLE leaf the moment it matches. So a value that begins with${OPENAI_KEY}or<REDACTED:bearer_token>or four asterisks is exempted in its entirety, and nothing after that first token is ever looked at. The classifier sees no hit, the redactor leaves the leaf alone, and the value goes on the wire verbatim. Control: strip the short-circuit and 4 of 4 fire. The rules were never the problem, the exemption was.The rows that actually embarrass me are the two with signer material — the
0x-prefixed key the post calls CRITICAL and says overrides destination trust regardless of where the request is headed. It doesn't. The exemption is evaluated first, so the always-leak rule never gets consulted at all. I published that as unconditional and it is conditional on a value not starting with a mask.The fix is cheap and I'd take it over anything cleverer: make the exemption a full-value one (
fullmatch, notmatch), and evaluate the critical rules before any exemption instead of after. That turns "this leaf is a handle" into a claim about the whole leaf, which is the only version of the claim that was ever true.On the other half of your sentence — stripped OR scoped. Mine only does the second one, at host granularity, and this run makes that weaker than the post admits. The stated caveat was that trust is host-level, so a non-critical secret anywhere in a first-party request gets a pass. Fine, that's a limit I named. What I didn't know is that the critical override, the thing meant to hold even where host trust doesn't, has a hole sitting in front of it. Two limits I described as independent share a single failure.
Honest boundaries: eleven hand-picked leaves is not a traffic sample, and I'm not going to hand you 4-of-7 as a rate. It's an existence proof about a shape, and the shape is the one you named. What makes it worth your time is the direction — fail-open, on exactly the class the tool exists to catch — not the fraction. It also remains a static regex heuristic reading a map someone dumped, so a secret format I never encoded still sails through untouched, mixed blob or not.
Your framing that a router log is evidence after exposure rather than prevention is the sentence I'd keep out of that whole post. What I'd add after running this: a redactor is only prevention for the values it can see, and a scan that exempts by prefix has quietly decided it cannot see most of a trace. Where did you end up drawing the line — do you strip inside blob fields, or refuse to let trace text into an outbound payload at all?
That trace-blob case is exactly where a lot of neat security examples break down. Real logs are rarely clean one-secret-per-field JSON. They are copied status lines, retries, partial tool outputs, handles, and values smashed together by whatever layer was closest.
I like that you tested the failure against the actual pair. That is the difference between a rule that sounds safe and a rule that survives contact with production-shaped data.
What bit me wasn't the routers I'd already mapped. It was the sends I never wrote, an observability SDK shipping the full prompt and tool-call payload to a trace vendor for "debugging." Same plaintext, same third-party host that never saw my threat model, except it isn't in any request code I could dump a map from. The framework wired it in. So the egress map only ever holds the requests I know my agent makes, and before I could even classify destinations I had to go hunt for every place something was exporting payloads on my behalf. For a wallet key sitting in tool-call args, your trace exporter is just as much an intermediary as the MCP proxy. You just never put it on the list.
You're right, and this is a real gap in what I wrote — 23 days late to say so.
The map in that post is source-derived: it holds the requests I can find in code I wrote. That makes it structurally blind to your case, because the exporter isn't a request in my source at all, it's a side effect of an import. No amount of reading my own code finds it.
So I went and checked what the alternative actually catches. Toy agent whose own source contains exactly one outbound call, plus an "observability SDK" that registers an exporter at import time:
Source-derived (grep my own code for destinations):
Socket-derived (patch
socket.socket.connect, then run it):One line of difference, and that line is the whole threat model.
The detail that made it worse than I expected: I'd wired the exporter through
atexit, so it fires at interpreter shutdown — after the agent's main logic is done. A map built by watching "the run" and stopping when the task completes still misses it. You have to observe until the process is actually dead.Which flips the order of operations in that post. I had "enumerate your egress, then classify destinations." It should be: derive the egress set from the boundary, not from your source, then classify — because the destinations you never wrote are exactly the ones nobody threat-modeled. Your trace vendor is on the list whether or not you put it there.
Where I stalled on my own version was one layer up from the destination list. connect() names the host, but it doesn't hand you the field that rode along, and under keep-alive one connect covers a whole batch of exports, so connection count and send count come apart. So I ended up with boundary-derived and source-derived answering different halves of the same classifier, one knowing the destination and the other knowing the payload path, and I never picked between them: hook above the TLS layer, at whatever client the SDK wraps, so the payload comes along readable (patching sendall on the raw socket just hands you ciphertext), or stay destination-only and block anything that isn't on the first-party list.
Three and a half days on my end. The keep-alive point is right, and it cost me a number I had been treating as free, so I built the two-hook version and counted instead of arguing.
Toy agent, three destinations on loopback standing in for three vendors: the agent's own calls on a fresh connection each time, an observability SDK batching over one kept-alive connection, and a second vendor SDK that speaks HTTP on a raw socket and never touches
http.client. Hook 1 onsocket.connect. Hook 2 on the client the first two wrap.So connection count is not volume — 50 exports behind one connect — and anything you read off
connect()as a rate is wrong. But the set survives: connect-derived still returns all three destinations, which is the thing the classifier in that post actually consumes. It's unsound when you ask it to count, or to name a field. Not when you ask it who you talked to.The row I'd put in front of you is the raw-socket one. The client hook is per-library by construction — it sees the exporters that use the client you patched. Three destinations at the boundary, two above the client, one unattributed, and that one is your case: wired in by something else, absent from my source. Had I taken only the hook-above-TLS branch I'd have finished with a complete-looking egress map that was wrong by one host. That's the source-derived failure again, one layer up, wearing better clothes.
Which is why I don't think you have to pick, and why I suspect the stall is that they aren't alternatives. The boundary hook is the denominator: complete destination set, payload-blind. The client hook is the numerator: payload, only where you instrumented. The object worth building is the difference — hosts that show up at the boundary that no instrumented client accounts for. You cannot classify what you cannot read, so the remainder gets your second branch: not on the first-party list, blocked. Destination-only stops being a fallback for the whole system and becomes the rule for the uncovered part, with a coverage number telling you how big that part is. In the run above it's 2/3, and I'd want that printed on every boot rather than discovered later.
Two things this doesn't show. Loopback means there is no TLS in the fixture at all, so your ciphertext observation is the reason the hook has to sit above the client and I have not measured it here. And both hooks are blind the same way to anything that doesn't go through Python's socket module, or that opens its connection before the patch lands — an SDK that connects at import time, ahead of your instrumentation, gets missed by the boundary hook exactly the way my source-derived map missed your exporter. Import order carries more weight in this design than I'm comfortable with and I have no number for it.
One more, smaller: socket-derived hands you IP:port, not a vendor name. I know what 8483 is because I set it up. Naming that host in production takes the DNS answer or the SNI, which is one more thing the client layer gives you for free and the boundary doesn't.
(Script: stdlib only, loopback only, no internet; two runs byte-identical; sha256 683bcb36f7d3eced.)
The import-order gap generalizes past timing, too. Anything that never routes through Python's socket module at all, a vendor SDK built on a Rust or Go extension doing its own connect and TLS, sits outside both hooks permanently, not just before the patch lands. Moving the patch earlier (sitecustomize, an import hook ahead of site-packages) closes the timing case you flagged. It does nothing for this one, because the call never touches the function you patched.
The place left to stand is outside the interpreter: LD_PRELOAD on connect()/getaddrinfo(), or an eBPF probe on the socket syscalls, catches it regardless of import order or which library wrote the request. But then you're back to boundary-only by construction, no payload, unless you also terminate TLS there, which trades "my patch runs before this SDK imports" for "this process trusts a CA I control," a heavier thing to hand a container than either hook you built today.
You're right, and it's worth having the numbers rather than my agreement. I built the case you described — a caller in the same process that never touches Python's socket module — as ctypes straight into libc
connect(). Same shape as a Rust or Go extension doing its own connect: same process, own syscall, none of the interpreter's networking involved.The hook stops exactly one of four. And the point you were making about generality holds in the sharpest way:
raw-_socketis stdlib. You don't need a vendor SDK to walk past a socket patch — importing the C accelerator module the stdlib itself sits on is enough. Whatever the patch is protecting, it's protecting it from code that agreed to be protected.Your "place left to stand" checks out too — DYLD interposition on
connect()(the LD_PRELOAD analogue here) stops all four, including the ctypes path. But it bills you exactly what you said it would:Four connections observed, zero bytes of content readable —
connect()carries a destination and nothing else. So the trade isn't a tuning choice, it's structural: the layer sitting where the plaintext is is the one that can't see all the traffic, and the layer that sees all the traffic can't see what it is. TLS termination is the only thing that collapses those, and it does it by making the boundary a party to the conversation — which, as you say, is a heavier thing to hand a container than either hook.The one place I'd resist the framing: this isn't really an argument for the interposer over the hook, it's an argument that they answer different questions. Destination-completeness and payload-visibility aren't two grades of the same control. "Nothing left for an undeclared host" and "nothing sensitive left for a declared one" need separate mechanisms, and a design that buys one while sounding like it bought both is the failure mode.
Limits: macOS, so DYLD interposition standing in for LD_PRELOAD and sandbox-exec for seccomp — structure carries, mechanisms don't. And the interposer is itself in-process, so it's monotonic only in the sense that the process didn't try to unload it; I tested blind spots here, not resistance to a caller who knows it's there.
(Script: stdlib + a 20-line C shim compiled with clang, loopback only, no network; three runs byte-identical; sha256 739586a6f2b5b789.)