DEV Community: Josh Waldrep

Per-Pod NetworkPolicy in Practice: Migrating Five Agents in a Day

Josh Waldrep — Thu, 21 May 2026 00:02:04 +0000

A working cluster ran five AI agents, each with an in-pod Pipelock sidecar scanning their traffic. The boundary was real but advisory: every container in a pod shares a network namespace, so a NetworkPolicy that says "no internet for this pod" applies to the sidecar too. Tightening the agent's egress required loosening the sidecar's egress, which defeats the point.

This post is a field report on migrating those agents from in-pod sidecars to a separate-pod model where the firewall lives in its own pod with its own NetworkPolicy, and the agent pod has no route to anywhere except the firewall. The migration took about a working day and surfaced six gotchas worth writing down.

The structural reason for separation

NetworkPolicy is per-pod. The Kubernetes networking spec does not allow a NetworkPolicy to scope to a specific container in a pod, because the policy is enforced by the CNI plugin which sees pods as the unit of identity. Three containers in one pod are one Kubernetes "you" from the CNI's perspective.

This means an in-pod sidecar architecture has a contradiction baked in: the agent container should have no internet, and the sidecar container needs internet because the sidecar is the agent's exit. Both are in the same pod. NetworkPolicy applies to both equally. You either let the whole pod out or block the whole pod, including the sidecar.

In practice, this gets papered over with HTTPS_PROXY configuration on the agent container and a wide-open NetworkPolicy on the pod. The sidecar can scan everything the agent sends through the proxy. The agent can clear HTTPS_PROXY in a subprocess and dial direct, and the kernel will let it through because the pod's NetworkPolicy says yes to internet.

The fix is structural. Move the firewall to its own pod. Tighten the agent pod's NetworkPolicy to allow egress only to the firewall pod's service. Now the agent pod has no internet route except through the firewall, the firewall pod has the internet it needs, and NetworkPolicy enforces both shapes correctly. This is the per-pod model.

The five agents and the migration sequence

The fleet had five distinct agent deployments. Each had been running with an in-pod sidecar for months. The migration sequence:

One agent first, fully end-to-end, including the bypass-closure verification probes from the three-UID containment playbook adapted to Kubernetes.
Three agents in parallel, applying the same pattern.
The fifth agent last, because it had a workload-specific quirk (anti-bot scraping, see below) that needed its own treatment.

Per agent, the steps were:

Generate a pipelock-companion Deployment + Service in the same namespace.
Generate a scoped pipelock-companion-egress NetworkPolicy permitting TCP 443/80 only for that pod selector.
Update the agent Deployment to drop the in-pod Pipelock sidecar.
Update the agent's environment to point HTTPS_PROXY at the companion service.
Tighten the baseline NetworkPolicy on the agent pod to remove the wide internet-egress rules and keep only the egress to the companion service.
Ship the change through the normal manifest pipeline.
Run the bypass-closure probes: raw TCP dial, env-clear subprocess, NO_PROXY domain match.

The first agent took four hours including two false starts on init container egress. The fourth agent took less than an hour. The pattern of "first one is a slog, the rest fall in line" held.

Gotcha 1: subPath ConfigMap mounts don't hot-reload

The single biggest unexpected friction during the migration. Pipelock has fsnotify hot-reload on its config file. Kubernetes ConfigMap updates are supposed to propagate to mounted files. They do, for directory mounts. They do not, for subPath: single-file mounts. The mount path stays pointing at an immutable file-handle from pod creation.

A handful of namespaces had their Pipelock config mounted with subPath:. Multiple commits added redaction allowlist entries that landed in the cluster ConfigMap object but never reached the running Pipelock instance. The fix was a kubectl delete pod to force a fresh mount.

The structural fix is to mount the ConfigMap as a directory whenever possible. The detailed writeup is in subPath ConfigMap Mounts Don't Hot-Reload. For this migration, the workaround was "always restart the pod after changing Pipelock config" until the mount shape gets fixed.

Gotcha 2: init containers that fetch from the internet

Several agent deployments had init containers that fetched the Pipelock binary before the main container started. The pattern:

initContainers:
  - name: install-pipelock
    image: alpine
    command: ["sh", "-c"]
    args:
      - |
        wget -O /opt/bin/pipelock https://github.com/luckyPipewrench/pipelock/releases/...
        chmod +x /opt/bin/pipelock

This worked fine when the pod's NetworkPolicy allowed wide-open egress. After the lockdown, the init container's wget was the first thing to fail, because the pod's NetworkPolicy now denied direct internet.

The fix was to make the init container copy the binary from the same image used by the companion pod. Zero network needed.

initContainers:
  - name: install-pipelock
    image: pipelock:VERSION
    command: ["sh", "-c"]
    args: ["cp /usr/local/bin/pipelock /opt/bin/pipelock && chmod +x /opt/bin/pipelock"]

The image is the same image the companion pod runs, so the binary is byte-for-byte identical to the running companion. That becomes the canonical pattern across agent pods.

Gotcha 3: browser automation that can't tolerate TLS interception

One of the agents had a browser-driver container for workloads that fail under MITM. The browser stack did its own TLS handling and broke under interception. Pipelock could be configured with passthrough_domains to skip MITM for those targets, but the browser's egress also needed direct internet, not loopback to a proxy.

The structural fix mirrored the firewall split: the scraping tool moved into its own Deployment, with its own egress NetworkPolicy permitting TCP 443/80 directly. The agent's NetworkPolicy got an additional allow rule for traffic to the scraping service. The agent calls the scraper through the cluster service, the scraper reaches the internet directly for its scraping work, and Pipelock no longer sits in the path for that traffic.

This is an architectural compromise the per-pod model accepts: tools that fundamentally cannot work behind MITM get their own pod with their own egress. Pipelock loses visibility into the scraping traffic, but the URLs the agent passes to the scraper go through Pipelock's MCP-stdio scanner, so the calling-side surface is still covered.

Gotcha 4: identity binding with shared namespaces

Two of the agents lived in the same namespace and shared PVCs and ConfigMaps but bound different identities to their Pipelock companion. A single companion deployment cannot bind two identities; the companion config has one default_agent_identity field.

Two options: deploy two companions, one per agent, or use the Pro agents.*.source_cidrs feature to match against pod IPs.

The migration chose two companions. The cost is roughly one extra pod's worth of memory (~50 MiB per companion). The benefit is that every agent identity maps to one companion deployment, which keeps the namespace's manifest set fleet-consistent. The Pro source_cidrs feature would be more elegant but is more complex to dogfood-validate and has not been the test path.

For a single-tenant namespace, one companion is enough. For a shared namespace where each tenant binds a distinct identity, the per-tenant companion is cleaner.

Gotcha 5: projected secret volume modes

The Pipelock companion runs as an unprivileged UID. The TLS-MITM CA is delivered via a Kubernetes Secret mounted into the companion pod. The first version of the manifest set the volume mode to 0444 (world-readable), thinking that would let the companion read the file regardless of UID.

Pipelock refused to start. Its validator (in internal/config/validate.go) rejects CA key files whose mode permits world-read, any-write, or any-execute, by masking against 0o137. That allows 0o600 (owner-read) and 0o640 (owner-read plus group-read), and rejects 0o444 and 0o644.

Two patterns work:

0o600 plus matching owner. Set the pod's runAsUser (or runAsNonRoot plus an explicit runAsUser) to a UID that matches the file's owner. Kubernetes Secret volumes default the file owner to root unless securityContext.runAsUser is set on the pod, so this path needs that field plus a matching runAsUser for the Pipelock container.
0o640 plus fsGroup. Set the volume's defaultMode to 0o440 or 0o640, and set the pod's securityContext.fsGroup. Kubernetes chowns the secret files to the fsGroup and adds the fsGroup to the container's supplementary groups. The container reads the file through the group bits, no matter what UID it runs as.

The companion deployment uses the fsGroup pattern because it composes cleanly with runAsNonRoot: true and arbitrary container UIDs. The narrow trap is 0o400 plus fsGroup: the file mode has no group-read bit, so the supplementary group does not grant access — the only reader would be the file owner, and Secret volumes do not put the container UID there by default.

This pattern is consistent across Pipelock companion deployments now, but the first time it surfaces is a confusing thirty minutes of debugging "why does my secret mount fail to be readable by the only container that should read it."

Gotcha 6: VPN sidecar flake during companion bring-up

One namespace's existing in-pod sidecar setup had been running a VPN sidecar for months without incident. The first time the new companion pod came up with a fresh VPN sidecar, the tunnel cycled for several minutes before stabilizing. Same image, same provider, same config. The difference was server selection, evidently a flaky one.

The lesson: do not put VPN on the companion pod when the agent does not strictly need exit-IP rotation. The agent firewall's job is content scanning. The cluster's normal egress is fine for that. The companion pods do not need a VPN.

The legacy VPN stays for one specific deployment that has other sidecars depending on it. New companion pods are VPN-free. This shaved another moving part out of the operational footprint.

What the day produced

End of day, the cluster had:

Five companion pods running, one per agent identity.
Five baseline NetworkPolicies tightened to remove direct internet-egress rules.
Five agent deployments with the in-pod Pipelock sidecar removed.
Two scraping deployments split out so the agent's NetworkPolicy could stay tight.
A small set of manifest commits, each with a clear rollback path.
A bypass-closure probe set verified on the first agent, partially run on the other four (full sweep is a v2.4.x followup).

The structural model is now consistent across the fleet: agent pod has no internet, companion pod has internet, NetworkPolicy enforces the separation, and the agent's runtime choices do not reach the kernel.

What I would do differently next time

Three changes for the next migration like this:

Audit ConfigMap mount shapes before starting. Every subPath: on a hot-reload-bearing config file is a future "why is the change not propagating" debugging session. Switch them to directory mounts up front.
Pre-build the migration commits per agent. The pattern of "drop sidecars, add companion, tighten NetworkPolicy" is the same across all agents. The first agent is bespoke; the rest can be templated. A small Kustomize generator would have saved time on agents 2-5.
Run the full bypass-closure probe set on every agent. Doing it on agent 1 and skipping it on agents 2-5 because "the pattern is the same" is the kind of confidence that bites you. The probes catch one-off regressions that the boilerplate cannot. v2.4.x followup is to make the probe set a CI-runnable artifact so every namespace has one.

The non-glamorous lessons compound. Most of the friction in this migration was operational, not architectural. The architectural model (per-pod separation) is correct and stable. The operational details (mount shapes, init container egress, secret modes, identity binding) are what consumed the day.

If your fleet runs in-pod agent firewall sidecars, the per-pod migration is worth the day. The structural change closes a class of bypass that no amount of HTTPS_PROXY tightening can close.

Mediator Receipts: The Question to Ask About Agent Attestation

Josh Waldrep — Wed, 13 May 2026 23:28:07 +0000

If your AI agent signs its own decision receipts, the agent is its own witness. That matters when an auditor, regulator, or customer security team asks "who signed this." The cryptography is fine. The chain holds. The question is who held the pen.

I'm not picking on any vendor. As more agent runtimes ship signed-receipt formats, the architecture question lives in the same place every time: where does the signing key sit, and what's its trust relationship to the agent process?

The two shapes

A signed receipt has three parts: payload, signature, signer. Payload says what happened. Signature proves the payload wasn't altered after signing. Signer is the entity holding the key.

Most formats pin down payload and signature up front. The thing that varies across formats is the signer.

Shape one: signer is the actor. The agent runtime holds the key. The agent decides, generates a receipt, signs with its own key, ships to the log. Chain of custody is one entity wide.

Shape two: signer is not the actor. A separate process holds the key. The agent makes a request, the request flows through that separate process, the separate process makes the policy decision and signs the receipt. The agent never sees the key. Chain of custody crosses a trust boundary.

Both shapes produce signed receipts. Only one of them survives a question about the signer.

What "outside the trust boundary" buys you

Trust boundary is the line around a set of components that share fate. If A is compromised, everything inside A's boundary is potentially compromised. A signature proves bytes came from the key holder. A signature doesn't prove the key holder is trustworthy.

If the agent holds the key, the boundary around the signature is the same boundary around the agent. A prompt injection that gets the agent to send a bad request is in the same position to influence what gets signed. The signature proves the receipt came from the agent. It says nothing about whether the receipt is honest.

Move the signer outside that boundary and the math changes. A separate process with its own key in its own memory writes the signature based on what it observed of the agent's traffic. The agent can't forge or rewrite mediator-signed receipts for traffic that passes through the mediator, because the agent never holds the signing key. A successful prompt injection still produces a bad request. The receipt records the mediator's verdict on that request, because the recorder is somebody else.

The same separation that lets a proxy enforce policy on a compromised agent lets the proxy attest to actions on a compromised agent. The attestation is honest about a dishonest action. That's the property an auditor wants.

That's why receipt architecture sits next to enforcement architecture in the three-UID containment pattern. Once the agent becomes the signer, the receipt is self-attestation. Move the signer behind the same boundary as the proxy and the receipt class flips.

This isn't a feature comparison

A vendor whose receipts are signed in-process isn't lying. The cryptography is sound. The chain is verifiable. The thing those receipts can't do is survive a question about who signed them.

An auditor asks: "How do you prove the agent didn't produce a fraudulent receipt that says 'access denied' while the actual access was granted?" In-process answer: "We trust the agent runtime." Out-of-process answer: "The receipt is signed by a process the agent has no key to. The agent couldn't have signed that fraudulent receipt with the mediator key."

The first answer is a statement of trust. The second is a structural property. For internal use, the first is plenty. For regulated industries, audit-bearing logs, or any case where the agent itself might be under scrutiny, the second one holds up.

Same distinction shows up in financial audit (self-prepared statements vs auditor-signed reports) and in legal evidence (defendant's own records vs witness testimony). Records can be true either way. Evidentiary weight differs because chain of custody differs.

Two pictures

In-process signer:

Agent process
├── Decision logic
├── Signing key
└── Signature output → Receipt log

Agent decides, holds the key, signs. Compromise of the agent process compromises the signing path.

Out-of-process signer:

Agent process              Mediator process
├── Decision logic     →    ├── Observes traffic
└── Action initiation       ├── Applies policy
                            ├── Signing key
                            └── Signature output → Receipt log

Agent initiates. Mediator observes, decides, signs. Compromise of the agent doesn't by itself compromise the signing path because the key sits in a different process.

Out-of-process is harder to deploy. Traffic has to flow through the mediator (which is what an agent firewall does). The mediator has to live in a different trust boundary from the agent (Linux UID separation, separate Kubernetes pod, separate VM). The mediator needs its own threat model, key management, and operational story.

The cost is real. The evidentiary weight is also real. Whether it's worth it depends on what you need the receipts for. Internal debugging log, in-process is fine. Audit evidence, capability separation is what makes the receipt worth more than its bytes.

The question to ask any vendor

Evaluating an agent security or governance product that ships signed receipts boils down to one short question:

Where is the signing key held, and what's the trust relationship between that location and the agent process?

Three possible answers map to three shapes:

"The agent process holds the key." In-process signer. Class: self-attestation. Useful for internal logs, debugging, observability.
"The runtime environment holds the key, separate from the agent process but inside the same operator's trust boundary." Operator-signer. Class: deployment-internal attestation. Stronger than self-attestation, weaker than independent.
"An entity outside the operator's deployment, with its own threat model and key management, holds the key." Independent attestor. Class: third-party witness. Strongest weight, hardest to deploy.

Pipelock implements the second shape today when deployed correctly. The binary is a separate process from the agent and signs receipts with its own key. The deployment puts that process behind a capability boundary the agent can't cross: Linux UID separation, a separate Kubernetes pod with NetworkPolicy, or equivalent isolation. The boundary is deployment-enforced, not binary-enforced. A deployment that runs Pipelock as the same UID as the agent collapses shape two back into shape one. Independent third-party attestation (a hosted verifier outside the operator's deployment) is on my roadmap, not shipped, and there are open questions about who the third party is and what trust relationship the operator has with them.

Pipelock doesn't require in-agent signing. It earns the second class when deployed behind a capability boundary the agent can't cross. The signing key lives in a process the agent has no access to. That's the property the architecture protects, and it depends on how you deploy.

What this means for posture

Three decisions follow if signed receipts are part of your AI agent posture:

Trust scope. Only the team running the agent? In-process is plenty. Auditors, customers, or regulators in the picture? Receipts have to survive the "who signed" question.

Threat model. A compromised agent (prompt injection, tool poisoning, jailbreak) producing fraudulent self-signed receipts is something in-process can't address. Out-of-process can, because the agent never holds the key.

Capability separation budget. Out-of-process signing means the signer lives in a different trust boundary from the agent. Linux: separate UID. Kubernetes: separate pod. Managed runtime: maybe a different service entirely. Each adds operational footprint. That footprint is the cost of the evidentiary property.

Decisions cascade. Internal use only gets you in-process. Audit-bearing evidence needs out-of-process. Third-party verifiable needs an attestor outside your deployment.

What I've shipped

The reason this question is worth asking right now is that the open-source pieces for verifying mediator-signed receipts exist today. Anyone can pull them and check.

Audit Packet v0 JSON Schema lives at pipelab.org/schemas/audit-packet-v0.schema.json. It pins the shape of a procurement-grade evidence bundle: receipt chain, verifier output, scanner config snapshot, posture metadata, plus tamper-detection cross-checks (claimed totals vs actual chain totals, claimed root hash vs actual, claimed final sequence vs actual).
Three verifier codebases in the Pipelock repo, none dependent on the others. Go is the reference implementation. Both the built-in pipelock verify-receipt subcommand and the standalone pipelock-verifier binary share it. TypeScript uses @noble/ed25519 and ajv. Rust carries its own canonical JSON, schema validation, Ed25519, chain replay, and packet cross-checks. All three cross-validate against the same fixtures at sdk/conformance/testdata/.
Python verifier ships in a separate repo: pipelock-verify-python. 0.1.x on PyPI covers ActionReceipt v1. 0.2.0 covering EvidenceReceipt v2 plus RFC 8785 JCS and an RFC 9421 well-known signing-key directory is prepared in the repo, PyPI publish pending.
Standalone pipelock-verifier is self-contained. No network surface, no proxy, no scanner. CI runners and auditors can drop it into an isolated environment with --offline and verify a packet against the public schema, zero vendor dependency.

That's the point of putting all of it on the public surface. An attestation that depends on the agent's trustworthiness is a different thing from an attestation that survives a question about the agent. The verifier path is open source so anyone, including people who have no trust relationship with me at all, can check whether a receipt is honest.

The honest summary

Signed receipts are a real defensive property. The signing architecture decides what class of evidence the receipts are.

In-process signers produce self-attestations. Receipts are signed, the chain holds, the cryptography is sound. The trust model is "the agent is honest about its own decisions."

Out-of-process signers produce attestations from a different actor. The trust model is "the proxy is honest about what the agent did." A proxy can be honest about a dishonest agent, which is what makes the receipts evidence and not self-report.

Both classes have a role. Knowing which class your vendor ships is the input to your posture decision. The architectural question is upstream of features, performance, and price. It decides what the receipts are for. Worth asking before signing anything, including the contract.

If you're evaluating Pipelock or anything else against this rubric, the conversation starts at "where is the signing key held." The answer should be one sentence, and it should be the same sentence regardless of who you ask on the vendor's team.

Three Things "Set HTTPS_PROXY" Cannot Stop

Josh Waldrep — Wed, 13 May 2026 12:44:55 +0000

Three bypass shapes for HTTPS_PROXY-only agent egress controls. The kernel does not enforce any of them. Each is reachable in a default Linux process unless additional kernel-level controls are applied.

This post is the listicle companion to Politeness vs Enforcement. The framing post is the long argument; this one is the short list of specific bypasses to know about and the kernel-level rule that would close each.

1. Subprocess that clears the environment

An agent process spawns a subprocess. If the spawn does not pass HTTPS_PROXY through, the subprocess has a different environment than its parent. Cooperative HTTP libraries in the subprocess never see the variable, so they never route through the proxy.

The bypass takes two lines:

import subprocess
subprocess.run(["curl", "https://example.com/"], env={})

The empty env={} clears every variable, including the proxy hint. curl runs, dials directly, the kernel sees an outbound connection from the agent UID, the connection succeeds.

If the agent is running as the same UID as the operator and the proxy, the kernel has no rule that distinguishes "should go through proxy" from "should not." Every outbound TCP from the agent UID is allowed. The proxy does not see the request, the audit log does not record it, the dashboards do not flag it.

The kernel-level fix: put the agent process on a UID that is denied direct internet by an nftables rule. Loopback to the proxy is allowed; everything else is dropped at the kernel level. The subprocess inherits the agent UID and inherits the same restriction. Clearing the environment changes the agent's behavior, not the kernel's.

The same fix in Kubernetes: separate the agent into its own pod with a NetworkPolicy that only allows egress to the proxy pod's service. The agent container can spawn subprocesses, the subprocesses share the agent pod's network namespace, the NetworkPolicy applies to all of them.

2. Transport that does not consult HTTPS_PROXY

HTTPS_PROXY is documented to affect HTTP and HTTPS client libraries. Other transports do not consult it:

Raw TCP sockets. A direct socket.connect((host, port)) in any language is unaffected by the variable. Agents that use lower-level networking primitives (gRPC over a custom transport, custom protocol clients, anything calling socket() directly) skip the proxy.
UDP. DNS queries, NTP, syslog, custom datagram protocols. The variable has nothing to say about UDP.
QUIC and HTTP/3. QUIC runs over UDP. Most QUIC client libraries do not consult HTTP_PROXY-style variables, even if the underlying request is HTTP/3-shaped. Browsers under modern Chromium can fall back to HTTP/3 for performance reasons; agent libraries that include QUIC support might prefer it.
ICMP. Raw ICMP from a process with the right capabilities. Less common in agent code, but listed for completeness.

A coding agent that wants to ping a host runs ping. The kernel sends ICMP. The proxy is not in the path. A coding agent that uses a gRPC client to talk to a service over HTTP/2 might or might not respect proxy variables depending on the library. A coding agent that constructs a DNS query directly to encode data into the hostname (a known DNS-exfiltration pattern) is exfiltrating bytes via a transport the proxy does not see.

The kernel-level fix: the same nftables rule that drops outbound from the agent UID drops every transport, not just TCP. UDP, ICMP, raw sockets, all get the same drop. The rule is meta skuid <agent_uid> drop, with allow-list exceptions only for loopback and DNS to a local resolver. Anything not on the allow list is gone.

The Kubernetes equivalent is a NetworkPolicy on the agent pod. NetworkPolicy is per-protocol and per-port: the egress section can specify TCP and UDP individually. A correctly tightened policy allows TCP to the proxy service on the proxy port and nothing else. UDP does not need to be allowed for an agent that does not need to send UDP.

3. Service whose hostname matches NO_PROXY

NO_PROXY is the variable that tells cooperative HTTP clients to skip the proxy for matching destinations. Operators set it to keep cluster-internal traffic from hairpinning through the proxy. Common patterns:

NO_PROXY=127.0.0.1,localhost,10.0.0.0/8,*.cluster.local
NO_PROXY=*.internal.example.com,internal-api

The bypass surface is whatever the operator has put on the list. If a NO_PROXY-listed destination has its own outbound, the agent has just reached the internet through it.

A specific shape: a Kubernetes cluster has an internal LLM gateway service at llm-gateway.cluster.local. The agent's NO_PROXY includes *.cluster.local. The agent makes an HTTP call to https://llm-gateway.cluster.local/chat/completions. The proxy never sees the call. The LLM gateway receives the call, forwards it to a third-party LLM API, and returns the response. From the agent's perspective, this is a successful API call. From Pipelock's perspective, no scanning happened.

The same shape applies to:

An internal MCP server with its own outbound, where the agent calls the MCP server directly.
An internal proxy or gateway, where the agent uses the gateway to reach external services.
An internal logging or metrics service, where the agent can encode data into log lines that get shipped to an external aggregator.

Any service in the NO_PROXY list with its own outbound is a path through the proxy boundary that the proxy cannot see.

The kernel-level fix: route every cluster-internal service through the agent firewall too. The cost is a hop of latency on intra-cluster calls. The benefit is that the agent firewall is the only egress path for the agent process, regardless of whether the destination is internal or external. NO_PROXY on the agent is reduced to 127.0.0.1,localhost. Everything else, including cluster services, goes through the proxy.

This requires the cluster's internal services to accept proxied requests, which most do. It also requires the proxy to be configured to scan internal traffic the same way it scans external traffic. The architectural pattern is "the agent firewall is the agent's only door, and the door applies the same scanning regardless of which floor you are going to."

The pattern

All three bypasses share a shape: the agent firewall lives in the application layer, and the agent process can route around the application layer in ways the application layer does not see. The kernel sees them all but has no rule to apply.

The kernel-level fixes for all three are versions of the same answer: put the agent process on an identity (UID or pod) that the kernel denies direct egress to anywhere except the proxy. The kernel does not care what the agent intended, what variable was set, what transport was chosen, or what hostname was the destination. The kernel sees the source identity, checks the rule, and drops or accepts.

That is what enforcement looks like. The application-layer hint, however well-intentioned, is policy. The kernel rule is the control.

Per-bypass severity

Not every bypass is equally dangerous in every deployment:

The subprocess env-clear bypass requires the agent to deliberately spawn a subprocess with a clean environment. Most coding-agent workflows do not do this by default, but a prompt-injection that says "run this command in a fresh shell" can produce it.
The non-HTTP-transport bypass is rare in practice. Most agent workloads are HTTP-shaped. DNS exfiltration is the realistic version of this class for agents that handle high-value secrets.
The NO_PROXY bypass is common in Kubernetes deployments. Internal cluster services are often wired into NO_PROXY, and any of them with outbound access can expand the agent's effective reach.

A defense posture worth shipping does not pick one of these to address. All three are reachable in a default Linux configuration. The kernel-level fix that addresses them is one fix, not three: deny direct egress from the agent identity at the kernel layer, allow only loopback to the proxy.

What to do with this list

If you operate AI agents and your egress story is HTTPS_PROXY plus a hopeful NO_PROXY, audit each of the three bypasses on your own environment. The audit takes minutes:

# 1. Subprocess env-clear
sudo -u <agent-uid> env -i curl https://example.com/

# 2. Non-HTTP transport
sudo -u <agent-uid> nc -z 1.1.1.1 53

# 3. NO_PROXY service
sudo -u <agent-uid> curl http://<no-proxy-service>/

Each command that returns success when you expected blocking is a bypass to fix. The fix shape is described above, in Politeness vs Enforcement, and in the three-UID containment pattern for workstations and per-pod NetworkPolicy for clusters.

The agents are going to find these bypasses on their own. Better to find them first.

Capture and Replay: Testing Security Policy Without Production Risk

Josh Waldrep — Tue, 12 May 2026 14:50:44 +0000

You cannot change a security policy in production without breaking somebody's workflow somewhere. Every allowlist update, every new DLP pattern, every tightened SSRF rule disagrees with at least one request that worked yesterday. The cost of finding the disagreements after promotion is the cost of a rollback under pressure: the agent fleet is paging, the dashboard is red, and the operator is editing YAML at 2 AM.

Capture and replay shifts the disagreements left. The proxy records what it saw and what it decided. A candidate policy gets replayed against the captured journal. The diff between live and candidate verdicts becomes a report. The operator reviews the report before promotion, not after. By the time the new policy goes live, the only surprises are the ones the operator already accepted.

The same deployment lesson appears in subPath ConfigMap Mounts Don't Hot-Reload: changing a policy object is not enough. You need proof that the running enforcement path will see and apply the change.

Pipelock's learn-and-lock pipeline is this pattern, with signed receipts on the lifecycle steps that change contract state. This post is the architecture, the design choices behind the cardinality cap and fidelity gates, and the lifecycle commands an operator runs.

The four phases

Learn-and-lock has four phases. Activation is a two-step lifecycle inside the fourth. Each produces evidence the next step depends on:

Observe. The proxy records URL verdicts, response verdicts, DLP verdicts, MCP tool-policy verdicts, and tool-scan verdicts to a JSONL journal. Each record carries the input summary that produced the verdict, the verdict itself, and a reference to the session and trace context. Encrypted payload sidecars can hold exact payloads when raw escrow is configured.
Compile. The compile phase takes the journal and produces a candidate contract: a normalized description of the URL paths, MCP method shapes, and argument patterns the proxy observed. Path normalization caps cardinality so a contract built from a million /users/{id} requests does not produce a million rules. Operators can pin or split paths the normalizer collapses incorrectly.
Shadow. The candidate contract gets replayed against more traffic. The proxy continues to enforce the live policy; the shadow contract produces verdicts in parallel. The output is a delta receipt: a signed record of every request where the candidate disagrees with the live policy, with severity and reason for each disagreement.
Activate (two steps). Ratify — the operator reviews the candidate and marks rules as enforce, capture-only, or reject. Ratification emits a contract_ratified evidence receipt. Promote — the ratified contract becomes active policy. Promotion writes signed intent and committed receipts that identify the target manifest, prior manifest, operator key, and selector.

The sequence is observe -> compile -> review -> shadow -> ratify -> promote. Each step produces evidence the next step depends on. The contract that gets promoted is the one that was ratified, which is the one whose shadow report the operator reviewed, which was generated from observed traffic. There is no implicit step where a config edit slips into the active state without going through the chain.

What capture actually records

The capture journal is a stream of typed records, one per verdict site. The record fields cover the surface needed to replay deterministically:

The transport. Fetch, forward, CONNECT, WebSocket, MCP stdio, MCP HTTP, body scan. Replay needs to know which scanner to invoke.
The subsurface. A label for the specific hook site, like forward_url or dlp_mcp_input. Replay uses the subsurface to dispatch to the right scanner method.
The input. A replayable summary in the JSONL record, with optional encrypted sidecar payloads when raw escrow is configured.
The verdict. Allow, block, warn, or strip, with the matching pattern names and any classification details.
The session and trace context. Replay of stateful surfaces (rate limiting, cross-request exfiltration, MCP tool baselines) needs the order and grouping of records to match production.

The record envelope reuses the existing Pipelock recorder, which gives the journal hash chaining, signing, retention, and rotation by default. The capture schema is versioned separately from the recorder envelope so the journal format can evolve without breaking older recordings.

Why path normalization caps cardinality

A naive compiler would produce one rule per observed URL path. For a working agent fleet that is millions of distinct paths, most of which are the same shape with different IDs. The compiled contract would be useless: too long to review, too brittle to maintain, too slow to evaluate.

Path normalization collapses paths with structural similarity. A request to /users/123/profile and a request to /users/456/profile collapse into /users/{id}/profile. The normalizer is conservative; it only collapses when the variable component looks like an identifier (numeric, UUID-shaped, or a short token), and it caps the cardinality so a path with too many distinct values stays unnormalized. The cap exists because some paths really do have a small fixed set of values where each value is its own rule, and collapsing those would be wrong.

Operators can pin a path that should not be normalized (/admin is /admin, not /{role}) or split a path that should be more granular than the default (/api/v1/foo is different from /api/v2/foo, even though the normalizer might collapse them). The pin and split commands are the operator's escape hatch for cases where the heuristics get the wrong answer.

The contract that comes out of the compile phase is something a human can review. Hundreds or low thousands of rules, not millions. Each rule corresponds to a request shape the proxy actually saw. The reviewer reads the contract and asks "is this the policy I want," instead of writing one from scratch.

Why fidelity gates exist

Replay produces verdicts against captured records. Replay is only useful if the verdicts it produces match the verdicts the live proxy would produce given the same inputs. The fidelity gates are the checks that hold the replay engine to that standard.

A simple example: an SSRF check during replay needs to see the same DNS resolution as the live proxy. If replay re-resolves a hostname and gets a different IP, the SSRF verdict changes for reasons that have nothing to do with the policy under test. Fidelity gates handle this by either pinning the resolution from the captured record or skipping the SSRF check during replay and reporting the path as "stateful, not replayable."

A harder example: cross-request exfiltration depends on session history. Replay has to process the journal in order so the second request in a sequence sees the first. The replay engine reads the journal as an ordered stream, not a parallel batch, to preserve the history. The fidelity gate flags any replay that deviates from order.

The list of stateful surfaces is small but real: URL rate limiting, URL data budget, MCP chain detection, MCP session binding, adaptive escalation, HITL overrides. Each has its own fidelity gate. Replay against a journal where the order is wrong, the session is missing, or the surface is skipped is replay against a different policy than the live proxy ran. The gates make the difference visible.

Shadow delta receipts

The output of the shadow phase is a delta receipt, signed with the same chain as every other Pipelock receipt. The receipt has, for each request where the candidate disagreed with the live policy:

The request shape (transport, subsurface, normalized input).
The live verdict.
The candidate verdict.
The classification of the disagreement: false-positive, false-negative, allowed-now-blocked, blocked-now-allowed, or neutral.

The receipt does not contain the raw request bodies. Those stay in the encrypted payload sidecar. The receipt is a summary an operator can review at scale, with pointers into the sidecar for the cases that need closer inspection.

A contract that produces zero deltas is one that exactly matches live. Useful for a baseline, less useful as a candidate for change. A contract with too many deltas is too aggressive a change for one promotion; the operator can split it into multiple smaller changes, each with its own shadow report. The right delta count depends on the change being made; the report makes the count visible so the operator can decide.

The lifecycle commands

Pipelock ships the lifecycle as CLI commands:

pipelock learn observe runs observation and writes capture evidence.
pipelock learn compile builds a signed candidate contract.
pipelock learn review renders deterministic review markdown.
pipelock learn shadow replays captured observations against the candidate and writes a shadow report.
pipelock learn diff compares two shadow JSON reports.
pipelock learn ratify records operator approval choices.
pipelock learn promote makes a ratified contract active.
pipelock learn rollback returns to a previously accepted manifest.
pipelock learn forget removes a rule from a candidate, signs the reduced candidate, and writes a tombstone.
pipelock learn split and pipelock learn pin fix over-broad path normalization before ratification.

The runtime evaluation hooks for active contracts run in the proxy on every request once promotion lands. The lifecycle receipts cover the promotion moment; the verifier can prove the contract running in production matches the contract that was ratified. If the artifact on disk has been modified since signing, promotion refuses.

Why this beats blind config edits

Three failure modes that capture and replay catches before promotion:

Tightened DLP that breaks a real workflow. A new pattern matches a string the agent fleet was sending in legitimate requests. Without replay, promotion produces a wave of failed requests. With replay, the delta receipt shows every request the new pattern would have blocked, and the operator either keeps the pattern (blocking real traffic the operator now classifies as exfiltration) or refines the pattern (so the legitimate string stops matching).
Loosened allowlist that exposes a path. An operator adds a domain to the allowlist for a new integration. Replay shows the new domain catches not just the integration's traffic but a category of requests the operator did not anticipate. The allowlist gets a more specific rule before promotion.
MCP tool policy change that breaks a tool chain. A tool gets added to the deny-list because the operator thinks it is unused. Replay shows the chain detector firing on a sequence the new deny-list would break. The operator either accepts the breakage (the chain was the use case being deprecated) or revises the deny-list (the chain is in active use).

In each case, the value is shifting the discovery from "production now" to "before promotion." The cost difference between those two timings is the cost of a paged-on call. Replay's purpose is to never have that call.

What this enables next

Once the lifecycle is in place, the same capture journals power adjacent work:

Regression testing. A change in scanner code can be replayed against historic journals to confirm the verdicts have not drifted.
Compliance evidence. A captured journal plus a signed contract is a record of "this is what we observed and this is what we approved."
Audit trail. The signed receipts cover contract lifecycle decisions, so an auditor can verify the chain from observation to promotion.

None of those uses required new infrastructure. They fall out of the same capture-and-replay foundation, with the existing receipt chain providing the integrity story.

If you are running a security policy in production today and your change-management story is "edit YAML, hot-reload, hope," capture and replay is the upgrade path. The change-management story becomes "observe, compile, review, shadow, ratify, promote, with signed receipts around lifecycle decisions." The proxy stops being a place where mistakes happen and starts being a place where mistakes are caught before they happen.

Pipelock Agent Egress Control: the missing CI primitive for AI agents

Josh Waldrep — Sun, 10 May 2026 21:06:27 +0000

TL;DR. Pipelock Agent Egress Control is a GitHub Action. It runs an agent script inside a Linux network namespace, forces supported egress through Pipelock, and writes a signed Audit Packet a security reviewer can verify offline with a pinned public key. v0.1.0 shipped 2026-05-09. Apache 2.0. Marketplace listing.

Agent jobs are starting to run inside pull requests, issue triage workflows, release pipelines, docs bots, and security automation. Those jobs touch source code, secrets, package registries, cloud APIs, MCP tools, and the public internet. A normal CI log can tell you what the agent said it did. An Audit Packet is meant to prove what the network boundary saw.

This is the launch post for pipelock-agent-egress-action v0.1.0, the first tagged release.

What it does

Wraps a bash script in a Linux network namespace with iptables enforcement. The script runs as a non-root pipelock-agent user with sudo denied and capabilities dropped. Pipelock listens on the loopback interface as a separate non-root pipelock-host user. All HTTP, HTTPS, and WebSocket traffic from the script is routed through Pipelock at the kernel level. Direct network, DNS, and raw TCP are blocked inside the namespace.

After the run, the action writes a signed Audit Packet: receipt chain, verifier output, policy hash, decision counts, scanner config snapshot, posture metadata, and a human-readable summary. The receipts are signed by Pipelock at the network boundary. The agent process never signs anything. With a pinned public key, a third party can verify the chain offline.

Quick start

- name: Install Pipelock
  run: |
    curl -fsSLO https://github.com/luckyPipewrench/pipelock/releases/download/v2.4.0/pipelock_2.4.0_linux_amd64.tar.gz
    curl -fsSLO https://github.com/luckyPipewrench/pipelock/releases/download/v2.4.0/checksums.txt
    grep 'pipelock_2.4.0_linux_amd64.tar.gz' checksums.txt | sha256sum -c -
    tar -xzf pipelock_2.4.0_linux_amd64.tar.gz
    sudo install -m 0755 pipelock /usr/local/bin/pipelock

- name: Pipelock Agent Egress Control
  uses: luckyPipewrench/pipelock-agent-egress-action@v0.1.0
  with:
    script-path: ./scripts/agent.sh
    pipelock-bin: /usr/local/bin/pipelock
    config: .pipelock/ci.yaml
    audit-packet-dir: pipelock-audit-packet

What is in scope for v0

On supported Linux runners (ubuntu-latest):

HTTP and HTTPS egress through Pipelock at the kernel level.
WebSocket destinations contained at the kernel level. Frame-level scanning when the script uses Pipelock's /ws?url=... proxy path.
Direct network, DNS, and raw TCP blocked inside the namespace.
Non-root execution for both the action script and the listener, with sudo denied and capabilities dropped.
Signed receipt chain verified at end-of-run; Audit Packet written locally.

What is out of scope for v0

Disclosed in the release notes rather than papered over:

Nested Docker workloads launched from the action script.
GitHub service containers.
Sibling steps in the caller workflow that run outside the action boundary.
macOS and Windows runners.
SSH egress (planned for v0.2).
MCP stdio (no network surface to enforce on).
MCP HTTP/SSE without explicit Pipelock MCP listener wiring.

If a path is not under the Pipelock control point, the action says so. No silent fallback.

Why a separate action from Pipelock Scan

The existing Pipelock Scan action is a static check on PR diffs: it grep-and-classify finds credential leaks and injection patterns in the changed code. Agent Egress Control is a runtime container: it executes an agent script inside Pipelock-enforced isolation and produces evidence about the run.

Different primitives, different scope, different SemVer cadence. Both belong on a CI workflow that runs AI agents. The two-action split keeps each artifact's CISO due-diligence story narrow.

Where to go next

Politeness vs Enforcement: Why "Set HTTPS_PROXY" Isn't a Security Control

Josh Waldrep — Sat, 09 May 2026 23:11:07 +0000

If your agent egress story is "we set HTTPS_PROXY to point at the proxy," the proxy is asking nicely. The kernel has no opinion on what the agent does next.

This post is about the line between asking nicely and actually preventing the thing. The line is whether the kernel agrees with you. Everything on the wrong side of that line is policy. Everything on the right side is a control.

The bestiary

Plenty of common AI security controls live on the asking-nicely side. A short catalog:

HTTPS_PROXY, HTTP_PROXY, NO_PROXY environment variables. Cooperative libraries read them. Uncooperative subprocesses ignore them. There is no kernel hook that says "this UID's traffic must traverse 127.0.0.1:8888."
Tool deny-lists at the model layer. "Do not call curl." The model agrees and then writes a Python script that imports requests. The deny-list never sees requests.
System prompts that say "do not exfiltrate." A system prompt is text inside a context window. The text shapes the model's output distribution. The model is free to be wrong, and a prompt injection further along in the context can rewrite the rules.
Allowlists in the agent's own configuration. A configuration the agent process can read, the agent process can edit. Anything under ~/.config is asking nicely with extra steps.
MCP server lists in ~/.mcp.json. Same shape. The agent picks which MCP servers to talk to. If the agent picks a different one, nobody stops it.

The asking-nicely controls all share a pattern: the agent process is the entity expected to comply. The check, the variable, the deny-list, the allowlist, all live somewhere the agent can read or modify. Compliance is voluntary.

The enforcement-side controls share a different pattern: the agent process is the entity being constrained. The check lives somewhere below the agent. A short catalog of those:

NetworkPolicy in Kubernetes. The CNI drops traffic that does not match the policy. The pod cannot opt out.
nftables rules with meta skuid matches. The kernel drops traffic from the agent UID that does not match the egress rule. The agent process cannot remove the rule.
Linux user namespaces with no route to the internet. A process inside the namespace has no IP routes that lead outside. Setting HTTPS_PROXY would be a no-op for traffic that has nowhere to go.
Capability separation by UID. The agent UID has access to a proxy on loopback. The proxy UID has access to the internet. The agent UID does not.

The line is the kernel. Below the line, no agent runtime choice changes the outcome.

A real bypass that landed

A working coding agent on a real Linux host had HTTPS_PROXY=http://127.0.0.1:8888 set as policy. Every browser, every CLI, every script the operator ran picked up the variable and routed through the proxy. The proxy scanned the traffic, the audit log filled up, the dashboards showed coverage.

The agent decided it needed to refresh an OAuth token. It spawned a subprocess. The subprocess did not inherit the proxy environment, by design or by accident. The subprocess dialed the OAuth endpoint directly. The OAuth refresh succeeded. The proxy never saw the request.

The DLP scanner did not run. The audit log did not record the request. The dashboards still showed compliant traffic for the requests that used the proxy. The operator was looking at metrics that confirmed partial compliance with a policy the agent had already routed around.

Nothing about this story requires the agent to be malicious or compromised. The agent did the thing agents do: it ran a process. The process did the thing processes do: it talked to the network. The kernel, watching the whole thing, had no policy to apply because the policy lived in an environment variable inside a process that no longer existed by the time the dial happened.

This is not theoretical in modern agent deployments. The fix is not "set the variable harder."

What enforcement actually takes

On a workstation that runs a coding agent, an AI CLI, and a browser-driver alongside the operator's normal applications, a kernel-enforced boundary takes a few specific things:

The agent runs as a different Linux UID than the operator and the proxy.
An nftables chain matches meta skuid <agent_uid> and drops everything except DNS to loopback.
A separate nftables rule allows the proxy UID to reach the internet, because the proxy is the agent's only legitimate exit.
The operator's UID is unaffected, so the desktop continues to work normally.

That last point is load-bearing. If enforcement breaks the operator's daily flow, nobody runs it. The three-UID model exists because two UIDs is not enough: the proxy needs internet, so the proxy UID has internet, so an agent running as the proxy UID inherits internet. The agent UID has to be a third identity that can only see loopback.

In Kubernetes, the same idea takes pod separation. NetworkPolicy is per-pod, not per-container. Every container in the same pod shares one network namespace, so a NetworkPolicy cannot say "agent container has no internet, proxy sidecar has internet." The proxy has to live in its own pod, and the agent pod gets a NetworkPolicy whose only egress is to the proxy pod's service IP.

Both stories rhyme. The kernel layer below the agent is doing the refusing. The agent's runtime choices do not reach the kernel.

Why this distinction matters

If you are evaluating an agent security tool, ask the vendor what happens when the agent ignores the tool. The answer separates policy from enforcement.

A vendor whose answer is "the agent is configured to use our proxy" is selling policy. That is fine if you trust your agent. If you are running production AI assistants that handle credentials, parse untrusted content, or execute attacker-controllable instructions, you should not.

A vendor whose answer is "the agent process cannot reach the internet without going through us, because the kernel says so" is selling enforcement. The implementation might be Kubernetes NetworkPolicy, Linux UID separation, or a managed-runtime environment that controls the egress. The detail varies. The shape is consistent: the agent is the entity being constrained, not the entity expected to comply.

This is not a critique of asking-nicely controls in general. They have a place. A correctly-set HTTPS_PROXY is real coverage for compliant traffic. A clear deny-list raises the bar for casual misuse. They are policies, and policies are useful.

They are not controls. Treating them as controls produces dashboards that confirm a policy the agent has already routed around.

The fix model in two sentences

On Linux: put the agent process on a UID the kernel firewall denies direct internet. Allow only loopback to the proxy.

In Kubernetes: put the proxy in a different pod from the agent, and write a NetworkPolicy on the agent pod whose only egress destination is the proxy pod's service IP.

The rest is wrappers, CA bundles, sudoers carve-outs, and operational care. Pipelock works inside both shapes today as the proxy that handles content scanning above the kernel-enforced boundary, and the agent firewall guide walks the layered model that sits on top of the egress boundary. The boundary itself is the load-bearing part. Without it, every layer above it is asking nicely.

What to do this week

If you run agents on a workstation:

Check whether the agent process and the proxy process run as the same UID. If yes, the agent has direct internet whenever it wants it.
Check whether your firewall has a rule that mentions the agent UID. If no, the policy is in HTTPS_PROXY and nowhere else.
Try the bypass. Open a shell as the agent UID, run env -u HTTPS_PROXY -u HTTP_PROXY curl https://example.com, and see what happens. If you get a 200, your enforcement layer is missing.

If you run agents in Kubernetes:

Check whether the agent container and the proxy container live in the same pod. If yes, the proxy can scan but cannot prevent.
Check whether the agent pod has a NetworkPolicy. If no, the agent has direct internet to anything inside or outside the cluster.
Try the bypass from inside the agent pod. kubectl exec in, curl https://example.com. A 200 is the same problem in a different shape.

A green dashboard with no enforcement layer below it is the most expensive form of theater in security work. Worth knowing whether you are running it.

What Pipelock Inspects, And What Tool Policy Inspects Instead

Josh Waldrep — Sat, 09 May 2026 21:22:45 +0000

A wire-only proxy scans wire bytes. Opaque media bytes pass through the wire layer untouched. Anyone evaluating an agent firewall should know which class of attacks gets caught at which layer, because pretending the wire layer covers everything is the wrong sales pitch and the wrong mental model.

This post is the layer split. Pipelock has two inspection layers that operate at different abstraction levels, and the marketing-friendly claim "we scan everything" is true for some shapes of attack and false for others. Saying so plainly is more useful to a buyer than saying nothing.

The wire layer

Pipelock's wire layer scans bytes as they cross the proxy. Every transport Pipelock supports gets the same set of scanners:

HTTP forward proxy. CONNECT and absolute-URI requests, request and response bodies on intercept paths, headers on every transport.
MCP stdio. JSON-RPC frames on the subprocess pipe, both directions.
MCP HTTP and SSE. JSON-RPC frames over HTTP, including streaming text/event-stream responses scanned per-event.
WebSocket. Frames in both directions, fragment reassembly, A2A envelope payloads.
Reverse proxy. Any HTTP-shaped agent backend Pipelock fronts.

What runs on those wire bytes:

DLP. Pattern matching for credentials, secret formats, and high-entropy strings. Runs on URLs, request bodies, response bodies, headers, MCP arguments, MCP responses.
Injection detection. Multi-pass content matching for prompt injection, jailbreak patterns, and tool-poisoning shapes. Runs on response bodies and MCP tool definitions.
Redaction. Class-preserving outbound scrub for known credential and PII shapes. Runs on request bodies and MCP tools/call arguments.
SSRF. Private-IP and metadata-endpoint protection on the URL pipeline. Runs on every transport with a URL.

The wire layer is good at credentials in headers, secrets in JSON, prompt injection in responses, and DLP-pattern leaks in tool calls. It is what stops an agent from POSTing an API key to a third-party logging service or fetching a markdown file with embedded jailbreak instructions and feeding it back to the model.

What the wire layer cannot do, and what no wire-only proxy can do without strapping on a perception model, is inspect the contents of opaque media:

Images. A PNG of a credential-bearing screen has the credential rendered in pixels. The proxy sees image bytes, not text.
Audio. A voice memo of a customer complaint contains words the proxy would have to transcribe to inspect.
Video. Same shape as audio plus pixels.
PDFs. A PDF can hold images, vector text, embedded fonts, and text-as-shapes. Naive PDF text extraction misses all of it.

Pipelock could in principle add OCR, ASR, and PDF extraction to the wire layer. None of those scans is free. OCR on every uploaded image multiplies proxy CPU by an order of magnitude. Latency budgets that work for text scanning collapse under perception. The architectural choice for the wire layer is to scan what is cheap, fast, and high-fidelity: text, structured data, and protocol headers. Opaque media gets a different treatment at a different layer.

The tool layer

Above the wire layer, the agent makes deliberate choices: it picks a tool to call, it constructs an argument, it sends a JSON-RPC request that names a method and a payload. The tool layer inspects those choices, not the bytes the choices move.

Two scanners run at this layer in Pipelock:

mcp_tool_policy. Pre-execution allow / deny / redirect rules that match on tool names, argument patterns, and URL shapes inside arguments. The "screenshot a URL" tool can have a rule that blocks calls whose URL matches a sensitive host pattern. The URL is text, even when the result will be image bytes.
tool_chain_detection. Sequence matchers that operate on the order in which an agent calls tools. A pattern like "screenshot the logged-in admin page, then upload the screenshot to a third-party host" is a sequence of calls whose individual calls are each plausibly fine. The chain matcher catches the shape of the sequence.

Both scanners operate on JSON-shaped data: method names, argument keys, URL strings inside arguments. None of them inspects the binary data the methods move. They operate one level above the bytes.

The thing they catch that the wire layer cannot: an agent that wants to exfiltrate something the wire scanner cannot read. The agent screenshots a page, uploads the screenshot, and the wire scanner sees a content-type of image/png and a stream of bytes. The wire scanner has nothing to say. The tool-policy rule, watching the URL the agent passes to the screenshot tool, can see "this is a sensitive page" and block before the screenshot happens. The chain detector, watching the sequence, can see "the agent is screenshotting and uploading" and break the chain.

The two layers cooperate. Wire scanning catches the credential leak the agent attempts as JSON. Tool-policy catches the equivalent leak the agent tries to launder through a screenshot. Neither alone is enough. Both together cover the surface a wire-only or tool-only design leaves open.

The enforcement boundary still matters. Tool policy and wire inspection only see traffic that reaches them, which is why the three-UID containment pattern and Kubernetes per-pod separation are part of the same posture.

What that means for the buyer

If your evaluation rubric reads "does this tool inspect images," the honest answer is that Pipelock does not, and that is the right design. The right question to ask any agent firewall is which layer catches which class of attack:

Credentials in JSON request bodies: wire layer, DLP scanner.
Credentials in screenshots uploaded as image bytes: tool layer, mcp_tool_policy URL rule on the screenshot tool.
Prompt injection in a markdown response: wire layer, injection scanner on response body.
Prompt injection in a PDF the agent fetches and processes: tool layer, policy rule on the fetch tool, plus DLP and injection scanning on whatever text the PDF parser eventually emits in tool arguments.
Tool poisoning via deceptive tool descriptions: wire layer, MCP tool scanner on tools/list responses.
Multi-step exfiltration where each step is plausibly benign: tool layer, chain detector on the call sequence.

The pattern: structured-text scanning belongs on the wire, semantic-action scanning belongs on the tool layer. Anyone selling a tool that claims to do both at the wire layer is either running an OCR pipeline that they are not budgeting for, or claiming coverage they do not have.

What Pipelock will not do

There are two specific things Pipelock does not do, and operators should plan around them:

Pipelock does not run OCR on uploaded images. The "screenshot of a credential" scenario relies on the tool-policy rule firing before the screenshot happens, not on inspecting the image after.
Pipelock does not transcribe audio. The "voice memo of a sensitive conversation" scenario relies on the policy rule on whichever tool initiated the recording, not on inspecting the audio file.

Both gaps are honest. Both are catchable at the tool layer if the rule set is configured correctly. The MCP security tools guide and MCP tool poisoning guide walk the surrounding control surface.

What this looks like in practice

A coding agent that handles customer data has tools for reading the database, screenshotting the admin UI, and uploading files to a code-review service. Three policies catch three different attacks:

Wire DLP catches a database row that contains an API key in a JSON column being dumped to the code-review service.
Tool policy on the screenshot tool catches a prompt injection that says "screenshot the admin user list and upload it for review."
Chain detection catches the pattern "read the database, then screenshot, then upload" even when the individual calls each look legitimate.

Each policy lives at the right layer. The wire DLP runs on the bytes. The tool policy runs on the JSON-RPC arguments. The chain detector runs on the sequence. Together they cover three shapes of attack with three different mechanisms.

A buyer who insists on a single-layer answer ("we scan everything at the wire") will end up with one of those three covered and the other two leaking. A buyer who asks "what catches each shape" gets a complete posture out of two scanners that each do their job at the right level of abstraction.

The honest summary

Pipelock scans wire bytes for everything that looks like text, structured data, or a protocol header. Pipelock catches semantic actions involving opaque media at the tool layer through policy rules and chain detection. The combination is what produces real coverage. Saying "we scan everything" undersells the design and overpromises the capability. Saying "we inspect at two layers, one for bytes and one for actions" is the model that holds up under scrutiny.

If your evaluation matrix has a column for "scans images," cross it off. Add a column for "blocks tool calls that produce images of sensitive content." That column is the one that matters.

Block-Reason Headers: Make Your Security Proxy Tell You Why

Josh Waldrep — Sat, 09 May 2026 21:19:27 +0000

When a security proxy blocks an agent's request, the agent sees a 4xx and has to guess what happened. Was the destination wrong? The body? A header? Did the proxy timeout? Did the proxy itself crash? Without context, every block looks the same and the agent burns its retry budget on a single attempt's worth of information.

X-Pipelock-Block-Reason is the header Pipelock emits on every block path so the agent knows. The vocabulary is small, the format is open-spec, and the impact on operator debugging is large. This post is about the design, the schema, and why making a security proxy explain itself is good for the security posture, not bad for it.

The problem the header solves

A coding agent runs a tool that fetches a URL, parses the response, and feeds the output back to the model. The fetch goes through Pipelock. Pipelock decides the response contains a prompt-injection pattern and returns 403 with no body.

The agent has no idea what happened. From the agent's perspective:

The host could be unreachable.
The proxy could be misconfigured.
The proxy could be down.
The destination could be returning 403 itself.
The agent's request could have failed scanning.
The agent's response could have failed scanning.

Each of these has a different correct response from the agent. "Host unreachable" might mean "try a different host." "Proxy misconfigured" might mean "tell the operator." "Scanning blocked the request" might mean "do not retry this exact body." Without a signal, the agent treats them all the same way: retry, hit the same block, retry again, eventually give up.

The operator's view is no better. The audit log records the block, but correlating an agent's confused retry sequence with the proxy's decision tree means cross-referencing two log streams by timestamp and request ID. For one block in a quiet period, fine. For a fleet generating thousands of requests an hour, painful.

A structured block reason on the response solves both sides. The agent knows what happened. The operator does not have to grep two logs to figure out what the agent saw.

This is the operator-facing half of enforcement. Politeness vs Enforcement explains how to make the kernel refuse bypasses; block-reason headers explain what the agent should do after the refusal.

The header schema

The full schema lives at docs/specs/block-reason-header.md in the Pipelock repo. The shape, in one paragraph:

X-Pipelock-Block-Reason: <reason> with companion headers for version, severity, retry, and the layer that fired:

X-Pipelock-Block-Reason: dlp_match
X-Pipelock-Block-Reason-Version: 1
X-Pipelock-Block-Reason-Severity: critical
X-Pipelock-Block-Reason-Retry: none
X-Pipelock-Block-Reason-Layer: dlp

X-Pipelock-Block-Reason-Receipt is reserved in v2.4: the schema and the WithReceipt validator ship in this release, but production block paths leave the value unset until the receipt-pointer wiring lands. When populated, the value will be a 26-character Crockford-base32 ULID.

The reason vocabulary is closed. Examples by category include:

Egress. ssrf_private_ip, ssrf_metadata, ssrf_dns_rebind, domain_blocklist, scheme_blocked, subdomain_entropy, url_length, path_entropy, rate_limit, data_budget.
Content. dlp_match, prompt_injection, redaction_failure, media_policy.
MCP. tool_policy_deny, tool_poisoning, tool_chain_blocked, session_binding.
Posture. airlock_active, kill_switch_active, envelope_verify_failed, outbound_envelope_failed, redirect_scan_denied, authority_mismatch, session_anomaly, cross_request_deny, compressed_response, browser_shield_oversize.
Contract. contract_default_deny, contract_enforce_default, contract_non_default_port, contract_invalid_path, contract_observed_only.
Generic. parse_error, timeout, bad_request, pattern_unavailable, not_enabled, block_reason_overflow.

The full canonical list lives at docs/specs/block-reason-header.md in the Pipelock repo and in internal/blockreason/blockreason.go.

A block can have at most one reason code. The code is the primary signal. Severity and retry are advisory: severity tells the agent how loud to be when logging the block, retry tells the agent whether retrying with the same payload could ever succeed.

The same reason vocabulary is used for WebSocket close frames. MCP stdio does not have an HTTP header surface, so stdio blocks flow through the JSON-RPC error envelope instead.

Why the schema is small

Two design choices kept the vocabulary small:

No free-form reason strings. A free-form string would let the proxy tell the agent things like "request body contained AKIA...EXAMPLE at offset 1024." That is too useful for an attacker who controls part of the request. The agent learns exactly what the scanner saw and can adjust the next attempt to avoid the match. Closed vocabularies do not leak that detail.
No retry-after seconds. A retry hint with timing would let the agent build a retry policy that matches whatever the proxy is rate-limiting. The hint is categorical: transient says "retrying might work because the cause is not your request," none says "this exact request will never work," and policy says "this might work only after an operator changes policy."

Both choices trade specificity for safety. The agent gets enough signal to react sensibly without learning how to evade.

What changes for operators

The operator's experience changes from "decode the audit log against the agent's trace" to "read the block reason on the agent's HTTP response." Two examples:

A coding agent's CI pipeline started failing on a fetch tool. The pipeline log shows:

fetch tool: HTTP 403, body empty
agent: retrying (1/3)
fetch tool: HTTP 403, body empty
agent: retrying (2/3)
fetch tool: HTTP 403, body empty
agent: failed after 3 retries

With the header on, the pipeline log includes:

fetch tool: HTTP 403, X-Pipelock-Block-Reason: ssrf_private_ip
fetch tool: X-Pipelock-Block-Reason-Severity: critical
fetch tool: X-Pipelock-Block-Reason-Retry: none
agent: not retrying (non-retryable block)
agent: surfacing block reason to user

The agent stops retrying because retry is none. The user sees a meaningful error instead of an opaque pipeline failure. The operator does not have to decode anything.

Another example, MCP-shaped:

A new MCP server gets added to the agent's config. The agent calls a tool from it. Pipelock's tool-policy denies the call.

Without the header, the agent sees a JSON-RPC error with no actionable detail. With the header (or its JSON-RPC analog), the agent sees tool_policy_deny and can route around it: ask the user, fall back to a different tool, or surface the block reason directly. The behavior change is small in code but big in the agent's ability to recover.

Pairing with retry-budgeted agents

Modern coding agents have explicit retry budgets. A budget of three on a tool call burns fast when every block looks the same. With block reasons:

retry=none consumes zero budget on retry. The agent should not retry. Surface the block.
retry=transient consumes budget normally. The agent should retry with the same payload, possibly with backoff.
retry=policy is the operator case. The block may clear after a policy change, but the agent should not keep guessing.

For a coding agent with a budget of three retries, this means an agent that runs into a none block on attempt one fails in 100ms instead of three round trips. The retry budget is preserved for transient failures where retry actually helps.

The schema is open

The vocabulary, header format, severity values, retry semantics, and the WebSocket and MCP variants are all documented in the open-source Pipelock repo. Anyone who wants to implement this header in their own proxy can do so without a Pipelock dependency. Any HTTP client that wants to parse the header can do so without a Pipelock-specific library. The header values are short identifiers and fixed allowlist values.

If a competing agent firewall wants to adopt the same vocabulary, that is a feature, not a leak. A common reason vocabulary across vendors means agents can react sensibly regardless of which proxy is in front of them. The schema lives at docs/specs/block-reason-header.md. Take it, use it, propose changes via issue or PR.

Where this fits in the agent firewall stack

Block reasons are the debugging surface of an enforcement layer. The enforcement is the load-bearing piece: the agent firewall scans the request, the response, the headers, the tool calls, and decides allow or deny. The block reason is what makes the deny actionable.

A proxy without block reasons can still enforce, but every block is a black box. Operators get noisier audit logs and longer debugging cycles. Agents get worse retry behavior and more confused error messages. The header fixes both without weakening the enforcement.

If you are running Pipelock on main after 2026-05-02, the header is already on. If you are running an older release, v2.4 ships with the header on every block path, and the block reason headers guide walks the operator-facing changes. If you are running something else and your proxy gives you opaque 403s, this is the kind of feature worth asking for.

subPath ConfigMap Mounts Don't Hot-Reload: Silent Drift in Kubernetes

Josh Waldrep — Sat, 09 May 2026 21:17:08 +0000

A Pipelock instance running in a Kubernetes cluster watched its config file for hours while four edits to the underlying ConfigMap landed in etcd. The dashboards showed updates. The pod showed an old config. The tests that exercised the new config kept failing for reasons that made no sense.

The problem is not Pipelock. The problem is subPath. Mount a ConfigMap key as a single file with subPath, and kubelet stops propagating updates to that mount. The behavior is documented but easy to miss, and it is the load-bearing reason any service that runs hot-reload in Kubernetes needs to think about how its config volume is mounted.

The shape of the bug

A reasonable-looking ConfigMap mount in a Deployment spec:

spec:
  containers:
    - name: pipelock
      volumeMounts:
        - name: config
          mountPath: /etc/pipelock/pipelock.yaml
          subPath: pipelock.yaml
  volumes:
    - name: config
      configMap:
        name: pipelock-config

The pod gets /etc/pipelock/pipelock.yaml populated from the pipelock.yaml key of the pipelock-config ConfigMap. Other files in /etc/pipelock/ are unaffected. This is what subPath was designed for: pin one file to one path without taking over the whole directory.

The drift surfaces when you kubectl edit configmap pipelock-config, change the value, and watch the running pod for the change to propagate. It does not propagate. The running pod's view of /etc/pipelock/pipelock.yaml is the same content it had at pod creation. The kubelet has updated the underlying ConfigMap volume, but the bind mount that subPath created points at a different inode that is not part of the update path.

Restart the pod and the new content shows up. fsnotify watchers configured to react to file changes never fire because the file the container sees is, from the container's perspective, the same file it has always been.

Why the directory mount works

Drop the subPath and mount the whole ConfigMap as a directory:

spec:
  containers:
    - name: pipelock
      volumeMounts:
        - name: config
          mountPath: /etc/pipelock
  volumes:
    - name: config
      configMap:
        name: pipelock-config

Now /etc/pipelock/ contains every key from the ConfigMap. Kubelet syncs the directory periodically, subject to its sync period and ConfigMap cache. The atomic-update mechanism Kubernetes uses for ConfigMap volumes replaces a symlink that points at the current "version" of the data. Watchers need to watch the mounted directory or reopen the file path after an update, because the inode under an old file descriptor can change. With that watch shape, the service hot-reloads correctly.

The cost is that /etc/pipelock/ now belongs to the ConfigMap. If you had other files in that directory (a CA certificate from a different volume, a generated state file written by an init container), the directory mount overwrites them. You have to mount each piece into a directory of its own and let the service compose them at runtime.

The kubelet propagation lifecycle

Kubelet runs a sync loop that watches the API server for ConfigMap and Secret updates. When an update lands, kubelet writes the new content to a versioned directory inside the volume's emptyDir on the node, then atomically swaps a symlink. The container, which had been reading through the symlink, now reads the new version. The whole swap is one syscall, so readers either see the old version or the new version, never a torn state.

subPath works by computing the source path at pod creation and creating a bind mount to that specific path. The bind mount captures the inode that backs the file at that moment. Kubelet's atomic swap operates on the symlink in the volume, not on the inode the bind mount points at. The bind survives the swap and continues to point at the original inode, which kubelet never updates.

There is no documented kubelet behavior that re-evaluates a subPath mount during a ConfigMap update. The upstream issue, kubernetes/kubernetes#50345, has been open since 2017. The current state of the world is "subPath plus ConfigMap is static for running containers."

Where this hurts

Anyone running a service that watches its config file for live updates. Pipelock has fsnotify-based hot-reload on its config (SIGHUP is also supported). Other services with the same shape:

Envoy and most service-mesh proxies, which use file-based dynamic configuration discovery.
Prometheus, which reloads scrape configs on file change.
Nginx with the auto_reload patches.
Any custom service that watches its config for runtime updates.

For all of these, subPath is a silent foot-gun. The service starts up, reads its config, watches the file, and never sees the file change because the file is a frozen bind mount.

The damage scales with how much you trust your config-update workflow. If you kubectl apply a new ConfigMap and assume the running pod picks it up, every minute between the apply and the next pod restart is a minute the cluster is running stale config. For a security tool, that gap means the new policy is not enforced, the new pattern does not match, the new allowlist is not honored. The dashboards say one thing. The reality is another.

This is the Kubernetes version of the same lesson in Politeness vs Enforcement: a dashboard that confirms the policy object changed is not the same thing as a kernel-enforced runtime boundary that changed.

Two patterns that work

Mount the directory, expose the file. Mount the ConfigMap as a directory volume, and read the file by path from inside that directory. This is the simplest pattern for services that own their config directory. The cost is that the directory is now ConfigMap-shaped, so anything else that needs to live there has to come from a different mount point.

Sidecar plus emptyDir. A sidecar container mounts the ConfigMap as a directory, watches for updates, and writes the consolidated file to a shared emptyDir volume that the main container reads. This adds a moving piece, but it lets you compose multiple sources (ConfigMap, Secret, downward API, environment) into a single config file at a single path. The sidecar pattern is heavier than the direct mount but more flexible when the file's content is built from multiple inputs.

If you see subPath: and your service expects hot-reload, change the mount shape. The sidecar pattern is the fallback when a directory mount conflicts with other tenants of the target path.

How to spot this in a fleet

A Helm chart or Kustomize overlay that mounts a ConfigMap with subPath: on a service that has a hot-reload code path is the warning. Two signals to look for in code review:

A volume mount with subPath: and a target path that matches a config file the service is known to watch.
The service's startup logs include "watching config file" or fsnotify-style messages.

If both are true, the hot-reload is broken. The service will read the initial value at startup and freeze.

The other signal is operational. If your platform has a story like "edit the ConfigMap, observe the pod pick up the new value," and that story sometimes does not work, subPath is the first thing to check. The behavior is consistent: subPath mounts always freeze, directory mounts always update. The trick is that the freeze is silent.

Why this surfaced

A fleet of agent-firewall sidecars all mounted their Pipelock config with subPath:. Operators added new entries to the redaction allowlist over a span of two weeks. The ConfigMaps in etcd reflected the changes. The running Pipelock pods continued to apply the older allowlist that had been in effect when the pods started. The drift surfaced when a new test fixture failed against the live agent because Pipelock had not picked up the allowlist entry that had been added to the ConfigMap five days earlier.

kubectl delete pod fixed the symptom. Switching to a directory mount fixed the cause. The lesson generalizes: any service that hot-reloads needs a directory mount for its config, not a subPath mount. If your platform team has standardized on subPath for "single file" ConfigMap injection, audit which of those services have hot-reload code paths and migrate them.

The Kubernetes docs warn about this behavior in the ConfigMap reference, but only in passing, and the subPath documentation does not link to the warning. The upstream issue is the canonical reference for the depth of the problem and the design discussion around fixing it. Until that discussion turns into a kubelet change, the workaround is structural: avoid subPath for hot-reload-bearing files.

The Three-UID Containment Pattern for AI Agents on Linux

Josh Waldrep — Sat, 09 May 2026 21:15:31 +0000

A correct AI agent containment model on a Linux workstation needs three Linux UIDs, not two. Two UIDs has a hole. The hole is structural, not a configuration mistake.

This post shows the three-UID model with a working nftables chain, the wrapper script that drops the agent process into the right identity, and the rollback path. The model came out of porting Kubernetes NetworkPolicy containment back to a single-machine setup, and the lesson it teaches is the same: the proxy needs internet because the proxy is the agent's exit. So the agent has to be a third identity.

Why two UIDs leaks

Naive containment says: run the proxy as one UID, run the agent as another. Add an nftables rule that drops anything from the agent UID except loopback. Done.

The problem surfaces the moment you ask which UID the agent runs as. If the agent runs as the proxy UID, the agent inherits direct internet because the proxy needs direct internet. The firewall cannot tell the agent's syscalls apart from the proxy's. They are the same UID.

If the agent runs as the operator UID, the agent has the operator's whole egress story, which is "anything I want." Same problem with extra steps.

The fix is to put the agent on a UID that is neither the operator nor the proxy. Three identities. The kernel firewall has a target to drop on. The proxy keeps its internet because it has its own UID. The operator keeps a normal desktop because the rules do not touch the operator UID. The agent process loses direct internet because it runs as a UID the firewall denies.

The model in one diagram and one chain

Three Linux UIDs:

operator: the human at the keyboard. Browser, terminal, git, kubectl. Normal egress.
pipelock-proxy: the proxy daemon. Runs the agent firewall. Has internet because that is its job.
cc-agent: every agent process. Coding CLI, AI assistant, browser driver, screenshot tool. Has loopback only.

The agent UID is denied direct internet by nftables. Loopback to the proxy is allowed. DNS to loopback is allowed because the operator's local resolver still serves names. Everything else from the agent UID drops.

The rule set lives in /etc/nftables.d/50-pipelock-containment.nft:

table inet pipelock_containment {
    chain output_filter {
        type filter hook output priority filter; policy accept;

        # Loopback always accepted. This is what the agent uses to reach the proxy.
        meta oif "lo" accept
        ip daddr 127.0.0.0/8 accept

        # Operator UID stays normal.
        meta skuid 1000 accept

        # Proxy UID has internet because the proxy IS the exit.
        meta skuid 988 accept

        # Agent UID: DNS to loopback resolver, then drop everything.
        meta skuid 987 udp dport 53 ip daddr 127.0.0.0/8 accept
        meta skuid 987 tcp dport 53 ip daddr 127.0.0.0/8 accept
        meta skuid 987 drop
    }
}

That is the whole boundary. The proxy listens on 127.0.0.1:8888, the agent UID can reach loopback, the agent reaches the proxy through loopback, the proxy reaches the internet through its own UID's accepted rule. Everything else from the agent UID hits the drop and stays inside the kernel.

UIDs in the example are placeholders. The values vary by host. useradd --system picks them; capture them into your install state file once and reference by number.

The wrapper that drops the agent into the contained UID

Containment is structural, but a wrapper makes it usable day-to-day. Operators do not want to type sudo -u cc-agent -- every time they launch an agent.

Two pieces. First, a generic launcher at /usr/local/bin/cc-launch:

#!/bin/bash
set -euo pipefail
TOOL="$1"; shift
exec sudo -u cc-agent -- env \
    HOME=/home/cc-agent \
    HTTPS_PROXY=http://127.0.0.1:8888 \
    HTTP_PROXY=http://127.0.0.1:8888 \
    NO_PROXY=127.0.0.1,localhost \
    NODE_EXTRA_CA_CERTS=/etc/pipelock/ca.pem \
    SSL_CERT_FILE=/etc/pipelock/combined-ca.pem \
    REQUESTS_CA_BUNDLE=/etc/pipelock/combined-ca.pem \
    CURL_CA_BUNDLE=/etc/pipelock/combined-ca.pem \
    PATH=/home/cc-agent/.local/bin:/usr/local/bin:/usr/bin:/bin \
    "$TOOL" "$@"

Second, per-tool wrappers like /usr/local/bin/cc-claude that just exec into the launcher:

#!/bin/bash
exec /usr/local/bin/cc-launch claude "$@"

A scoped sudoers entry at /etc/sudoers.d/50-cc-agent allows the operator to drop into cc-agent without a password, but only via the launcher. The shape is:

operator ALL=(cc-agent) NOPASSWD: /usr/local/bin/cc-launch *

This is not general-purpose sudo -u cc-agent access. The operator can run cc-launch to start agents, and that is all. The kernel firewall handles the network side. The sudoers handles the launch side. Together they keep the agent in its lane.

The CA bundle is load-bearing

If the proxy intercepts TLS, the agent UID needs the proxy's MITM CA in its trust store. The wrapper environment points every common library at the combined bundle:

NODE_EXTRA_CA_CERTS for Node.js and anything that uses tls.createSecureContext.
SSL_CERT_FILE for OpenSSL-linked clients.
REQUESTS_CA_BUNDLE for Python requests.
CURL_CA_BUNDLE for curl.

The bundle gets built once with pipelock tls export plus the system roots concatenated. Rebuild whenever the proxy CA rotates. Wrappers read the bundle by path, so a refresh of the file picks up automatically.

If you skip the CA bundle, the agent's HTTPS calls fail at TLS verification, and you spend an afternoon convinced the firewall is broken when the cert chain is the problem.

The verification probes

Containment is only real if you can prove it. Run these probes after install:

# 1. Operator still has internet.
curl -s -o /dev/null -w '%{http_code}\n' https://example.com/

# 2. Proxy UID still has internet.
sudo -u pipelock-proxy curl -s -o /dev/null -w '%{http_code}\n' https://example.com/

# 3. Agent UID cannot dial direct.
sudo -u cc-agent curl -s -o /dev/null -w '%{http_code}\n' \
    --max-time 5 https://example.com/ 2>&1 \
    | grep -E '000|Connection refused|Network is unreachable'

# 4. Agent UID can reach the internet through the proxy.
sudo -u cc-agent curl -s -o /dev/null -w '%{http_code}\n' \
    -x http://127.0.0.1:8888 https://example.com/

# 5. The wrapper end-to-end.
cc-launch curl -s -o /dev/null -w '%{http_code}\n' https://example.com/

Probes 1 and 2 prove the operator and proxy paths still work. Probe 3 proves the boundary holds. Probe 4 proves the proxy path is the legitimate exit. Probe 5 proves the wrapper sets up the agent's egress correctly.

If any of these fail, the boundary is not real. Roll back, fix the offending step, try again. Half-installed containment is worse than no containment because the dashboard says "secure" and the kernel disagrees.

Rollback

The boundary is reversible. The teardown:

Disable the system pipelock unit, re-enable the user-mode unit.
Delete the nftables table and remove the rule file.
Remove the wrappers and the sudoers carve-out.
Optionally remove the system users.

If the rollback procedure does not exist as a script, the install procedure is incomplete. Production systems get installed and uninstalled. Skipping rollback design is how operators end up afraid to touch the firewall later.

Why a CLI is the natural endpoint

The procedure described above is fifteen pages long when written out as a runbook. It collapses to five commands when written as a CLI: pipelock contain install / verify / rollback / add-tool / ca-refresh. The runbook proves the model. The CLI makes the model deployable to more than one workstation.

pipelock contain is being scoped for a future release. Until it lands, the runbook is the documented procedure. Either way, the load-bearing piece is the three-UID separation. The wrappers, sudoers entries, CA bundle, and probes are operational glue around that core idea.

What this post is and is not

This post is a description of a working pattern for one Linux workstation. It is the same shape as Kubernetes per-pod NetworkPolicy: the kernel below the agent is the boundary, and the agent's runtime choices do not reach the kernel.

This post is not a substitute for content scanning at the proxy. The boundary stops the agent from leaving without going through the proxy. The proxy is what catches credential leaks, prompt injection in responses, and tool-call abuse. Containment without scanning is a tunnel with no inspection. Scanning without containment is inspection that the agent can route around. Both layers exist for a reason.

If you are running agents on a Linux box and the only egress control is HTTPS_PROXY, this is the upgrade path. The kernel will agree with you for the first time.

Webhook vs Egress: Two Architectures for AI Agent Security

Josh Waldrep — Fri, 08 May 2026 02:21:28 +0000

Two architectures keep showing up in AI agent runtime security in 2026. Both promise to stop bad agent actions before they complete. Underneath they work differently, and the difference matters when an agent goes wrong in production.

The first is webhook-based runtime monitoring. The AI platform calls out to a policy service before executing an action, the service decides, and the platform respects the answer. Obsidian Security frames its product this way. From their AI Agent Runtime Security page: "evaluate every agent against OWASP-aligned risk factors in real time, and use webhooks to intercept and stop policy-violating, high-risk executions before they complete."

The second is the network-egress firewall. A proxy sits between the agent and the network. Traffic routed through the proxy gets inspected before it leaves or before the response reaches the agent. The proxy decides allow or block based on the content of the traffic itself, not on whether the agent asked for a policy decision.

Both are real defenses. Both ship in production today. They cover different parts of the agent attack surface. I use "webhook" here as the concrete version of the broader cooperative-runtime pattern: any architecture where the platform has to surface the proposed action to the policy layer before the action happens. This post is about which boundary each one actually controls, and why a thorough posture often ends up running both.

What webhook-based monitoring sees

The webhook flow is straightforward. An agent running on a supported AI platform proposes an action: a tool call, a model invocation, a data lookup. Before execution, the platform sends a webhook to the policy service. The service evaluates the request against rules, behavioral baselines, identity, and any other signal it has. It returns a verdict. The platform respects the verdict and either runs the action or blocks it.

When this works, the coverage is tight. The policy service can correlate the action against user identity, upstream prompt, SaaS context, and anything else the platform exposes. It can refuse a high-privilege action a low-privilege user kicked off. It can flag a sudden spike in data access for one agent versus its baseline. That cross-context awareness is harder to build at the network layer alone.

The places I have seen this approach shine:

Fully-platformed enterprises where every agent runs through a managed runtime, and that runtime has a real webhook integration with the policy product
Agentforce-style closed loops where the platform vendor controls the runtime and the policy hook
Bedrock Guardrails where the cooperation flow is built into the platform itself

For those cases, webhook-based monitoring is a reasonable answer to a real problem.

Where webhook-based monitoring falls short

The hard part is that webhook-based monitoring depends on cooperation. The agent or the platform has to actually call the webhook for the policy to run. Three failure modes show up in practice.

The agent path does not cooperate. A managed platform can enforce its own hook, but only for actions that stay inside that platform's action path. A custom agent, a local coding agent, a CI workflow, or an uninstrumented MCP tool can send traffic without ever calling the policy service. Prompt injection that steers the agent into an alternate tool path bypasses the cooperation step if that path is not instrumented. I made the same point in a Help Net Security interview on Pipelock on May 4, 2026: "Most agent-security tools still need the agent to cooperate." The rest follows from that architecture. Controls that depend on agent cooperation only work while the agent keeps calling them.

The platform does not cover everything. Webhook integrations exist for the platforms vendors choose to integrate with. Custom MCP servers, internal tools, dev environments, CI agents, and on-prem deployments often have no managed platform layer at all. A cooperative-runtime product that integrates with Bedrock, Agentforce, Copilot, Vertex, Claude, and ChatGPT covers commercial managed runtimes. It does not cover the agent your platform team built last quarter that talks straight to a private MCP server.

The integration can fail open. If a webhook times out, what does the platform do? In practice it depends on the integration. Some platforms fail closed and refuse the action. Others fail open to keep the agent responsive. The end-to-end security posture is only as strong as the weakest fail-mode in the chain.

These are not theoretical gaps. The cooperation problem is the architectural shape of the model.

What network-egress firewalls see

A network-egress firewall sits between the agent and the network. The agent process holds API keys and credentials. It runs without direct internet access. The proxy holds network access and no agent secrets. When deployment is correct, direct egress is blocked and every outbound request from the agent crosses the proxy. Every response comes back through it.

The useful pattern is separation of duties: the component being checked is not also the component enforcing the check or producing the evidence.

For AI agents, that pattern means the proxy can scan traffic regardless of what platform the agent is using, as long as that traffic is forced through the proxy. Credential leaks in tool arguments, prompt injection inside MCP responses, SSRF through tool-triggered URLs, tool poisoning in MCP server descriptions: all of it is visible at the boundary because that is where the traffic crosses. There is no cooperation step for the agent to skip.

I run Pipelock as one of these. Tophant ClawVault, iron-proxy, and Agent Vault sit in or near the same request-path and credential-proxy lane, with different scopes. The shared idea is that the policy layer is outside the agent's own decision loop. A compromised agent cannot opt out by skipping a webhook because there is no webhook to skip.

The places this approach shines:

Custom agents that talk to private or internal MCP servers
Dev environments and CI agents that do not run on a managed platform
Compromised-agent scenarios where the platform integration fails or the agent is steered around it
Compliance and audit traffic where the evidence layer needs to be independent of the runtime

Where network-egress firewalls fall short

Network-egress firewalls do not see what does not cross the network. That is the limitation.

If an agent runs entirely inside Salesforce, talking to Salesforce data with Salesforce-internal calls, the egress proxy never sees that traffic. The agent reading sensitive records and acting on them inside the SaaS app stays invisible to the wire-level firewall. This is where SaaS posture management and platform-side webhook integrations earn their keep.

The same applies to embedded AI features that ship as part of a SaaS application. Microsoft 365 Copilot against SharePoint internals, Salesforce Einstein against CRM data, ServiceNow Now Assist against ITSM workflows: each runs inside the SaaS layer. An outside-the-agent proxy sees cross-SaaS traffic, not internal queries.

Network-egress firewalls also do not solve identity. They see traffic, not who the agent is supposed to be. They cannot tell you that a service account is impersonating a user, that a token has been re-used across agents, or that an agent has escalated privilege within a SaaS application. Identity and non-human identity governance is a separate layer.

They also depend on enforcement. If the agent can open raw sockets, bypass proxy environment variables, use an uncontained browser, or reach the network namespace directly, the firewall is not actually inline. The architecture only works when direct egress is closed and the proxy is the route out.

This is why the Pipelock category page lists six AI agent security boundaries, not one. Network egress and content firewall is the sixth boundary. Identity and platform governance are different boundaries. Different products. Different threat models.

The honest mapping

Webhook-based monitoring works when the agent and platform actually cooperate, the platform integration is real, and the traffic stays inside the platforms the policy product integrates with. For a Bedrock-only shop running Bedrock-only agents through Bedrock Guardrails plus a SaaS posture product on top, the webhook flow covers a lot.

Network-egress firewalls work when the agent or platform might not cooperate, when traffic crosses platforms that have no policy integration, when the agent is custom or internal, or when the evidence needs to be independent of the runtime. They only work if direct network bypass is closed. For platform teams running custom MCP, CI agents, or dev environments, that is most of the agent traffic.

Most serious deployments end up with both. They control different parts of the system. A compromised agent that bypasses the webhook is still constrained at the network egress. A SaaS-internal action the egress proxy never sees is still visible to the platform-layer policy hook.

The trap is treating either layer as the whole answer. A landing page that promises end-to-end agent security with a single architecture is selling one slice of the problem. The category split is real and the architectures cover different failure modes. Naming the boundary each one controls makes the trade-off legible.

What I tell platform teams to ask

When a platform team asks me whether to add an egress firewall, a posture product, or both, the question I push back with is usually: where does your agent traffic actually go?

If the answer is "all through Bedrock, all through Agentforce, all through Copilot, no custom MCP, no internal tools," webhook-based controls cover most of the surface. Add an egress firewall when you start running custom workflows that the platform does not see.

If the answer is "custom MCP servers, agents in CI, dev environments, internal-tool integrations," start with an egress firewall. Add platform-layer governance when managed-platform deployments scale.

If the answer is "all of the above, in production, with real PII and real money," you already need both, and the only question is which order to ship them.

That is the honest read. Webhook and egress are not competing answers to the same question. They are answers to different questions. Buyers who run their procurement that way end up with stronger postures than buyers who treat the category as one undifferentiated slot.

What CSA, SANS, and OWASP Just Told Every CISO About Runtime Agent Security

Josh Waldrep — Wed, 15 Apr 2026 02:29:18 +0000

The paper

On April 13, 2026, the CSA CISO Community, SANS, and the OWASP GenAI Security Project published "The AI Vulnerability Storm: Building a Mythos-Ready Security Program" (v0.4). The paper was authored by the CSA Chief Analyst, the SANS Chief of Research, and the CEO of Knostic. Contributing authors include the former CISA Director, the Google CISO, and the former NSA Cybersecurity Director. Many CISOs and other practitioners reviewed and edited it.

The paper describes what happens to security programs when AI compresses time-to-exploit from years to hours. It is a coordinated call to action, not a marketing document. The runtime layer it describes fits the same category as an agent firewall: egress filtering, content scanning, and containment that operates faster than a human can respond.

The stat that frames everything

Mean time-to-exploit went from 2.3 years in 2018 to approximately 20 hours in 2026. That data comes from the Zero Day Clock by Sergej Epp, based on 3,529 CVE-exploit pairs from CISA KEV, VulnCheck KEV, and XDB.

At 20 hours, patching is still necessary but no longer sufficient as a primary defense. The paper's response: shift to containment and resilience. Build the architecture that limits blast radius when (not if) something gets exploited before the patch ships.

The four priority actions that describe runtime agent controls

The paper lists 11 priority actions. PA 1 (Point Agents at Your Code) names specific vulnerability scanning tools. PA 3, 8, 9, and 10 describe runtime controls in detail but name zero tools for those actions. That gap is where the interesting question lives.

PA 3: Defend Your Agents (CRITICAL, start this month)

"Agents are not covered by existing controls and introduce both cyber defense and agentic supply chain risks. The agent harness -- prompts, tool definitions, retrieval pipelines, and escalation logic -- is where the most consequential failures occur; audit it with the same rigor as the agent's permissions." (Section IV, p.20)

The paper calls for scope boundaries, blast-radius limits, escalation logic, and human override mechanisms before deploying agents in production. And then: "Do not wait for industry governance frameworks. Define your own now."

That is unusually direct language from CSA and SANS. The message: existing security frameworks do not cover agents yet, and waiting for them to catch up is not an acceptable posture.

PA 8: Harden Your Environment (HIGH, start this month)

"Implement egress filtering (it blocked every public log4j exploit). Enforce deep segmentation and zero trust where possible. Lock down your dependency chain." (Section IV, p.21)

The log4j parenthetical matters. Log4j exploitation required outbound connections to attacker infrastructure. Organizations with egress filtering in place were not affected. Agent exfiltration works the same way: compromised agents leak data through outbound requests. If the request can't leave, the leak doesn't happen.

PA 9: Build a Deception Capability (HIGH, next 90 days)

"Deploy canaries and honey tokens, layer behavioral monitoring, pre-authorize containment actions, and build response playbooks that execute at machine speed." (Section IV, p.21)

Three things in one sentence: plant traps, watch behavior, and pre-authorize automated response so containment doesn't wait for a human to wake up and log in.

PA 10: Build an Automated Response Capability (HIGH, next 90 days)

"Examples: asset and user behavioral analysis, pre-authorized containment actions, and response playbooks that execute at machine speed." (Section IV, p.21)

The phrase "execute at machine speed" appears in both PA 9 and PA 10. That's the paper's way of saying: if your containment action requires a human clicking a button in a UI, the window has already closed.

The runtime layer they describe but don't name

Across PA 3, 8, 9, and 10, the paper describes a runtime enforcement layer that:

Filters agent egress traffic
Scans for credential leaks in outbound requests
Enforces scope boundaries and blast-radius limits
Monitors behavior and escalates restrictions automatically
Provides pre-authorized containment that triggers at machine speed
Supports canary tokens and deception
Produces tamper-evident logs for incident response

The paper names six tools for PA 1 (vulnerability scanning). It names zero tools for PA 3, 8, 9, or 10.

Pipelock is an open source runtime proxy that addresses these four priority actions. The full mapping, with verbatim quotes and framework codes from the paper's risk register, is at the Mythos-Ready Playbook page.

The Glasswing constraint

The paper also addresses the Glasswing early-access model directly (Section III, p.10):

"The world's exploitable attack surface is vastly larger than what any curated partner ecosystem can cover."

About 40 vendors and maintainers had early access to Mythos through Glasswing. The rest of the ecosystem is responding now. Open source runtime controls are deployable today without a partnership or a waitlist.

For organizations the paper describes as below the "Cyber Poverty Line" (a concept from Wendy Nather, cited in Section II), the runtime layer is free. Pipelock's scanning and enforcement features are Apache 2.0 with no feature gating.

What to do this week

The paper's own aggressive timetable says "start this week" for six of the eleven priority actions. For the runtime controls in PA 3 and PA 8:

curl -L https://github.com/luckyPipewrench/pipelock/releases/latest/download/pipelock_linux_amd64 -o pipelock
chmod +x pipelock && sudo mv pipelock /usr/local/bin/
pipelock init
pipelock claude setup   # or: pipelock cursor setup
pipelock assess

pipelock init discovers IDE configs and generates a starter configuration. The setup commands rewrite IDE configs to route MCP traffic through the proxy. pipelock assess runs a multi-step posture evaluation covering config, scanning, and MCP wrapping status.

The Mythos-Ready Playbook has the full priority action mapping, framework table, and the CISO self-assessment questions the paper asks on page 15.

Source

"The AI Vulnerability Storm: Building a Mythos-Ready Security Program." Version 0.4, April 13, 2026. CSA CISO Community, SANS, [un]prompted, OWASP GenAI Security Project. CC BY-NC 4.0.