I’m building an autonomous pentesting agent. Last week it told me an attack succeeded. It hadn’t. The bug that let it lie is one every agent builder will hit eventually, so here’s the story and the fix.
The setup
The agent can write and run custom Python when its standard tools fall short. Running model-authored code on your host is obviously a bad idea, so that path goes through a sandbox: a rootless Podman container with two phases. A test phase with no network at all (--network=none) to prove the script runs, then an attack phase with networking enabled to run it against the real target. Each phase is behind a human approval gate.
I gave it a simple goal to smoke-test the whole thing: connect to an FTP server on my lab target and print the banner.
The thing that looked like success
Both phases printed the banner:
Both phases printed the banner:
--- TEST PHASE OUTPUT (isolated, no network) ---
220 (vsFTPd 2.3.4)
--- ATTACK PHASE OUTPUT (target: 10.x.x.x:21) ---
220 (vsFTPd 2.3.4)
[GATE] Attack phase succeeded
--- TEST PHASE OUTPUT (isolated, no network) ---
220 (vsFTPd 2.3.4)
--- ATTACK PHASE OUTPUT (target: 10.x.x.x:21) ---
220 (vsFTPd 2.3.4)
[GATE] Attack phase succeeded
--- TEST PHASE OUTPUT (isolated, no network) ---
220 (vsFTPd 2.3.4)
--- ATTACK PHASE OUTPUT (target: 10.x.x.x:21) ---
220 (vsFTPd 2.3.4)
[GATE] Attack phase succeeded
Looks great. It’s also impossible.
The test phase runs with no network. A socket connect to a remote FTP server from inside that container cannot succeed — it should throw “network unreachable.” But it printed a live banner, identical to the attack phase. If the sealed room and the open room both report “I can see outside,” either the room isn’t sealed or someone taped a photo to the wall.
Two bugs wearing one costume
Bug one: the sandbox wasn’t running. The service was launching from a stale copy of the code — an older path left over from a refactor. That copy predated the sandbox entirely, so “test phase, no network” was just a label being printed. Both phases were running the same unsandboxed code on the host, twice. The banner was real; the isolation was fiction.
Fixing the service to point at the current code brought the actual sandbox online. Now the test phase correctly failed with Network is unreachable, and only the attack phase got the banner. Good.
Bug two, the interesting one: the success check was a lie detector that always said “truth.” The gate decided success like this:
ok = result.get("status") == "success"
And status was set from the exit code of the sandbox runner — not the script inside it. The runner can exit 0 (it did its job: it ran the container) while the script inside printed nothing, threw an exception, or ran against a placeholder target the model forgot to fill in. Exit 0 meant “I successfully ran something,” not “the something worked.”
This is the trap. An agent’s tools report on themselves, and they’re generous. Left alone, the agent banks these false successes as progress and moves on, building a plan on top of steps that never happened.
The fix: trust the output contract, not the exit code
The sandbox already emits a structured block for every run:
=== STDOUT ===
=== STDERR ===
=== EXIT ===
So instead of trusting the runner’s exit code, parse the sandbox’s own contract and judge the inner result:
raw = result.get("stdout", "")
inner_exit = None
for line in raw.splitlines():
t = line.strip()
if t.startswith("=== EXIT ") and t.endswith("==="):
n = t[9:-3].strip()
if n.isdigit():
inner_exit = int(n)
inner_stdout = ""
if "=== STDOUT ===" in raw and "=== STDERR ===" in raw:
inner_stdout = raw.split("=== STDOUT ===", 1)[1].split("=== STDERR ===", 1)[0].strip()
if inner_exit is None:
fail("no exit marker from sandbox")
elif inner_exit != 0:
fail("script exited non-zero")
elif not inner_stdout:
fail("no output; no-op treated as failure")
elif "TARGET_IP" in code: # model left a placeholder
fail("placeholder target, not a real run")
else:
succeed()
Note there’s no regex here. My first version used one, and a stray trailing newline in the output made the pattern miss, which flipped the guard to a false negative — now it failed real successes. Plain string splitting on the markers is boring and it doesn’t care about whitespace. Boring won.
The takeaways
If you’re wiring tools into an agent, three things earned their keep:
Exit code 0 is not success. It means a process ended without crashing. Whether the task succeeded is a separate question your tool has to answer explicitly. Define what “worked” means — non-empty output, an expected marker, a parsed field — and check for it.
A test that can’t fail isn’t testing anything. The whole bug was invisible until the isolated phase started correctly failing. If your “isolated” mode never errors, it isn’t isolated.
Watch for placeholder output. Models will happily hand you TARGET_IP or example.com and a green checkmark. Treat obvious placeholders as failure, not success.
The guard is deliberately strict: no-op, error, empty output, or placeholder all report failure. I’d rather the agent retry a real success it undercounted than march forward on a fake one it believed.
Verified end-to-end against a local lab target: test phase blocks the network, attack phase returns the real banner, and a deliberately empty script now correctly reports failure instead of success.
If you’re building in this space, I’d genuinely like to hear how you’re validating tool results — the exit-code trap feels like something a lot of people hit quietly.
Top comments (1)
Lets have a discussion.