I built an autonomous multi-agent AI pentester — and why it's not another GPT wrapper

Most "AI pentester" projects are a single LLM in a while-loop with a shell. You
give it a target, it runs commands until it decides it found something. That's
how you get confident nonsense — a model that writes a beautiful vulnerability
report for a bug that doesn't exist.

I wanted the opposite: an engine where a finding has to be earned. So I built
OIHK — an autonomous, multi-agent
AI penetration-testing engine. It's open source (MIT) and runs locally.

Not one model in a loop — a team of agents

OIHK is a multi-agent engine. A root planner delegates to specialist agents —
recon, discovery, validation, reporting — that all share two things:

• a versioned scan plan (optimistic concurrency, revision history, resume), and
• an evidence ledger (immutable execution records).

Agents don't coordinate by vibes in a chat log. They claim explicit plan steps,
attach real evidence, and update state through a revisioned store. The root can't
close a run while critical work is still open.

The rule I care about most: no evidence, no finding

Here's the design decision the whole thing is built around:

An LLM writing a convincing PoC string is not a finding.

A finding requires a real, successful, governed tool execution and a
separate validation record. Only a validation agent can turn evidence into a
finding. If there's no execution record and no independent validation, it never
becomes a finding — no matter how confident the model sounds.

Safety enforced in code, not prompts

Offensive tools + autonomous agents is a scary combo if "be careful" is just a
line in a prompt. In OIHK the guardrails are actual code:

• PASSIVE mode is a policy layer, not an instruction. Active tools are rejected even if the agent tries to route them through the generic shell.
• Scope is exact. Declaring example.com doesn't authorize its subdomains or resolved IPs. Declared hosts are resolved once and DNS-pinned for the whole run.
• Egress fails closed. The declared scope is compiled into a netfilter allowlist inside the sandbox's own per-run network namespace. On platforms that can't guarantee it, startup aborts instead of pretending to be isolated.
• Hardened sandbox: read-only rootfs, dropped capabilities, no-new-privileges, non-root, no sudo surface.

Swap the model, keep the engine

OIHK is provider-agnostic. Any OpenAI-compatible endpoint works (LM Studio by
default), with per-role model routing and no hardcoded provider. You can run a
strong reasoning model as the planner and a fast one for the specialists.

It's also its own benchmark

This is my favorite part. OIHK doubles as an evaluation environment: it runs
the real engine against 16 local, deliberately vulnerable scenarios and scores
the model programmatically — never by asking a model to grade itself.

There's a deterministic offline mock solver for CI and demos:

bash
uv run oihk eval run-all --model mock