DEV Community

kabirnarang39
kabirnarang39

Posted on • Originally published at kabirnarang39.github.io

I let Claude Code build a real security tool, here's the honest writeup

AI agents now call tools, MCP servers, and each other on their own. That is the
whole point of an agent. It is also the problem: the moment an agent is
compromised — a prompt injection, a leaked key, a poisoned tool result — it
keeps its credentials and its network path, and nothing between it and your
systems says no.

Most of the stack around agents watches and reports. It logs the calls, maybe
scores them, and pages a human. By the time the human reads the page, the agent
has made a thousand more calls.

I built Wardline to enforce instead of watch. It's a control-plane proxy: an
AI agent reaches its MCP servers, tools, and gRPC upstreams only through
Wardline, which applies identity, policy, budget, and anomaly detection
in-process and writes every decision to an audit trail. One static Go binary —
no database, no identity provider, no sidecar to start.

The part that's actually different: auto-block

Alerting is easy and everyone does it. The claim worth making is enforcement.

Wardline keeps a per-identity behavioral baseline using Welford's algorithm
a running mean and variance over four features per time window: call rate,
distinct-tool count, deny ratio, and mean inter-arrival time. No training data,
no external model, no history to store. Each completed window is scored as a
combined z-score against that identity's own baseline.

When the score crosses a configured threshold and auto_block is on, Wardline
doesn't just write an anomaly record — it rejects that identity's calls for a
bounded TTL. The compromised agent is cut off in real time, with no rule written
for the specific attack and no human in the loop.

// The detector wires a real BlockChecker as the blocker; when the combined
// z-score exceeds the auto-block threshold, the identity is blocked.
if d.cfg.AutoBlock.Enabled && d.blocker != nil && blockScore > d.cfg.AutoBlock.ScoreThreshold {
    d.blocker.Block(e.Identity, e.Tenant, /* reason */)
}
Enter fullscreen mode Exit fullscreen mode

The decision path benchmarks at ~33ns / 0 allocations on the default YAML
backend.

The part most projects won't tell you: what it does not catch

The auto-block catches abrupt deviation. It does not catch low-and-slow.

Because the baseline is self-learned and unsupervised, an attacker who ramps
activity gradually — staying within a few standard deviations of the moving
baseline each window — is never blocked. The baseline adapts upward and absorbs
the ramp. Wardline blocks the agent that suddenly does 10x its normal rate; it
does not block the agent that patiently climbs to 10x over an hour.

This isn't a threshold you can simply tighten. Tighten it and you start blocking
normal agents — the false-positive rate is regression-guarded to stay near zero
on steady traffic, and that guard is the thing keeping the feature usable. It's
an inherent tradeoff of unsupervised, per-identity baselining.

Both behaviors are pinned by tests in the repo —
TestDetector_AutoBlock_AbruptSpikeIsBlocked and
TestDetector_AutoBlock_LowAndSlowEvades — so the boundary is documented, not
marketed around.

The takeaway isn't "anomaly detection is weak." It's that anomaly detection is
the last line, not the only one. Keep explicit policy and budget limits as the
hard floor — they bound absolute behavior regardless of ramp speed — and let
auto-block catch the fast, obvious compromise that policy didn't anticipate.

Secure by default is a claim; read the defaults

Wardline fails closed on policy. It does not fail closed on identity or the
dashboard by default: identity is trusted from the X-Wardline-Identity header
(spoofable) and the dashboard's read views are unauthenticated, until you turn on
the flags that change that. The binary logs a WARN on startup for every
insecure default still in effect, so the posture is never silent.

features:
  credential_issuance: true   # verify a signed bearer token instead of trusting the header
  rbac: true                  # gate the dashboard and admin actions on real permissions
Enter fullscreen mode Exit fullscreen mode

About the "built with Claude Code" part

I'll be transparent, because the honesty is the whole theme: Claude Code did the
bulk of the implementation, under my direction. Architecture, threat model, and
every design decision are mine and were reviewed by me. The workflow was
spec-driven — spec → plan → build → and on any failure, trace the root cause and
update the spec before retrying. That loop is what kept ~46k lines of Go coherent
instead of drifting into slop: ~950 tests, race + coverage in CI, golangci-lint
clean, Clean Architecture actually enforced.

The contribution I valued most wasn't code volume — it was that it kept pushing
me to document limitations (like the low-and-slow gap) and write tests proving
them, rather than ship an impressive demo that overpromised. For a security tool,
that's exactly the instinct you want.

Where it fits

Wardline is young and unproven at scale — that's the honest status. It's not
trying to replace an LLM router like LiteLLM or a managed gateway like Portkey.
It's the enforcement-first control plane for the traffic between an agent and
everything it calls, in a single self-hosted binary, Apache-2.0.

If that's the layer you're missing, the repo is here:
github.com/kabirnarang39/wardline.
Feedback on the anomaly approach and the threat model is exactly what I'm after —
poke holes.

Top comments (0)