DEV Community

Cover image for I Built a Security Firewall for AI Agents: From 73 Tests to 1,600+
Shubhbhangoo
Shubhbhangoo

Posted on

I Built a Security Firewall for AI Agents: From 73 Tests to 1,600+

AI agents are becoming increasingly capable of using tools.

They can call APIs, access databases, execute code, interact with MCP servers, make HTTP requests, and potentially perform actions with real-world consequences.

That creates a question I kept coming back to:

What actually stands between an AI agent and a dangerous tool call?

I decided to build that layer.

I called it Agent Firewall.

What is Agent Firewall?

Agent Firewall is an authorization and security layer that sits between AI agents and the tools they want to use.

The basic idea is:

AI Agent
|
v
Agent Firewall
|
+--> Allow
+--> Deny
+--> Require Approval
|
v
Tool / API / MCP Server

Instead of trusting an agent simply because it is authenticated, the firewall evaluates whether that specific operation is authorized.

The project started as a relatively small policy engine.

It became considerably larger.


v0.1: Start with the obvious problem

The first version focused on basic authorization.

A tool request could result in:

ALLOW
DENY
APPROVAL

The firewall was designed to fail closed.

If there wasn't an applicable rule, the request wasn't allowed to magically pass through.

I also started testing things that shouldn't work, rather than only testing happy paths.

That became a recurring theme throughout the project.

A security boundary needs to be much more interested in what happens when someone tries to break it than a normal application usually is.


v0.2: Make policies harder to break

The next version made the policy system more expressive and defensive.

Policies gained things like:

Generic argument matching

Argument validation

Policy validation

Request IDs

Better auditability

MCP bypass testing

The test suite reached 73 tests.

At this point I started treating the firewall less like a feature and more like a security boundary.

That distinction matters.

A normal application can sometimes recover from an unexpected input.

An authorization layer should preferably say:

No.


v0.3: Agents need identities

A tool shouldn't just ask:

Is this operation allowed?

It should also ask:

Is this operation allowed for this agent?

So Agent Firewall became identity-aware.

This introduced:

Agent-specific authorization

Identity-aware policies

Conflict resolution

More precise policy matching

Concurrency testing

Performance benchmarking

The test suite reached 145 tests.

The architecture was starting to become something more interesting than a collection of if statements.


v0.4: Cryptographic identity

An agent name is not an identity.

If a request simply says:

agent = finance-agent

then anyone who can claim that string could potentially impersonate the agent.

So the next step was cryptographic identity.

Agent Firewall gained:

Cryptographic agent identities

Key lifecycle management

Key rotation

Key revocation

Persistent identity state

Cryptographically chained audit logs

The test suite reached 264 tests.

This was one of the points where the project really started feeling like security infrastructure rather than an ordinary authorization library.


v0.5: Capabilities

Identity answers who the agent is.

But I also needed to answer:

What is the agent actually allowed to do?

That led to capability-based authorization.

Capabilities became first-class permissions.

For example:

finance-agent
|
+-- payments.send

I also added:

Rate limiting

Spending budgets

Persistent security state

Human approval workflows

Capability requirements

Policy conflict handling

The test suite reached 390 tests.

Now an agent could be identified and restricted by the authority it possessed.


v0.6: Signed capabilities

This was a much bigger step.

Capabilities themselves became cryptographically signed permissions.

A capability could contain things such as:

Agent
Capability
Issuer
Constraints
Expiration
Signature

I added explicit capability namespaces:

payments.send
payments.refund
payments.*

So:

payments.* → payments.send ✅
payments.* → payments.refund ✅
payments.* → accounts.read ❌

Then came attenuation.

A capability with:

payments.*
amount_max = 1000

could be narrowed to:

payments.*
amount_max = 100

but it couldn't suddenly become:

amount_max = 10000

That led naturally into delegation.

An agent could delegate authority to another agent, but the delegated authority couldn't exceed the original authority.

Replay protection was also introduced using nonces and capability fingerprints.

The test suite reached 737 tests.


v0.7: Leave the Python sandbox

At this point I had a fairly serious authorization model.

But there was another problem.

A security system isn't particularly useful if it only protects a function inside your application.

Real agents interact with external systems.

So v0.7 pushed the capability model across actual protocol boundaries.

Agent Firewall gained:

MCP authorization

HTTP authorization

Signed capability transport

HTTP method/path namespace mapping

Cross-agent capability binding

Request constraint enforcement

Adversarial protocol testing

For example:

POST /payments

http.POST.payments

Capability verification

Authorization

Handler

The same security model could now be applied across different ways an agent might reach a tool.


v0.8: Capabilities need a lifecycle

Then I ran into another problem.

A capability isn't simply:

valid

or:

invalid

It has a history.

It might be:

ISSUED

DELEGATED

ATTENUATED

USED

REPLAYED

REVOKED

EXPIRED

So v0.8 introduced explicit capability lifecycle tracking.

More importantly, lifecycle and revocation state became persistent.

I added SQLite-backed storage so security state could survive process restarts.

That meant the system could remember:

Revoked capabilities

Lifecycle history

Capability usage

Replay events

Authorization denials

Expiration events

The v0.8 checkpoint reached 1,438 passing tests.


v0.9: Make it usable

Security infrastructure isn't very useful if developers hate integrating it.

So v0.9 focused on the developer experience.

Instead of forcing developers to manually construct every authorization call, they can use:

from firewall.protect import protect

@protect(
sdk=sdk,
capability=capability,
)
def send_payment(amount):
return amount

The important invariant remains:

authorize()

ALLOW

handler()

Not:

handler()

oops, authorization failed

Denied operations don't reach the handler.

I also added reusable protected tools:

from firewall.tools import ProtectedTool

tool = ProtectedTool(
sdk=sdk,
capability=capability,
handler=send_payment,
)


Tool adapters

v0.9 also introduced vendor-neutral and vendor-specific tool adapters.

The project now supports adapters for:

OpenAI tools

Anthropic tools

Generic tool calls

The adapters translate provider-specific formats, but they don't create authority.

Authorization stays inside the firewall.

A vendor-neutral request can be normalized into a common representation before reaching the authorization layer.

That means the security model doesn't need to change every time an agent framework changes its tool-call format.


Lifecycle investigation

Another v0.9 addition was a read-only explanation layer.

Instead of creating another authorization engine, the explanation system reads lifecycle history and answers questions like:

What happened to this capability?

Was it revoked?

Was it replayed?

Was the request denied?

What was the latest lifecycle event?

This makes investigating authorization decisions much easier without introducing a second source of truth.


CLI

v0.9 also introduced a CLI:

firewall --help
firewall init --path firewall.yaml
firewall validate firewall.yaml
firewall inspect-token
firewall explain lifecycle.db

The CLI uses the same underlying Python APIs as the SDK.

One authorization engine.

Different interfaces.


Property-based testing

I also started using Hypothesis alongside the existing adversarial tests.

Instead of writing only:

test_input_1
test_input_2
test_input_3

I can describe properties the system should maintain and let generated inputs explore the edges.

The property-based tests cover areas such as:

Input normalization

Lifecycle snapshots

Persistence round trips

Authorization stability

Capability transport

Property-based testing isn't a magic security certificate.

It's another way to find weird combinations that manually written tests might miss.


v1.0: Persistent key management

The v1.0 release moves deeper into cryptographic key infrastructure.

One of the major additions is persistent key management through SQLite.

Previously, key state could exist only in memory.

Now the key manager can persist:

Key IDs

Private/public key material

Active key state

Retired keys

Trusted issuers

Key rotation can therefore survive process restarts.

The model is roughly:

Active Key
|
| rotate
v
Retired Key

New Key
|
v
Active Key

The implementation also checks for invalid persistent states, such as multiple active keys.

This is important because cryptographic infrastructure shouldn't quietly accept corrupted or contradictory security state.


The architecture now

The system has evolved quite a bit from the original:

Agent → Firewall → Tool

The security path is now closer to:

AI AGENT
|
v
Provider Adapter
|
v
Tool Normalization
|
v
Agent Identity
|
v
Signed Capability
|
v
Namespace Check
|
v
Constraints
|
v
Validity Check
|
v
Replay Protection
|
v
Policy
|
v
Rate Limit / Budget
|
v
Approval
|
v
Authorization
/ \
DENY ALLOW
| |
X v
Tool Handler
|
v
Real Tool

And underneath it:

Persistent Security State
|
+-- Revocations
+-- Lifecycle
+-- Keys
+-- Audit

The goal is simple:

The tool should never execute before the security boundary has authorized it.


1,600+ tests later

One thing I really wanted from the project was for the test suite to grow alongside the attack surface.

The progression looked roughly like:

v0.2 73 tests
v0.3 145 tests
v0.4 264 tests
v0.5 390 tests
v0.6 737 tests
v0.8 1,438 tests
v0.9 1,602 tests

The tests cover areas including:

Policy conflicts

Identity security

Cryptographic verification

Key lifecycle

Capability authorization

Delegation

Attenuation

Replay protection

Rate limits

Budgets

Approvals

Persistent state

Lifecycle tracking

Audit integrity

Concurrency

MCP

HTTP

Tool adapters

Adversarial combinations

The number itself isn't a security guarantee.

But watching the test suite grow alongside the architecture has been useful.


v1.0 is released 🔐

After all those iterations, Agent Firewall v1.0 is now released.

What started as a small policy firewall evolved into an authorization layer with:

Cryptographic agent identities

Signed capabilities

Capability namespaces

Constraints

Delegation

Attenuation

Replay protection

MCP authorization

HTTP authorization

Human approval

Budgets

Rate limiting

Capability lifecycle tracking

Persistent revocation

Persistent key management

Tamper-evident audit logging

Developer APIs

Tool adapters

CLI tooling

Property-based testing

The project has now crossed 1,600+ tests across the development releases.

But v1.0 isn't the finish line.

Security infrastructure needs continued testing, hardening, review, and scrutiny.

The goal is to build a system where AI agents can remain powerful without giving them unlimited authority.

Agents should be powerful. Their authority should be explicit, bounded, and enforceable.


Install

pip install agent-firewall

For development:

git clone https://github.com/Shubhbhangoo/agent-firewall.git
cd agent-firewall
pip install -e ".[dev]"

Then protect a tool:

from firewall.protect import protect

@protect(
sdk=sdk,
capability=capability,
)
def send_payment(amount):
return amount

Try Agent Firewall

The project is open source:

[Agent Firewall on GitHub](https://github.com/Shubhbhangoo/agent-firewall

If you're working on AI agents, MCP tools, automated workflows, or security infrastructure, I'd be interested in seeing how you're approaching authorization and tool security.

Agent Firewall v1.0 is out. 🔐

Top comments (0)