It started with a $2,800 Stripe charge.
An AI agent we'd deployed to handle customer onboarding hit a flaky webhook endpoint. The standard retry logic kicked in. But instead of retrying once or twice, the agent discovered it could call a different endpoint to achieve the same result. Then another. Then it started chaining calls — each one triggering a separate Stripe charge because the underlying action wasn't idempotent.
By the time our billing alert fired four hours later, the damage was done. No single call looked abnormal. The spend curve was smooth and gradual. And our monitoring dashboard showed every single transaction in perfect detail — after the money was already gone.
That's the day I realized: observability is not enforcement.
The Problem: Agents Are Autonomous Spenders
AI agents are fundamentally different from traditional applications. A web app makes API calls based on user actions — predictable, bounded, and rate-limited by human interaction speed. An AI agent makes calls autonomously, at machine speed, and can discover new endpoints and tools at runtime.
This creates a new class of financial risk:
- Runaway loops: An agent enters a reasoning loop, burning tokens indefinitely
- Cascade discovery: An agent finds a new API and starts making calls you never anticipated
- Tool re-execution: Retry logic triggers duplicate payments when tools aren't idempotent
- Delegation spirals: An agent delegates sub-tasks to itself, each spawning more tool calls
- Infinite reasoning: Thinking-mode loops consume token budgets without producing output
I looked at existing solutions. LangChain had cost tracking. CrewAI had guardrails. AutoGen had governance discussions. But they all shared the same fundamental flaw: they observed spending instead of preventing it.
A dashboard that tells you "you spent $2,800 today" is useless when the money is already gone.
Building AgentShield: A Spend Firewall
I needed something that sits between the agent and the tools it calls — a firewall that inspects every transaction before it executes and blocks anything that violates a budget rule.
The design was inspired by traditional network firewalls: deterministic, fast, and non-bypassable.
The 7 Rule Types
After analyzing our incident and dozens of similar failure modes reported across GitHub issues in CrewAI, AutoGen, LangChain, and Claude Code, I identified seven distinct spending attack patterns. Each maps to a specific rule type:
1. transaction_limit — Blocks any single transaction above a threshold. Simple but effective. If no individual API call should cost more than $5, this catches the $50 call immediately.
2. daily_total — Rolling 24-hour cumulative spend cap. The failsafe against any single-day disaster. Our $2,800 incident would have been caught at $50.
3. velocity — Rate limiting per merchant/tool. Catches burst attacks where an agent fires 100 calls in 30 seconds. This is the rule that catches retry storms.
4. merchant_allowlist — Only approved endpoints can be called. This is critical for agents that can discover new tools at runtime. If your agent shouldn't be calling unknown APIs, don't let it.
5. category_block — Block entire categories of spending. Useful for compliance ("no cryptocurrency purchases") or cost control ("no premium model calls during batch jobs").
6. session_budget — Per-agent-run budget cap. When an agent session starts, assign it a budget. When it's exhausted, the agent gets a budget-exceeded error and must terminate gracefully.
This is the rule that catches infinite reasoning loops and delegation spirals. The agent might be stuck reasoning forever, but it can't spend more than its session allows.
7. cascade_cost — The most nuanced rule. It detects when total spending is growing super-linearly — the signature of a runaway pattern. If call #1 costs $0.01, call #2 costs $0.01, but by call #50 you're spending $1.00/call, something is wrong.
This is the rule that would have caught our original incident. The agent's per-call cost didn't change, but the frequency was accelerating exponentially as it discovered new endpoints.
The 56-Scenario Evaluation Gym
Rules are only as good as their test coverage. I built a 56-scenario evaluation harness that simulates every spending attack pattern I could find — from GitHub issues, production incidents, and adversarial testing.
The scenarios fall into six categories:
| Category | Scenarios | Example |
|---|---|---|
| Single-call attacks | 8 | Agent attempts $500 single transaction |
| Burst/velocity attacks | 12 | Agent fires 200 calls in 10 seconds |
| Cumulative spending | 10 | Agent spends $1/call for 500 calls |
| Loop patterns | 14 | Agent enters infinite reasoning loop |
| Discovery attacks | 8 | Agent finds undocumented endpoint |
| Multi-agent cascades | 4 | Agent delegates to sub-agents, each spending |
Each scenario runs the agent against AgentShield with a configured rule set. The test passes if the firewall blocks the transaction before it reaches the provider, and fails if the spend leaks through.
Current pass rate: 54/56 (96.4%). The two remaining edge cases involve multi-agent systems where spending is distributed across agents — we're working on cross-agent budget aggregation for v2.
Enforcement vs. Observability
This is the distinction that matters most, and it's the one most teams get wrong.
Observability answers: "What happened?"
- Dashboards, alerts, logs
- Cost tracking per provider
- Token usage analytics
- Post-incident analysis
Enforcement answers: "What is allowed?"
- Rule evaluation before execution
- Hard blocks on policy violations
- Budget caps that cannot be exceeded
- Pre-call authorization
You need both. But enforcement is the one that prevents financial loss.
Think of it this way: a security camera (observability) helps you catch a thief after they've stolen from you. A locked door (enforcement) stops them from entering in the first place.
AgentShield is the locked door. It sits in the tool dispatch path and evaluates rules in under 1 millisecond — fast enough that the agent never notices the checkpoint, strict enough that no transaction can bypass it.
How It Works in Practice
AgentShield runs as middleware. You configure your rules, and every tool call from your agent passes through the firewall:
from agentshield import SpendFirewall
firewall = SpendFirewall(
rules=[
{"type": "daily_total", "limit": 50.00},
{"type": "velocity", "max_calls": 20, "window_seconds": 60},
{"type": "session_budget", "budget": 10.00},
]
)
@firewall.guard(tool_name="stripe_charge")
def process_payment(amount, customer_id):
return stripe.charges.create(amount=amount, customer=customer_id)
If a rule is violated, the agent receives a BudgetExceededError instead of completing the call. The agent can handle this gracefully — log it, notify a human, or terminate the session.
What's Next
AgentShield is open source and free to use. The hosted version at agentshield.fly.dev includes a dashboard for rule management and spend visualization — the observability layer on top of the enforcement engine.
- GitHub: kindrat86/agentshield
- Live demo: agentshield.fly.dev
- Free tier: Covers most side projects and small teams
If you're running AI agents in production, the question isn't whether you'll have a spending incident — it's when. A spend firewall doesn't prevent bugs, but it ensures that when something goes wrong, the financial damage is bounded.
Because the alternative is finding out from a Stripe bill.
AgentShield is MIT-licensed and runs on Python 3.10+. If you've experienced an agent spending incident, I'd love to hear about it — the 56-scenario eval gym is always accepting new attack patterns.
Top comments (0)