DEV Community

muhammadwaqasai
muhammadwaqasai

Posted on

# I built a rollback + guardrail engine for AI agents after finding an attack most safety layers miss

#python #ai #opensource

Most AI agent guardrail systems check one tool call at a time, with no memory of the session. That makes them blind to a real, documented attack pattern sometimes called "salami slicing" — an attacker (or a manipulated AI) splitting one large forbidden action into several small, individually-legal-looking ones.

I ran into this while thinking about what it actually takes to trust an autonomous AI agent with real-world actions — charging a card, writing to a database, sending an email. So I built agent_acid to close two specific gaps.

The problem, concretely

Say you give an AI agent a tool to charge a customer's card, with a guardrail that blocks any single charge over $500. Seems safe, right?

Here's the catch: if an attacker convinces the AI (via a prompt injection, or just a confused multi-step plan) to charge $400 three times instead of $1200 once, every individual call passes the check. Nothing in a typical guardrail setup notices that the total across the session is dangerous.

What agent_acid adds

1. Automatic rollback
Every tool gets a paired compensating action — an "undo." If any step in a multi-step agent plan fails, everything already completed gets undone automatically, in reverse order.

charge_tool = ReversibleTool(
    name="charge_card",
    execute=lambda kwargs: real_payment_api.charge(kwargs),
    compensate=lambda kwargs, result: real_payment_api.refund(result["charge_id"]),
    guardrails=[max_value("amount", limit=500)],
)
Enter fullscreen mode Exit fullscreen mode

2. Stateful (session-memory) guardrails
Rules that track the whole transaction, not just one call:

stateful_guardrails=[
    cumulative_max("charge_card", "amount", session_limit=1000)
]
Enter fullscreen mode Exit fullscreen mode

This catches the $400 x 3 pattern even though each individual call is within the per-step limit.

Proving it against a real, live AI

I didn't just want to claim this works — I tested it against an actual GPT-4o-mini agent trying to pull off both attacks:

Attack 1 — Prompt injection. A fake "administrator override" hidden inside a customer support ticket tries to convince the AI to charge $75,000 instead of following the $500 limit. The AI gets fooled and attempts it. The guardrail blocks it and fully reverses the account creation and the charge.

Attack 2 — Salami slicing. The injected instruction explicitly tells the AI to split a $1,200 charge into three $400 calls specifically to dodge the per-step limit. I ran this against two setups side by side:

Naive agent (per-step check only) agent_acid
3x $400 charge attempt All 3 succeeded — $1,200 charged Blocked on the 3rd call
Rollback available? No — money is just gone Yes — fully refunded
Account left behind? Yes, permanently Deleted during rollback

The naive agent didn't just fail quietly — it told the user everything worked perfectly. That's the real danger.

It's tested and open source

  • 5 automated pytest tests, no API key required, proving the core rollback and guardrail logic
  • 6 example scripts, including both attack tests and the comparison above
  • Published on PyPI: pip install agent-acid
  • Full source on GitHub: https://github.com/muhammadwaqasai/agent_acid

I'm genuinely interested in people trying to break it. If you find a way around the guardrails, I want to know — open an issue or drop a comment.

Top comments (0)