DEV Community

muhammadwaqasai
muhammadwaqasai

Posted on

Not every AI agent action deserves the same trust: adding risk-tiered permissions to agent_acid

Most AI agent safety systems treat every action the same way -- either blocked or allowed. But a routine email reply and a $5,000 refund are not the same risk category, and treating them identically is either too restrictive or too dangerous.

I just added a 3-tier permission system to agent_acid:

🟢 GREEN -- auto-executes, no extra check (same as before)
🟡 YELLOW -- must pass a verify_fn before executing (e.g. a business rule or LLM-as-judge check)
🔴 RED -- pauses execution entirely. Nothing happens -- not even a "do it then undo it" -- until a human explicitly approves or rejects it.

refund_tool = ReversibleTool(
    name="issue_refund",
    execute=lambda kwargs: payment_api.refund(kwargs),
    compensate=lambda kwargs, result: payment_api.reverse(result["id"]),
    risk_level=RiskLevel.RED,
    risk_reason="Refunds always require human sign-off.",
)
Enter fullscreen mode Exit fullscreen mode

I tested this by attempting a $99,999 refund through a live agent. It got flagged RED, held in a pending state, and I rejected it as the human approver. The real payment API was never called -- not once.

This is now the 4th safety layer in agent_acid, alongside rollback, guardrails, and shadow execution. 14 automated tests, all passing.

GitHub: github.com/muhammadwaqasai/agent_acid
pip install agent-acid

Curious how others are handling human-in-the-loop approval for AI agents in production.

Top comments (0)