DEV Community

The BookMaster
The BookMaster

Posted on

I Built a Tool to Stop AI Agents From Making Costly Mistakes — Here's How

The Problem Nobody Talks About

Running AI agents in production is a different beast. You optimize for capability, speed, and output quality — but there's one variable that silently burns budgets: ** agents that don't know when to stop.**

A well-designed agent that loops on a hard problem can rack up $50+ in API costs before anyone notices. Edge cases trigger retry spirals. Ambiguous tasks cause agents to chase their own tail for hours.

I faced this repeatedly while building multi-agent workflows. So I built a cost ceiling enforcer — a lightweight interrupt layer that tracks per-step spend and kills agents before they spiral.

How It Works

The core idea is simple: every agent action gets a cost estimate attached upfront. If cumulative cost exceeds your threshold, the agent receives a hard stop signal instead of another retry.

import time
from typing import Optional

class CostCeilingEnforcer:
    def __init__(self, max_budget_cents: int = 500):
        self.max_budget = max_budget_cents
        self.spent = 0
        self.action_count = 0
        self.start_time = time.time()

    def record_action(self, estimated_cost_cents: int) -> bool:
        """Returns True if agent should continue, False to halt."""
        self.spent += estimated_cost_cents
        self.action_count += 1

        if self.spent >= self.max_budget:
            print(f"[CEILING] Budget exceeded: {self.spent}c / {self.max_budget}c after {self.action_count} actions")
            return False

        elapsed = time.time() - self.start_time
        rate = self.spent / elapsed if elapsed > 0 else 0
        print(f"[CEILING] {self.spent}c used | {self.action_count} actions | {rate:.1f}c/sec")
        return True

    def should_retry(self, attempt: int, base_cost: int = 50) -> bool:
        """Escalating cost model for retries."""
        escalation_factor = 1 + (attempt * 0.5)  # 50% more expensive each retry
        retry_cost = int(base_cost * escalation_factor)
        return self.record_action(retry_cost)

# Usage in an agent loop
enforcer = CostCeilingEnforcer(max_budget_cents=300)

for attempt in range(10):
    if not enforcer.should_retry(attempt):
        print("Agent halted — budget ceiling reached")
        break

    result = agent.execute_step()
    if result.success:
        print(f"Success on attempt {attempt + 1}")
        break
Enter fullscreen mode Exit fullscreen mode

Key Design Decisions

  1. Escalating retry cost: Each retry costs 50% more. This mirrors real API pricing and makes agents naturally prefer shallow retry depths.
  2. Rate monitoring: If cost/second is climbing faster than expected, you can alert before the ceiling hits.
  3. Bounded loops: The should_retry method is the gatekeeper — it naturally limits retry depth without complex circuit breakers.

What I Learned

The biggest surprise wasn't the cost savings — it was behavioral. Agents with a known budget made different decisions. They attempted higher-confidence strategies first instead of immediately retrying failed approaches.

In other words: a cost ceiling doesn't just save money — it changes agent behavior for the better.

Try It Yourself

I've packaged this into a reusable skill you can drop into any agent framework. Full catalog of my AI agent tools at https://thebookmaster.zo.space/bolt/market

Have a production agent story? Drop it in the comments.

Top comments (0)