DEV Community

Cover image for my ai agent ran for 6 hours on a 2-minute task and cost me $200
Debashish Ghosal
Debashish Ghosal

Posted on

my ai agent ran for 6 hours on a 2-minute task and cost me $200

my ai agent ran for 6 hours on a 2-minute task and cost me $200

it's 2am. your phone buzzes. the on-call alert says your AI agent pipeline has been running for 6 hours on a task that should take 2 minutes.

you open the logs. the agent changed code. tests failed. it changed something else. a different test failed. it reverted. the original test failed again. 500 iterations. zero progress.

your LLM API bill for this one task: $200. for a task that didn't complete.

this isn't hypothetical. it's the rite of passage for anyone building agentic systems. non-deterministic systems against deterministic environments — code, APIs, databases — fall into stuck states. and they burn real money.

the dirty secret of cheap AI models: they're cheap per call, but expensive per completed task. a $0.001 call that loops 10 times costs more than a $0.05 call that solves it once.

in traditional software, circuit breakers prevent cascading failures. when a service is failing, the circuit opens, traffic stops, and the system degrades gracefully. why are we deploying AI agents into production without them?

I looked for a library that detects stuck agent loops and escalates to a stronger model. LangGraph, CrewAI, AutoGen — they're frameworks, not failure detection. LangSmith, LangFuse — observability, after the fact. cost trackers — they tell you what happened, not stop it. nobody sits between frameworks and cost trackers. a quality layer that drops into whatever you've already built.

so I built one.


what I built

ai-loopguard — a drop-in circuit breaker for LLM agent loops. it detects stuck patterns and escalates to a stronger model before costs spiral. 4 lines of code. works with LangGraph, CrewAI, or raw Python.

from ai_loopguard import Guard

guard = Guard(escalation_model=gpt4_model, workhorse_model_name="qwen2.5-coder")

@guard.protect
def agent_step(state):
    return cheap_model.generate(state)
Enter fullscreen mode Exit fullscreen mode

that's it. if agent_step fails 3 times, loopguard packages the context, sanitizes it, redacts secrets, escalates to your configured cloud model, logs the event, and returns the escalated output. the loop continues. 4 lines to add, zero refactoring.

the repo is here: github.com/deghosal-2026/ai-loopguard — MIT licensed, 436 tests, 97.3% coverage.


the two problems nobody talks about

building a circuit breaker for AI agents is two hard problems, not one.

problem 1: how do you detect a loop in a non-deterministic system?

the naive approach is a counter: max_iterations=10. kill the agent after 10 steps. but that can't distinguish between an agent making progress on a hard task and one spinning in circles. and it completely misses the most insidious pattern: oscillation — the agent bouncing between two states. test passes. agent changes code. test fails. agent reverts. test passes. same code changed. test fails. the counter goes up, but the agent goes nowhere.

loopguard uses pattern detection, not counters. error clustering — same exception type and message across N consecutive steps. test-failure cycles — same test fails N times or oscillates pass→fail→pass→fail. schema validation failures — output fails schema validation N consecutive times. custom callbacks — your own domain heuristics, wall-clock timeout, token budget exceeded.

no self-grading. no "let the model decide if it's stuck." triggers are deterministic or user-defined.

problem 2: what do you do when a loop is detected?

most tools crash or log a warning. loopguard handles it with an escalation layer. model upgrade — package the context, sanitize it, escalate to a stronger model. human-in-the-loop — pause execution and ask "agent is stuck, escalate to GPT-4?" or fail-open — if loopguard itself fails, your agent keeps running. the circuit breaker never takes down your system.


does it actually work?

I ran a field study. 15 tasks across 3 external open-source repos: SWE-agent (~14k stars), Aider (~46k stars), LangGraph (~100k stars). same tasks, with and without loopguard.

without loopguard: 0 out of 15 tasks completed. the agents looped, burned tokens, and failed.

with loopguard: 15 out of 15 tasks completed. every task that failed without loopguard succeeded with it. every escalation produced a valid result. average integration: 4 lines of code.

detection overhead is 2.75 microseconds — less than a CPU branch misprediction. you'll never measure the cost of loopguard watching your agent. until it saves you.


the security part most teams skip

when your agent gets stuck, its state might contain API keys, user PII, or malicious instructions from a web page it scraped. loopguard treats all agent output as untrusted and does three things before sending anything to the escalation model.

context compression — not a full dump, just first + last + summary, bounded by a token limit. sanitization — clear system vs user separation, delimiters around agent-produced content. redaction — built-in patterns for OpenAI keys, AWS keys, GitHub tokens, plus your own regex and field names.

every escalation event log includes sanitized: true and redacted_fields: ["api_key", "password"] — you can audit what was stripped.

there's also a threat nobody thinks about: economic DoS. if your agent loop is buggy and loopguard escalates on every step, you could rack up $50 before you notice. the max_escalations_per_run cap prevents that — one escalation per guarded execution by default, configurable up to 10.


what I learned building this

the biggest surprise was that nobody had built this yet. LangGraph gives you DAGs, conditional edges, parallel stages, approval gates. it does not give you stuck-loop detection, test-failure cycle detection, automatic escalation, or cost-per-completed-task tracking. those are the things that actually matter when your agent is in production.

the second surprise: the cost metric everyone tracks is wrong. most cost trackers tell you cost per call. that's the wrong metric. a task that loops 10 times at $0.001/call costs $0.01. a task that succeeds once at $0.05/call costs $0.05. the "cheap" model is 5x more expensive per completed task. loopguard tracks escalation rate, cost per completed task, and routing vs failover — the metrics that actually tell you whether your routing config is working.

the third surprise: fail-open is non-negotiable. if the circuit breaker itself crashes your agent, it's worse than not having one. three modes — re-raise the original exception, return the last successful output, or return a sentinel. every fail-open event is logged. you'll know loopguard failed even if your agent didn't.


what's in the repo

this wasn't built by vibe-coding. the repo includes a full PRD (886 lines), SPEC (1,511 lines), and WBS (1,247 lines). 12 milestones, all complete, with checkpoints and LLM routing strategy per task. 436 tests across 4 layers: unit, integration, e2e, field study. ruff clean, mypy strict clean, pip-audit zero vulnerabilities. CI matrix: Python 3.10/3.11/3.12 across ubuntu/macos/windows.

good first issues if you want to contribute: a hallucination_cycle trigger (agent claims to fix something it already attempted), LangGraph Send API support for parallel agent fan-out, AutoGen/Semantic Kernel/LlamaIndex integrations, CLI improvements like loopguard watch for live log tailing.

full contributing guide here: CONTRIBUTING.md. 3 commands to get started, full dev toolchain included.


discussion

if you're running AI agents in production, you will hit a stuck loop. not a question of if — it's when. the question is whether you'll have a circuit breaker when it happens.

I want to hear from people actually running agents in production, not experimenting:

how are you handling agent loops today? rigid iteration limits? manual monitoring? just hoping it doesn't happen? I want to know what your current stack looks like.

what triggers are you missing? the 4 current triggers came from my own experience. your agents might get stuck in ways I haven't seen. hallucination_cycle is the most requested — if you've seen hallucination patterns, your field experience would shape the detection logic.

is auto-escalation the right default? or should interrupt mode be the default? I chose auto for production throughput, but I might be wrong.

are you running multi-agent systems? the CrewAI integration gives each agent its own state. would you want cross-agent escalation — agent A gets stuck, agent B picks up the task?

your input shapes the v0.2.0 and v0.3.0 roadmap. the best ideas will come from people running agents in production — not from me sitting in a vacuum.

star the repo, open an issue, drop a comment. github.com/deghosal-2026/ai-loopguard

Top comments (1)

Collapse
 
michael_salinas_472fbf6c1 profile image
Michael Salinas

Great! When can we discuss further? You can schedule and let me know when.
I think this is nice opportunity for each other.
Okay?