DEV Community

Cover image for When Your Tool Returns Garbage, Agents Loop Forever. Here's the 30-Line Guard.
Gabriel Anhaia
Gabriel Anhaia

Posted on

When Your Tool Returns Garbage, Agents Loop Forever. Here's the 30-Line Guard.

#agents #ai #llm #python

A tool returns a malformed JSON blob. The model squints at it, decides "I'll try that again," and calls the same tool with the same arguments. The same garbage comes back. The model tries again. And again. Your billing dashboard ticks up, the user is watching a spinner, and on-call is about to get paged for cost anomaly alerts.

This isn't a model bug. The model is doing what it's supposed to do: retry on transient failure. The bug is that nothing between your tool and the model translates "this came back broken" into a signal the model can act on.

You don't need a smarter model to fix it. You need 30 lines of Python around every tool call.

The retry loop that drains accounts

Here's the shape, slightly anonymised, from a production trace a team I work with shared recently:

turn 12: assistant → tool_call(search_orders, {"customer_id": "C-9921"})
turn 12: tool_result → "{\"orders\": [{\"id\": \"O-1\", \"total\""  // truncated, invalid JSON
turn 13: assistant → "Let me try that again."
turn 13: tool_call(search_orders, {"customer_id": "C-9921"})
turn 13: tool_result → "{\"orders\": [{\"id\": \"O-1\", \"total\""  // same truncation
turn 14: assistant → "Apologies, let me retry."
...
turn 28: tool_call(search_orders, {"customer_id": "C-9921"})  // 17 identical retries
Enter fullscreen mode Exit fullscreen mode

Seventeen turns. Each one a full prompt round-trip with growing context. The tool kept returning a truncated payload because an upstream gateway had a 4KB response cap nobody documented. The model didn't stand a chance.

What the model needed wasn't more compute. It needed someone to say "the response is malformed, stop trying." Or better: "the response is malformed, the field truncated was orders[0].total, here's a smaller query that fits the cap." That message never came.

That someone is the guard.

Three classes of tool failure

Before writing the guard, you need a taxonomy. Tool failures don't all look the same, and the model's right move depends on which class you hit.

Class 1: Schema mismatch. The tool returned data, but it doesn't conform to the contract the agent was promised. Wrong field types, missing required keys, an enum value the agent doesn't know about. This is the truncated-JSON case above, but also the "I added a new status code last sprint and forgot to update the agent" case.

The right model move: stop retrying. Either ask the user for clarification, fall back to a different tool, or give up cleanly. Retrying the same call will never help.

Class 2: Partial data. The tool returned valid, well-formed data, but it's incomplete. Pagination cut off mid-result. A timeout returned what was available so far. An external API rate-limited and gave you the first 3 of 47 records.

The right model move: continue with what you have, or retry with different parameters (smaller page, narrower filter). Same call won't help; modified call might.

Class 3: Semantic garbage. The tool returned valid, well-formed, complete data, and the data is wrong. A search returned 0 results because the model passed a customer name in a field that expected an ID. A weather API returned a Celsius value when the agent asked for Fahrenheit. The shape is fine; the content makes no sense.

The right model move: re-think the call. Different arguments, different tool, or escalate to the user. Naive retry burns money without progress.

The point of the taxonomy isn't theory. It's the shape of the error message you send back to the model. Each class wants different guidance, and a generic "tool failed" loses that information.

The 30-line guard

Here's the core. It wraps any tool invocation, validates against an expected Pydantic schema, classifies the failure, and returns a structured error that the model can actually reason about.

from typing import Callable, Any
from pydantic import BaseModel, ValidationError
import json

class ToolError(BaseModel):
    error_class: str  # schema_mismatch | partial_data | semantic_garbage
    code: str         # invalid_status, truncated_response, empty_result, ...
    detail: str       # one human-readable line
    hint: str | None  # what the model should try next

def guarded_call(
    tool: Callable[..., str],
    schema: type[BaseModel],
    validate_semantics: Callable[[BaseModel], ToolError | None] = lambda _: None,
    **kwargs: Any,
) -> dict:
    try:
        raw = tool(**kwargs)
        parsed = json.loads(raw)
    except json.JSONDecodeError as e:
        return ToolError(
            error_class="schema_mismatch", code="invalid_json",
            detail=f"Tool output isn't valid JSON: {e.msg} at pos {e.pos}.",
            hint="Don't retry with the same args. The tool itself is broken.",
        ).model_dump()
    try:
        validated = schema.model_validate(parsed)
    except ValidationError as e:
        first = e.errors()[0]
        return ToolError(
            error_class="schema_mismatch", code="schema_violation",
            detail=f"Field `{'.'.join(map(str, first['loc']))}`: {first['msg']}.",
            hint="Don't retry with the same args. The contract is broken.",
        ).model_dump()
    semantic_err = validate_semantics(validated)
    return semantic_err.model_dump() if semantic_err else validated.model_dump()
Enter fullscreen mode Exit fullscreen mode

Thirty lines if you don't count the imports. The shape:

  • tool is whatever callable runs the side effect (an HTTP request, a DB query, a shell-out).
  • schema is a Pydantic model describing what success looks like.
  • validate_semantics is an optional second pass for class 3. Content checks that schema can't express ("empty result with a query that should have returned something").
  • Return value is either the validated payload (dict) or a ToolError dict.

The wrapper catches class 1 in the JSONDecodeError and ValidationError branches. It catches class 3 in validate_semantics. Class 2, partial data, falls through to validate_semantics too, because it's a content question, not a shape question.

Using it for a real tool

Suppose your agent has a search_orders tool. The schema:

class Order(BaseModel):
    id: str
    total_cents: int
    status: str  # placed, shipped, delivered, cancelled

class SearchOrdersResult(BaseModel):
    orders: list[Order]
    page: int
    has_more: bool
Enter fullscreen mode Exit fullscreen mode

The semantic validator for partial data and empty-result anomalies:

def check_orders(result: SearchOrdersResult) -> ToolError | None:
    if result.has_more and result.page == 1 and len(result.orders) == 0:
        return ToolError(
            error_class="semantic_garbage", code="empty_first_page",
            detail="has_more=true but page 1 returned 0 orders.",
            hint="The query probably matched a filter index but no rows. "
                 "Try a broader date range or check the customer_id format.",
        )
    if result.has_more:
        return ToolError(
            error_class="partial_data", code="more_pages_available",
            detail=f"Page {result.page} returned {len(result.orders)} orders, more exist.",
            hint=f"Call again with page={result.page + 1} to continue.",
        )
    return None
Enter fullscreen mode Exit fullscreen mode

And the agent's dispatcher calls:

result = guarded_call(
    tool=raw_search_orders_api,
    schema=SearchOrdersResult,
    validate_semantics=check_orders,
    customer_id="C-9921",
    page=1,
)
Enter fullscreen mode Exit fullscreen mode

What the model sees back from the tool now is one of:

  1. The validated payload (happy path).
  2. {"error_class": "schema_mismatch", "code": "invalid_json", ...}. The model knows to stop.
  3. {"error_class": "partial_data", "code": "more_pages_available", "hint": "Call again with page=2"}. The model knows exactly what to do.
  4. {"error_class": "semantic_garbage", "code": "empty_first_page", "hint": "Try broader date range"}. The model has a concrete next move.

No stack traces. No raw exception strings. No "Internal Server Error" with no further context.

Why the model handles "error: invalid_status" better than a stack trace

A stack trace is debugging output for a human reading it in a terminal. The model treats it as opaque text, scans for keywords like "error" or "exception," and falls back to its prior: "tool failed, try again."

A structured error with code, detail, and hint reads to the model like API documentation. The model has seen thousands of OpenAPI error responses in training. It knows what invalid_status, expected one of [placed, shipped, delivered, cancelled], got "shipping" means. Don't pass "shipping"; pass one of the four listed values.

Compare the two payloads after a bad status filter:

# Stack trace form
"Traceback (most recent call last):\n  File ...\n  TypeError: ..."

# Structured form
{
  "error_class": "schema_mismatch",
  "code": "invalid_status",
  "detail": "Field `filters.status`: value 'shipping' is not one of "
            "['placed', 'shipped', 'delivered', 'cancelled'].",
  "hint": "Use 'shipped' (past tense) if you want orders that left the warehouse."
}
Enter fullscreen mode Exit fullscreen mode

The first one might prompt three retries with identical args before the model gives up. The second one gets one corrected call.

Pair the guard with a retry budget

The guard prevents the model from looping on shape errors. It doesn't prevent the model from looping on transient errors it correctly identifies as retryable, and that's where you need a budget.

Track per-tool retry counts in the dispatcher. If the same (tool_name, args_hash) pair is called more than N times in a turn, refuse the call and surface a ToolError(error_class="schema_mismatch", code="retry_budget_exceeded"). This caps the worst case at N round-trips even if you missed a failure class in your validator.

A budget of 3 is generous for most tools. Read-only tools can be higher; tools with side effects should be 1 with explicit retry only on classified-transient errors.

This pairs with whatever loop detector you have at the agent level: the budget is per-tool-per-turn, the loop detector is across turns. They catch different things.

The gotcha: strict validators reject valid edge cases

A Pydantic model is only as good as the schema you write. Tight validators feel safer, but they reject responses that are legitimately weird: a customer with no last name, an order with zero line items because of a refund, a payment in a currency you've never seen.

Before deploying a guard to production, run it against a week of real traffic offline. Log every ValidationError and review them by hand. The pattern is always the same: 90% are bugs your validator caught correctly, 10% are real edge cases your validator was too strict about.

Loosen the schema for the 10% (often by switching a field from str to str | None, or widening an enum to a Literal | str union). Don't loosen it for the 90%, because those bugs were what you wrote the guard to find.

The alternative, shipping a strict validator without calibration, gives the agent a new class of failure: "everything returns an error, even when the real tool worked fine." The model loses trust in its tools and starts asking the user for confirmation on every call. That's a worse UX than the original problem.

What's the worst loop you've watched an agent fall into, and what stopped it?

If this was useful

This pattern shows up in the Reliability and Recovery chapter of the AI Agents Pocket Guide: Patterns for Building Autonomous Systems with LLMs, alongside the loop-detector pattern that catches the cross-turn version of the same bug. If you're building agents that run for more than a few turns, the failure-class taxonomy and the guarded-call wrapper save you from the bills nobody warned you about.

AI Agents Pocket Guide: Patterns for Building Autonomous Systems with LLMs

Top comments (0)