DEV Community

Cover image for AI Agent Error Handling: 5 Failures a 200 OK Won't Catch
Rajni Walia
Rajni Walia

Posted on

AI Agent Error Handling: 5 Failures a 200 OK Won't Catch

An AI agent calls an API. The server responds with 200 OK. No exceptions, no timeouts, and no failed requests.

The agent then tells a customer their refund has been approved.

There's just one problem. The refund was never approved.

The API returned valid data, but the agent interpreted it incorrectly.

This is a problem developers face when building production AI agents. A successful tool call doesn't guarantee a successful outcome.

Traditional error handling catches technical failures. AI agents introduce another category of errors: workflows that execute successfully but produce incorrect decisions, incomplete actions, or misleading responses.

Here are five failures that ordinary API monitoring can miss, along with practical ways to prevent them.

1. Successful API Calls Can Still Produce Wrong Decisions

Consider an ecommerce support agent handling a refund request.

It follows a seemingly straightforward workflow:

Customer requests a refund
          |
          v
Retrieve order details
          |
          v
Check refund policy
          |
          v
Determine eligibility
          |
          v
Approve or reject refund
Enter fullscreen mode Exit fullscreen mode

Every step can execute successfully while the final decision is incorrect.

Maybe the customer received a special refund exception. Maybe the policy changed yesterday. Or perhaps the agent misunderstood an eligibility condition.

The problem isn't necessarily with the API.

It's that the application allowed the model to make a consequential decision without independently validating the business rules.

Separate AI Reasoning From Business Authorization

An AI agent can interpret a customer's request and collect relevant information.

However, the application should determine whether an action is authorized.

For example, a refund eligibility check might look like this:

type RefundDecision = {
  approved: boolean;
  requiresReview: boolean;
  reason: string;
};

type Order = {
  daysSinceDelivery: number;
  amount: number;
  hasPolicyException: boolean;
};

type RefundPolicy = {
  returnWindowDays: number;
  autoRefundLimit: number;
};

function checkRefundEligibility(
  order: Order,
  policy: RefundPolicy
): RefundDecision {
  if (order.hasPolicyException) {
    return {
      approved: false,
      requiresReview: true,
      reason: "Policy exception needs review"
    };
  }

  if (
    order.daysSinceDelivery < 0 ||
    order.daysSinceDelivery > policy.returnWindowDays
  ) {
    return {
      approved: false,
      requiresReview: true,
      reason: "Outside standard return window"
    };
  }

  if (order.amount > policy.autoRefundLimit) {
    return {
      approved: false,
      requiresReview: true,
      reason: "Human approval required"
    };
  }

  return {
    approved: true,
    requiresReview: false,
    reason: "Meets standard refund conditions"
  };
}
Enter fullscreen mode Exit fullscreen mode

This is a simplified example, not a complete production refund system. Real applications also need authentication, order ownership checks, fraud controls, and transaction authorization.

The important principle is to keep business rules outside unrestricted model decisions.

This aligns with OWASP's guidance on excessive agency, which recommends limiting agent permissions and independently approving high-impact actions.

Let the model understand the request. Let the application enforce the rules.

2. AI Agents Can Retrieve Outdated Information Without Errors

Retrieval-augmented generation (RAG) allows AI agents to generate responses using documents, databases, and other external sources.

However, retrieving information successfully doesn't guarantee that the information is current.

Imagine a knowledge base containing two refund policies:

refund-policy-2025.pdf
Return window: 30 days

refund-policy-2026.pdf
Return window: 14 days
Enter fullscreen mode Exit fullscreen mode

Both documents remain indexed.

A customer requests a refund 25 days after receiving an order.

The agent retrieves the older policy and says the order is eligible.

The retrieval worked. The response was grounded in a document. But the answer was still wrong.

Validate Knowledge Sources Before Using Them

For policy-sensitive applications, retrieving the closest matching document isn't sufficient.

Developers should also check whether the source is approved, current, and applicable to the request.

A policy record might contain metadata such as:

{
  "document": "refund-policy",
  "version": "2026-04",
  "status": "active",
  "effectiveDate": "2026-04-01",
  "source": "approved-policy-database"
}
Enter fullscreen mode Exit fullscreen mode

The application can use this information to reject expired documents before the model generates an answer.

Useful safeguards include:

  • Maintaining one authoritative source for each policy.
  • Recording document versions and effective dates.
  • Excluding expired content from active retrieval.
  • Detecting conflicting information before taking action.
  • Escalating requests when no reliable source is available.

Metadata only helps when it is maintained correctly. An incorrectly labeled document can still produce a misleading answer.

This is particularly important for agents handling refunds, billing, warranties, account information, and other policy-dependent requests.

3. A Successful API Request Doesn't Mean the Action Is Complete

Suppose an AI agent sends a refund request:

POST /refunds
Enter fullscreen mode Exit fullscreen mode

The payment service responds:

{
  "refundId": "rf_1024",
  "status": "pending"
}
Enter fullscreen mode Exit fullscreen mode

The API accepted the request.

But the refund hasn't been completed.

If the agent tells the customer, "Your refund has been processed successfully," it has made an incorrect claim.

Handle Transaction States Explicitly

Developers should distinguish between an accepted request, an action still processing, and a completed transaction.

An illustrative implementation:

type RefundStatus =
  | "pending"
  | "completed"
  | "failed";

function getRefundMessage(
  status: RefundStatus
): string {
  switch (status) {
    case "completed":
      return "Your refund has been completed.";

    case "pending":
      return "Your refund is being processed.";

    case "failed":
      return "We couldn't complete the refund. Your request needs review.";
  }
}
Enter fullscreen mode Exit fullscreen mode

Real payment providers may use different status values and additional states, so the application must map its messages to the provider's actual transaction model.

For instance, Stripe's Refund API documentation defines refund statuses and related failure information that applications can use when checking outcomes.

Prevent Duplicate Actions During Retries

Another problem appears when the API request times out.

The application doesn't know whether the server completed the action or whether the request failed before execution.

Blindly retrying a refund could initiate the same operation twice.

Where supported, developers should use idempotency keys and reconcile the existing transaction before retrying.

Stripe's idempotent request documentation explains how supported API requests can be retried without unintentionally repeating the same operation.

A reliable workflow should follow this logic:

Refund request
      |
      v
Validate permissions
      |
      v
Submit refund request
      |
      v
Check transaction status
      |
      +-- Completed --> Confirm outcome
      |
      +-- Pending ----> Report pending status
      |
      +-- Failed -----> Investigate or escalate
      |
      +-- Unknown ----> Reconcile before retry
Enter fullscreen mode Exit fullscreen mode

Accepted, pending, and completed are different states. Your agent should never treat them as interchangeable.

4. AI Agents Sometimes Continue When They Should Escalate

Some failures happen because an agent doesn't know when to stop.

Imagine a customer saying:

"I've contacted support three times, and my replacement still hasn't arrived."

The AI retrieves the latest tracking information and provides another delivery update.

Technically, the response may be accurate.

But it doesn't address the actual problem.

The customer has already tried that route and needs someone to investigate.

Define Clear Human Handoff Conditions

A production AI agent should have explicit escalation rules for situations such as:

  • Repeated unsuccessful resolution attempts.
  • Refund or compensation exceptions.
  • Customer requests for human assistance.
  • Conflicting account or transaction information.
  • Actions exceeding predefined authorization limits.

These rules should not depend entirely on the model deciding whether a customer sounds frustrated.

Developers can use automatic validations and human approval checkpoints for sensitive operations.

The OpenAI Agents SDK documentation on guardrails and human review provides one example of how applications can pause workflows for review before consequential actions.

Preserve Customer Context During Escalation

An escalation should transfer more than a ticket number.

For example:

{
  "caseId": "case_1024",
  "orderId": "order_456",
  "escalationReason": "repeated_unresolved_issue",
  "actionsAttempted": [
    "checked_order_status",
    "retrieved_tracking_details",
    "checked_replacement_status"
  ],
  "currentStatus": "requires_human_review"
}
Enter fullscreen mode Exit fullscreen mode

The human support agent should also be able to retrieve the conversation history, relevant records, and previous actions through authorized systems.

Avoid duplicating sensitive customer data in logs when references to protected records are sufficient.

A customer shouldn't have to repeat everything simply because an AI agent transferred the conversation.

Human handoff is not just a fallback message. It's part of the application's workflow design.

5. Automated Ticket Closure Doesn't Guarantee Resolution

This is one of the hardest AI agent failures to catch because dashboards can make the system look successful.

Imagine an AI customer support agent handling 1,000 conversations.

It closes 850 without human involvement.

That's an 85% automated closure rate.

Sounds impressive.

But what if customers reopen many of those tickets because the original issue wasn't resolved?

The automation metric alone won't reveal the problem.

This is a hypothetical example, not a reported performance result.

Track Agent Outcomes, Not Just Tool Execution

Developers need to measure whether an agent completed the customer's intended task, not just whether it executed tools without errors.

Useful metrics include:

Metric What it measures
Tool success rate Whether API calls completed successfully
Task completion rate Whether workflows reached their expected final state
Verified resolution rate Whether the original customer issue was resolved
Recontact rate Whether customers returned with the same problem
Escalation rate How frequently human intervention was needed
Incorrect action rate How often an agent attempted or completed an invalid action
Cost per resolved request Total cost of successfully resolving an issue

These metrics answer different questions.

A tool success rate measures technical execution.

A resolution rate measures the actual customer outcome.

Both matter, but they aren't interchangeable.

Building a Reliable AI Agent Error-Handling Architecture

The five scenarios above share a common problem: the application assumes that successful execution means the agent made the correct decision.

A safer architecture adds validation and verification around model-generated actions.

A Practical Agent Workflow

Customer Request
       |
       v
Understand Intent
       |
       v
Authenticate Customer
       |
       v
Retrieve Approved Data
       |
       v
Validate Business Rules
       |
       v
Authorize Requested Action
       |
       v
Execute Tool
       |
       v
Verify Final State
       |
       +---- Success ----> Confirm Outcome
       |
       +---- Uncertain --> Check Status
       |
       +---- Exception --> Human Review
Enter fullscreen mode Exit fullscreen mode

The model can interpret natural language, identify user intent, and select from approved tools.

But sensitive actions should remain subject to application-level authorization, validation, and outcome verification.

What Should Developers Log?

Traditional application logs are necessary, but AI agents benefit from additional execution context.

Useful fields include:

  • Workflow ID: Identifies the request being processed.
  • Tool-call details: Records which tools executed and whether they succeeded.
  • Knowledge-source references: Identifies documents and policy versions used.
  • Authorization results: Records whether an action was permitted.
  • Transaction status: Distinguishes pending, completed, and failed operations.
  • Escalation reason: Explains why the agent transferred control.
  • Final outcome: Records whether the intended task was completed.

For example, the OpenAI Agents SDK tracing documentation describes how model calls, tool execution, handoffs, and guardrails can be recorded within an agent run.

However, tracing tools do not automatically know whether a business outcome was correct. Developers may still need custom events and evaluation checks.

Logging also needs privacy controls. Avoid storing sensitive information unnecessarily, and apply appropriate redaction, permissions, and retention policies.

Testing AI Agent Failures Before Production

A successful demonstration usually shows what happens when everything works.

Production systems also need to handle missing information, conflicting records, retries, and unexpected customer requests.

Seven Failure Scenarios Worth Testing

Test scenario Expected behavior
Shipping API times out Retry safely or report unavailable status
Customer requests someone else's order Deny unauthorized access
Two refund policies conflict Use the authoritative source or escalate
Refund API returns pending Avoid claiming the refund is complete
Customer requests human support Transfer with relevant context
Tool returns incomplete data Avoid unsupported conclusions
Refund request is retried Prevent duplicate transactions

Run these tests whenever you change a prompt, model, integration, or business rule.

An effective regression test should inspect the final response and the intermediate actions the agent attempted.

For example, a test should fail if the agent tries to execute an unauthorized refund, even if the final message politely tells the customer that their request requires review.

This is where tracing and outcome-based evaluation work together.

AI Agent Reliability Goes Beyond Ecommerce

Retail customer service is a useful example because it combines simple information retrieval with actions that have real consequences.

Order tracking and inventory checks are generally good candidates for automation when reliable data is available.

Refunds, billing disputes, and unusual policy exceptions require stronger controls.

I covered these business considerations in more detail in AI in Retail Customer Service: Where AI Fits and Where It Doesn't, including use cases, limitations, and human handoff requirements.

But the same engineering problems appear in other applications.

An AI scheduling assistant shouldn't confirm an appointment before the calendar system accepts it.

An AI financial assistant shouldn't report a transfer as completed while the transaction is pending.

An AI coding agent shouldn't declare a task finished simply because its commands exited without errors.

Different applications, same underlying issue.

Final Thoughts

AI agent error handling isn't just about catching exceptions, managing API timeouts, or retrying failed requests.

It's about recognizing when a technically successful workflow produces an incorrect, incomplete, or unauthorized outcome.

Reliable agents need validated data, restricted permissions, verified actions, and clear escalation rules. They also need monitoring that captures what happened and evaluation that determines whether the result was correct.

A 200 OK tells you the request succeeded at the protocol level.

It doesn't tell you whether the agent solved the problem.

I'm curious how other developers handle this.

When building AI agents that can take real actions, do you let the model decide what is allowed, or do you enforce business rules and authorization outside the model?

Top comments (0)