DEV Community

Cover image for I Built an AI Agent That Worked in Testing but Failed in Production
Aarti Jangid
Aarti Jangid

Posted on

I Built an AI Agent That Worked in Testing but Failed in Production

Building an AI agent that works in a development environment feels like a major achievement. The agent understands instructions, communicates with external tools, generates accurate responses, and completes tasks with minimal intervention.

But getting an AI agent to work locally is very different from making it reliable in production.

Imagine spending weeks building an AI-powered customer support agent. It passes every initial test, responds accurately to customer questions, and retrieves information from a knowledge base.

*Then you deploy it.
*

Suddenly, responses become slower, API requests fail, the agent selects incorrect tools, and users begin reporting inconsistent results.

This is an illustrative account of how that failure can happen, what the debugging process looks like, and what developers should consider before deploying agentic AI systems.
The AI Agent I Was Building
The goal was simple: create an AI customer support agent capable of answering common questions and performing basic support operations.

The architecture included:

  • A large language model for understanding user requests.
  • A retrieval system for accessing support documentation.
  • External APIs for checking order information.
  • A tool-calling mechanism for executing actions.
  • A Python backend for handling requests.
  • Logging for monitoring agent activity. The development version worked with a limited dataset and simulated API responses.

For example, a user could ask:
Where is my order, and when will it arrive?

The agent would identify the request, call the order-tracking function, retrieve shipment details, and generate a response.

During local testing, the process appeared stable.

The problems became visible when the same workflow encountered real traffic, inconsistent external responses, and production infrastructure limits.
Problem 1: The Agent Worked With Mock APIs but Failed With Real Services
One of the biggest mistakes in AI agent development is assuming that successful integration tests guarantee production reliability.

During testing, I imagined using a mock API that always returned predictable data.
def get_order_status(order_id): return { "order_id": order_id, "status": "shipped", "delivery_date": "2027-01-15" }

*Every request returned the expected structure.
*

However, real APIs behave differently. They can return errors, incomplete responses, authentication failures, or temporary service interruptions.

A production API might respond with:
{
"error": "Service temporarily unavailable",
"status_code": 503
}

Without appropriate error handling, the agent may attempt to interpret this response as valid order information.
How I Would Fix It
The first improvement would be adding timeout handling, bounded retries, and response validation.
import requestsfrom requests.exceptions import RequestExceptiondef get_order_status(order_id): url = f"https://api.example.com/orders/{order_id}" try: response = requests.get( url, timeout=5 ) response.raise_for_status() data = response.json() if not isinstance(data, dict) or "status" not in data: return { "success": False, "error": "Invalid response format" } return { "success": True, "data": data } except (RequestException, ValueError): return { "success": False, "error": "Order service unavailable" }

In a real application, I would also include authenticated API access, authorization checks, request tracing, and retries limited to safe operations.

The lesson is simple: AI agents must be prepared for external systems to fail.
Problem 2: The Agent Selected the Wrong Tools
Tool calling makes AI agents powerful.

Instead of generating text only, an agent can interact with APIs, databases, calendars, and other services.

However, tool selection is not always consistent.

Imagine an agent with these tools:
tools = [ "get_order_status", "cancel_order", "create_support_ticket", "update_customer_profile"]

During testing, the model might correctly select get_order_status when a customer requests shipping information.

In production, a slightly ambiguous request could lead to an incorrect tool choice.

For example:
My order is late. Can you do something about it?

The agent might incorrectly interpret this as authorization to cancel an order.

That creates a serious business risk.
My Approach to Safer Tool Execution
I would separate the model's proposed action from the application's permission to execute it.
SAFE_READ_TOOLS = { "get_order_status",}CONFIRMATION_REQUIRED_TOOLS = { "cancel_order", "update_customer_profile",}def execute_tool(tool_name, arguments, confirmed=False): if tool_name in CONFIRMATION_REQUIRED_TOOLS: if not confirmed: return { "success": False, "error": "User confirmation required" } if tool_name not in SAFE_READ_TOOLS | CONFIRMATION_REQUIRED_TOOLS: return { "success": False, "error": "Tool not allowed" } return dispatch_tool(tool_name, arguments)

Here, dispatch_tool represents an application-controlled function registry.

For production use, confirmation must come from a trusted application workflow, not a boolean the language model can set itself. The backend should also validate arguments, authenticate the user, and check permissions for the specific resource.

The language model can recommend an action, but the application must control execution.
Problem 3: Response Times Increased Under Real Traffic
An AI agent that responds within a few seconds during testing may become considerably slower when multiple users interact with it simultaneously.

This happens because agentic workflows often involve several operations:

  1. Interpret a user request.
  2. Retrieve relevant information.
  3. Select an appropriate tool.
  4. Execute an external API call.
  5. Process the returned data.
  6. Generate a final response. Each operation adds latency.

If an agent repeatedly calls the model or external tools, response times can grow quickly.
What I Would Change
I would start by measuring latency at every stage.
import timeimport logginglogger = logging.getLogger(name)def timed_operation(name, operation): start = time.perf_counter() try: return operation() finally: duration = time.perf_counter() - start logger.info( "operation=%s duration_seconds=%.3f", name, duration )

This helps identify whether delays originate from model inference, database retrieval, external APIs, or application logic.

For production environments, I would also introduce concurrency limits, request timeouts, safe caching, and a maximum number of agent steps.

Importantly, caching should respect authorization boundaries so information retrieved for one user is not accidentally exposed to another.
Problem 4: The Agent Produced Inconsistent Answers
AI agents rely on probabilistic language models.

Even when users submit similar requests, the generated answers may differ.

This becomes particularly problematic in customer support, financial services, healthcare, or other environments where accuracy matters.

For example, the agent might receive incomplete information about a customer's refund eligibility.

Instead of explaining that the information is insufficient, it could generate an unsupported answer.
What I Learned About Grounding
For business-critical applications, agent responses should be grounded in reliable data.

A retrieval-augmented generation system can help by retrieving relevant information from an approved knowledge base before producing an answer.

*However, RAG does not eliminate hallucinations automatically.
*

I would evaluate:

  • Whether the retrieved documents actually answer the question.
  • Whether the documents are current and accessible to that user.
  • Whether the response accurately reflects retrieved evidence.
  • Whether the agent knows when to escalate to a human. When sufficient information is unavailable, the agent should acknowledge uncertainty rather than manufacture an answer. Problem 5: Local Testing Did Not Reflect Production Security Security is another major difference between prototypes and production applications.

A development environment might contain test credentials, sample customer information, and unrestricted access to tools.

Production systems contain real user data and potentially sensitive business information.

An AI agent deployed without proper security controls may introduce risks involving prompt injection, unauthorized tool execution, credential exposure, and cross-user data leakage.

For example, a malicious document retrieved by the agent might contain instructions attempting to override the system's intended behavior.

Developers must treat retrieved documents, tool outputs, and user-provided content as untrusted data.

A strong security strategy should include least-privilege permissions, restricted tool access, secret management, output validation, and ongoing security testing.

The OWASP Top 10 for LLM Applications provides useful guidance for identifying these risks.
The Production Readiness Checklist I Would Use
Before deploying another AI agent, I would evaluate the following areas.
Area What to verify
API reliability Timeouts, retries, error responses
Tool execution Permissions and confirmation requirements
Model behavior Accuracy and unexpected tool choices
Retrieval Relevant, current, authorized data
Security Prompt injection and data isolation
Performance Latency, concurrency, and resource limits
Monitoring Logs, traces, error rates
Cost Model usage and tool execution expenses
Recovery Fallback workflows and incident handling

*Testing should include more than successful examples.
*

I would deliberately simulate unavailable APIs, incomplete responses, ambiguous requests, adversarial prompts, and concurrent users.

The objective is not merely to prove that the agent can complete a task. It is to understand how it behaves when it cannot.
What I Would Do Differently Before the Next Deployment
The biggest change would be treating production readiness as part of development rather than a final deployment activity.

I would begin with a smaller number of tools, define strict execution permissions, and use evaluation datasets representing real user behavior.

I would also introduce structured observability from the beginning.

Instead of logging only the final response, I would track the agent's tool decisions, execution times, failed operations, and escalation events—while minimizing the collection of sensitive information.

Finally, I would use a staged rollout. A limited group of users could evaluate the system before introducing larger workloads.

This approach makes failures easier to detect and reduces the impact of unexpected agent behavior.

**Final Thoughts
**Building a functional AI agent is only the beginning.

The real engineering challenge is making that agent reliable, secure, observable, and predictable when it interacts with production systems.

A successful demo may show what an AI agent can do. A production deployment reveals what happens when the environment stops behaving as expected.

For developers working on agentic AI applications, the most important lesson is to test failure scenarios as thoroughly as successful workflows.

Because the question is no longer just:
Can my AI agent complete this task?

It is:
Can my AI agent handle the task safely and reliably when something goes wrong?

Top comments (1)

Collapse
 
digitalgh0st profile image
Digital Ghost •

😁