Since AI agents operate with complex decision chains and external tool integrations, understanding their behavior and troubleshooting their errors requires a different approach than traditional system monitoring. The non-deterministic nature of these systems, prompt variations, and multi-layered tool usage make it critical to understand whether agents are performing as expected, how much they cost, and where they get stuck. Establishing an effective observability strategy is essential to ensuring the reliability, efficiency, and cost-effectiveness of agent-based applications.
In this post, we will detail the necessary tracing mechanisms, cost management approaches, and how to detect and resolve tool errors for monitoring AI agents. Our goal is to ensure your agents are transparent, predictable, and controllable in a production environment.
Why is AI Agent Observability Critical?
AI agents are autonomous systems that use Large Language Models (LLMs) to dynamically plan steps, call external tools, and operate within feedback loops to achieve specific goals. While a specific code path in traditional software systems is usually static and predictable, AI agents can follow a different path in each run due to different prompts, LLM output variations, and tool calls. This variability makes it difficult to understand why an agent made a particular decision, why it failed, or how many resources it consumed.
This complex structure means that traditional logging and metric collection methods are insufficient. Monitoring only the final output or general system metrics is not enough to reveal problems or suboptimal behaviors in the agent's internal workings. Therefore, detailed monitoring of every step of the agent, its decisions, the tools it uses, and its LLM interactions—that is, providing "observability"—is vital for the development, maintenance, and optimization of agent-based applications.
💡 Difference from Traditional Software
In traditional software, the output of a function is usually always the same for the same inputs. In AI agents, however, different LLM responses or tool selections can occur despite the same inputs. This non-deterministic nature deeply affects monitoring strategies.
Challenges Posed by AI Agent Architectures and Limitations of Traditional Monitoring
AI agent architectures present unique monitoring challenges due to their internal logic being driven by LLMs and their use of various tools for interacting with the external world. Understanding an agent's behavior must encompass not only the code execution flow but also the LLM's thought process, prompt effectiveness, and the success/failure status of tools. This layered structure complicates troubleshooting and performance optimization.
Traditional system monitoring tools typically collect metrics such as function call frequency, CPU/memory usage, or error codes. However, for an AI agent, this information is insufficient. For example, an agent's continuous high CPU usage does not indicate whether the problem stems from the LLM call itself or from the LLM constantly choosing the wrong tool. Furthermore, a small change in prompt engineering can alter the agent's entire behavior, and tracking the effects of such changes requires deeper, contextual data.
⚠️ The Black-Box Problem
The "thinking" processes of agents (the intermediate steps of the LLM) are often like a black box. Making this internal operation transparent is critical for diagnosing problems and increasing the agent's reliability.
These limitations necessitate the need to record each step of the agent as a "trace." These traces allow us to see the entire process, from the prompt to the LLM response, tool selection, and tool execution result, as a whole. While developing the backend for my own AI-powered side product, especially when switching between different LLM providers (Gemini Flash, Groq, OpenRouter), I needed such in-depth traces to monitor how successfully the agent worked with each provider and what costs it incurred. This allowed me to detect not only performance regressions but also cost regressions at an early stage.
Monitoring Agent Behavior: Step-by-Step Tracing Mechanisms
To understand the complex and dynamic nature of AI agents, every step and decision must be traceable. This is possible with tracing mechanisms that record the agent's "thinking" process and tool interactions from start to finish. Standards like OpenTelemetry define how traces should be created and correlated in such distributed systems. Each LLM call, tool selection, and tool execution by an agent can be represented as a separate "span" within a trace.
Trace Segmentation and Correlation
An agent's operation typically involves multiple LLM calls and tool usages. Each of these steps should be recorded as a separate span and correlated under the same trace_id. This allows us to visualize all agent activity resulting from a single user request, step by step. For example, a planning span might contain multiple LLM_call and tool_invoke spans. This hierarchical structure clearly shows at which stage the problem occurred.
The Mermaid diagram below illustrates the workflow and trace segmentation of a simple AI agent:
In this diagram, each box represents a span, and arrows indicate the flow. The trace_id links all these spans together, while span_id is the unique identifier for each step. This structure allows us to analyze why an agent chose a particular tool, how the LLM responded to the prompt, and where the process got stuck in a bottleneck.
Capturing Prompt and LLM Interactions
For every LLM call made by the agent, it is vital to record the entire prompt used, the parameters sent to the LLM (temperature, max_tokens, etc.), the response returned by the LLM, and the token count in that response. This information is used to evaluate the results of prompt engineering experiments, diagnose unexpected LLM behaviors, and perform cost analyses. Especially, the prompt history is indispensable for understanding the root cause of issues like "hallucination" or incorrect tool selection in LLM responses.
Managing Costs: FinOps Approach and Token Monitoring
AI agents, especially when using large language models and external APIs, can incur significant costs. These costs typically vary based on the number of tokens used by your LLM, the number and duration of API calls made. Therefore, monitoring and optimizing agent costs means integrating FinOps principles into AI applications. Transparently viewing the cost of each agent run is a critical step to prevent unnecessary spending and ensure budget control.
LLM Token Costs
Most LLMs charge based on the number of input (prompt) and output (response) tokens. These costs can rapidly increase depending on how much the agent "thinks" and how detailed a response it generates. Therefore, it's important to record the prompt tokens and completion tokens used for each LLM call. With this data, we can identify which prompts consume more tokens, whether the LLM is generating unnecessarily long responses, or if it's getting stuck in a loop and spending excessive tokens.
ℹ️ Importance of Token Costs
Token costs vary by LLM provider and model type. For example, with one provider, 1M input tokens might cost $1, and output tokens $3. For most LLM providers, output tokens are 3 to 6 times more expensive than input tokens. This means even small optimizations can lead to significant savings in the long run.
Tool Usage Costs
External tools used by agents (e.g., a search engine API, a database query, an external financial calculator API) also come with their own costs. These costs typically vary based on the number of calls, data transfer, or processing time. Monitoring which tool made each tool call, how long it took, and any associated costs provides a complete view of the agent's overall cost profile.
The table below shows an example of how cost metrics can be collected within a trace:
| Metric Name | Description | Example Value (LLM) | Example Value (Tool) |
|---|---|---|---|
trace_id |
Unique identifier for the entire operation | abc-123 |
abc-123 |
span_id |
Unique identifier for the step in the operation | span-001 |
span-002 |
event_type |
Type of event (LLM call, tool execution) | llm_call |
tool_invoke |
model_name |
LLM model used (if any) | gpt-4o-mini |
N/A |
tool_name |
Name of the tool used (if any) | N/A | search_web |
prompt_tokens |
Number of tokens in the input prompt | 250 | N/A |
completion_tokens |
Number of tokens in the LLM response | 120 | N/A |
total_tokens |
Total number of tokens | 370 | N/A |
cost_usd |
Estimated cost for this step (USD) | 0.0005 | 0.0002 |
latency_ms |
Latency of this step (milliseconds) | 800 | 1500 |
Such detailed cost metrics show which agent flows are more expensive, which tools reduce cost-effectiveness, and where to focus for cost optimization. In my own system, I developed strategies to dynamically optimize costs by using different LLM providers; this was only possible with such detailed FinOps data.
Strategies for Detecting and Resolving Tool Errors
One of the most powerful aspects of AI agents is their ability to interact with the real world using external tools. However, these integrations also introduce potential points of failure. If an agent selects the wrong tool or if the called tool does not function as expected, the entire agent flow can fail. Early detection and resolution of such errors are critical for the agent's reliability.
Tool Selection Errors
Agents decide which tool to use based on the LLM's reasoning for a given task. Sometimes, the LLM might select the wrong tool or fail to select any tool at all due to incorrect prompt interpretation or insufficient information. This can lead to the agent entering a meaningless loop or failing to complete the task. To monitor such errors, it's necessary to record the LLM's "thought" process for why it chose that tool and the name of the selected tool at each tool selection step.
🔥 Incorrect Tool Selection
An agent choosing a "database query" tool for a "get weather" task is a typical tool selection error. This indicates that the LLM misunderstood the prompt or the available toolset.
Tool Execution Errors
Even if the agent selects the correct tool, the tool itself can fail for various reasons. These errors range from network outages to API limits, incorrect input parameters, or external services being down. It's important to record the inputs sent to the tool, the returned output (successful or erroneous), and any error messages at each tool execution step. This data helps us understand whether the error indicates a flaw in the tool's internal logic or a problem with an external dependency.
In a customer project, an AI-powered planning agent used in a production ERP system was constantly receiving time-out errors when calling an external supply chain integration tool. When we examined the traces, we saw that the agent had selected the correct tool, but the API being called by the tool was not responding. This revealed that there was no problem with the agent itself, but that the observability of the external dependency was also critical. In such cases, the agent needs to monitor not only its own errors but also the errors of the systems it depends on.
Practical Monitoring Applications and Tool Integrations
Monitoring AI agents requires specially designed solutions and the adaptation of existing observability tools. Popular agent frameworks like LangChain and LlamaIndex offer built-in mechanisms to collect such monitoring data. Through these mechanisms, we can capture the agent's internal steps and send them to a central monitoring system.
Integration with Callback Handlers
Frameworks like LangChain and LlamaIndex provide "callback handler" mechanisms that respond to different agent lifecycle events (LLM call start/end, tool call start/end, etc.). These handlers can be used for custom logging, metric collection, or creating trace spans.
Below is a simple example of a custom callback handler for LangChain:
from langchain.callbacks.base import BaseCallbackHandler
from typing import Any, Dict, List, Optional, Union
class CustomAgentTraceHandler(BaseCallbackHandler):
"""Custom callback handler for monitoring AI Agent steps."""
def on_llm_start(
self, serialized: Dict[str, Any], prompts: List[str], **kwargs: Any
) -> None:
"""Triggered when an LLM call starts."""
print(f"LLM call started. Model: {serialized.get('name')}, Prompt: {prompts[0][:100]}...")
# Here, an OpenTelemetry span can be started or logging can be performed.
# For example: start_new_span("llm_call", attributes={"model": serialized.get('name'), "prompt": prompts[0]})
def on_llm_end(self, response: Any, **kwargs: Any) -> None:
"""Triggered when an LLM call ends."""
# In current versions of LangChain, token_usage info is usually found within response.llm_output.
token_usage = response.llm_output.get('token_usage') if response.llm_output else None
print(f"LLM call ended. Token Count: {token_usage}")
# Token information can be added to the current span and then ended.
# For example: current_span().set_attributes({"input_tokens": ..., "output_tokens": ...}).end()
def on_tool_start(
self, serialized: Dict[str, Any], input_str: str, **kwargs: Any
) -> None:
"""Triggered when a tool call starts."""
print(f"Tool call started. Tool: {serialized.get('name')}, Input: {input_str[:100]}...")
# A new span can be started here.
def on_tool_end(
self, output: str, observation_uuid: Optional[str] = None, **kwargs: Any
) -> None:
"""Triggered when a tool call ends."""
print(f"Tool call ended. Output: {output[:100]}...")
# The output can be added to the span and then ended.
def on_agent_action(self, action: Any, **kwargs: Any) -> Any:
"""Triggered when the agent takes an action (like tool selection)."""
print(f"Agent action: {action.tool} with '{action.tool_input}'")
def on_agent_finish(self, finish: Any, **kwargs: Any) -> Any:
"""Triggered when the agent completes its task."""
print(f"Agent finished. Result: {finish.return_values['output'][:100]}...")
# Example of running an agent with this callback handler
# agent = initialize_agent(...)
# agent.run("task", callbacks=[CustomAgentTraceHandler()])
This example shows that you can add custom logic for every significant event (on_llm_start, on_tool_end, etc.). In a real scenario, you would integrate these events with the OpenTelemetry SDK to create spans, add relevant data to these spans (prompt, response, token count, tool name, input, output, cost), and send them via an OpenTelemetry Collector to an APM (Application Performance Monitoring) solution.
The Role of Existing APM Solutions
Existing APM solutions (Datadog, New Relic, Grafana Tempo, etc.), as long as they are compatible with standards like OpenTelemetry, can be used to visualize and analyze AI agent observability data. These platforms can visualize traces, collect metrics, and store logs in a central location. A flame graph or Gantt chart showing the agent's complex flows step-by-step significantly speeds up troubleshooting processes. Furthermore, we can set up alerts through these tools based on specific thresholds (e.g., an agent run exceeding 5 seconds or an LLM call exceeding 1000 tokens).
Conclusion
The complex and dynamic nature of AI agents necessitates a specialized observability strategy that goes beyond traditional monitoring approaches. Tracing every step of the agent, managing costs in detail, and proactively detecting tool errors are indispensable for increasing the reliability, cost-effectiveness, and overall performance of these systems. Thanks to standards like OpenTelemetry and the callback mechanisms of frameworks, it is possible to integrate such in-depth monitoring capabilities into your applications. It should be remembered that the success of an AI agent in a production environment depends not only on making correct decisions but also on transparently monitoring why these decisions were made and what resources they consumed. This allows you to continuously optimize your agents and ensure they serve your business goals more consistently.
Top comments (0)