When Your Agents Go Dark: Observability in Multi-Agent Systems with OpenTelemetry

This article was written by Diego Freniche.

In recent months, AI applications have radically evolved. Earlier, prototypes looked like a single loop: prompt, model call, optional tool call, response. In production, an agent often becomes a small system: model calls, tool calls, routing logic, memory, retries, and error handling. We can have an agent that acts as a router: delegating tasks to other specialized agents; a retrieval agent that fetches information from databases and sends it to an agent responsible for reasoning; or an agent that critically reviews the work of other agents. Each of these agents will have its own model calls, its own reasoning and execution logic, and at each of these points, a potential point of failure and an error-handling mechanism.

The parallel with traditional systems is more applicable today than ever. Once two agents coordinate through a handoff, a queue, a tool call, or shared state, the system has distributed-system failure modes.
That means the debugging problem changes. The hard part is no longer fixing the failing step; it is finding which step failed and why.

Years of production incidents have given us an operational model built on logs, metrics, traces, and alerts tied to expected service behavior. For new AI applications that use agents, we often have the same tools, but not the trace structure (or attributes) needed to answer the question: what happened and why?

Logs can show local events, but they do not show the request path unless those events are correlated into a trace.

The standard for producing those correlated traces is OpenTelemetry (OTel): an open-source, vendor-neutral observability framework hosted by the Cloud Native Computing Foundation that defines a common way to generate and export telemetry. It gives us a protocol for shipping data (OTLP) and a shared vocabulary of attribute names and semantic conventions, so that a span means the same thing across tools. The rest of this article applies it to multi-agent systems.

This article walks that path in order: the failure modes that logs miss, how to instrument an agent handoff in code, what the resulting trace looks like in a real run, and when the simpler approach is still the right one.

How We Got Here

Let’s take it one step at a time to understand what we’ve done so far and the main issues with this approach. This is essential for us to understand how we can move toward full observability.

The first agent that most teams develop consists of a single loop: a prompt, a call to a model, a call to a tool (if needed), and a response. Debugging in this scenario is straightforward: log the prompt, log the calls to the tools and the model, and finally, the response. If something goes wrong along the way, we can review all the logs linearly and figure out where the error is. Everything lives within the same process, like a traditional monolithic application.

For an agent this simple, structured logging is enough, and the tracing infrastructure described later would add cost with little return.

The system grows and becomes more complex. Instead of a single, all-purpose agent, we now have a router that delegates tasks to specialized agents. In addition to the various specialized agents, we need to add a memory management service to persist the context and an orchestrator to manage everything consistently.

Figure 1. A representative multi-agent orchestration. One request spans five services, and without a shared trace ID, their logs cannot be correlated into a single request path.

This is a representative example of a common orchestrator pattern, where a router delegates to specialized agents, rather than a specific product.

In a scenario like this, each box shown in the diagram logs to its own standard output, and execution is no longer linear. Sometimes control passes to an agent that returns its response. At other times, two or more agents may work in parallel. The earlier mental model of reading logs backward no longer holds because the request path crosses agent boundaries and may branch.

Four Problems That Compound

1. The correlation Problem

When five or more agents each generate their own logs, we end up with a timeline of uncorrelated events. Reconstructing the path of a single user request means stitching events across services, comparing timestamps, and guessing which call belongs to which step. Parallel calls make timestamps unreliable, too. This holds whether a person or an automated, agentic log-analysis tool performs the reading. In both cases, the events lack a trace ID, so any reader must infer the request path.

The cost of this shortcoming can be measured using one of the DORA (DevOps Research and Assessment) metrics: Mean Time To Resolve (MTTR). A small diagnosis task can become a multi-hour reconstruction exercise when most of the effort is spent piecing together and connecting the dots of what happened rather than resolving the problem.

2. The Causality Problem

Even when we find the logs relevant to the incident, we only have part of the story. We know what happened, but we don’t always know why. We can see that an agent returns a particular response. Still, we cannot determine if what we receive is generated by a truncated context window, a retrieved document that was actually deprecated, or a threshold set too generously, allowing the model to hallucinate.

Because execution paths can vary, traces need to capture the decisions made along the way: tool choice, arguments, retrieved sources, model parameters, and token use. The same input can produce two different execution paths, depending on what the agent decides to do in a fully autonomous, non-deterministic manner.

3. The Cost Problem

The cost principle for an agent is very simple: every agent interaction is a model call, and every model call consumes tokens. In a single-agent system, calculating the cost is straightforward because there is a single stream of requests associated with a single cost account.

In a multi-agent system, however, a single user request can trigger an indefinite number of agent actions, with subsequent model calls, each of which may affect the responses provided by the LLM and potentially trigger further calls. A common pattern amplifies this: in an agentic evaluation loop, an agent reviews its own answer (or the evidence supporting it) and retries until it is satisfied. The result is a multitude of model calls that consume a high number of tokens. Attributing that cost back to the agent that generated it becomes difficult.

4. The Latency Problem

Latency is another concern. In multi-agent systems, there are numerous potential sources of delay: serialization/deserialization of responses or calls, a slow-responding model, or a poorly configured timeout.

When responses slow down, span timing shows whether time is spent on retrieval, model calls, tool execution, serialization, or waiting on another agent.

Tracing Agent Handoffs with OpenTelemetry

We have seen a version of this before. It is the microservices observability problem in new clothing. That is good news because distributed tracing provides a useful base, and OpenTelemetry is the standard way to model and export those traces. Agent orchestration adds GenAI-specific attributes on top:

• A trace represents a user request from the moment the prompt is typed until the agent’s final response.
• A span represents a single unit of work, such as invoking an agent, calling a tool, retrieving information, or calling a model.
• Each span is embedded in a parent-child tree, creating a hierarchical relationship that starts with the orchestrator’s span and extends down to the individual operation performed by specialized agents.
• Context propagation carries the trace ID across process and service boundaries. This allows spans to be joined into a single request trace.

OpenTelemetry now includes semantic conventions for Generative AI. It records model calls, token usage, prompts, completions, tools, and related events. This standardized vocabulary includes various attributes such as the model name, the number of tokens used, and the type of operation performed. That keeps the instrumentation portable across open-source and proprietary backends.

Figure 2. Proposed instrumentation: one correlated trace per request

Each agent emits a span under the orchestrator span. The application exports them over OTLP to an OpenTelemetry collector that forwards traces to a trace backend and metrics to a metrics backend. The trace backend reassembles the spans into a single trace per request, so the full causal path is visible in one place.

Instrumenting an Agent Handoff

Building on those tracing concepts, we can now instrument an actual agent handoff in code. The principle behind this strategy is straightforward: wrap every meaningful unit of work within a span, and track the relevant information about that specific unit within the span’s attributes.

Let’s illustrate this situation with an example, using an agent written in Java and instrumented with the OpenTelemetry APIs. In this example, the agent is delegating the task to a specialized agent.

Span orchestratorSpan = tracer.spanBuilder("agent.orchestrator")
        .setSpanKind(SpanKind.INTERNAL)
        .startSpan();


try (Scope scope = orchestratorSpan.makeCurrent()) {
    // Child span for the model call, following GenAI semantic conventions
    Span policySpan = tracer.spanBuilder("chat policy-agent")
            .setSpanKind(SpanKind.CLIENT)
            .setAttribute("gen_ai.operation.name", "chat")
            .setAttribute("gen_ai.system", "openai")
            .setAttribute("gen_ai.request.model", "gpt-4o")
            .setAttribute("gen_ai.request.temperature", 0.2)
            .startSpan();


    try (Scope policyScope = policySpan.makeCurrent()) {
        PolicyResult result = policyAgent.evaluate(context);


        // Record the decision and the cost         policySpan.setAttribute("gen_ai.usage.input_tokens", result.promptTokens());
        policySpan.setAttribute("gen_ai.usage.output_tokens", result.completionTokens());
        policySpan.setAttribute("agent.policy.version", result.policyVersion());
        policySpan.setAttribute("agent.retrieval.doc_ids", result.sourceDocIds());
    } catch (Exception e) {
        policySpan.recordException(e);
        policySpan.setStatus(StatusCode.ERROR);
        throw e;
    } finally {
        policySpan.end();
    }
} finally {
    orchestratorSpan.end();
}

From this example, we can see that the child span is created while the orchestrator’s current span is active, which allows OpenTelemetry to link the two spans within the same trace.

A note on versions: the OpenTelemetry semantic conventions for Generative AI are still in Development status and may change between releases. The attribute names used here follow version 1.42, pinned in the companion repository. Confirm the current version before reusing these names. Some attributes are still being renamed or relocated.

What to Capture on Each Span

It is not necessary to record every attribute included in the convention, though it is worth maintaining a minimal subset that serves as a baseline. In this way, we can transform diagnostic traces into something more meaningful and useful for our purposes.

Concern	Attribute (key)	Why it matters
Identity	gen_ai.system, gen_ai.request.model	Attribute behavior to a specific model/provider
Cost	gen_ai.usage.input_tokens, gen_ai.usage.output_tokens	Per-span token spend, summable across the trace
Determinism	gen_ai.request.temperature, top_p	Explain non-reproducible outputs
Causality	agent.retrieval.doc_ids, http://agent.tool.name/ (custom attributes)	Tie an answer back to its source and decisions
Routing	http://agent.name/, agent.handoff.reason (custom attributes)	Reconstruct why the orchestrator delegated where it did
Errors	Span status + recordException	Surface failure inside loops that logs would swallow

Many of these attributes do not need to be set manually when using Spring AI. Spring AI is the Spring team's framework for building AI applications in Java that wraps model providers with a common set of abstractions. For observability, it builds on Micrometer, the metrics and tracing facade used across the Spring ecosystem. Micrometer bridges to OpenTelemetry, so the data it produces can be exported over OTLP.

On Spring AI 1.0, add two dependencies to the pom.xml (spring-boot-starter-actuator) plus a Micrometer-OTel bridge, such as micrometer-tracing-bridge-otel. The framework records Micrometer observations for supported model calls (gen_ai.client.operation) and tool invocations (spring.ai.tool), along with token-usage metrics (gen_ai.client.token.usage), without manually wrapping each call in tracing code. Those observations become spans and metrics through Micrometer's bridge to OpenTelemetry.

The manual spans created above, on the other hand, serve to show the framework what it cannot see: the orchestration layer. Spring AI does not know the decisions the orchestrator makes between model calls. It doesn’t know which routing logic was chosen or why a specific agent was called rather than another. These decisions lie within the application’s business logic and not in the framework, so they require explicit instrumentation.

Observability In Action: A Real Trace

Jaeger is an open-source distributed tracing backend, hosted by the Cloud Native Computing Foundation. It collects spans and displays them as a trace tree. We use it here to see what the instrumentation produces. We export the agent's spans to Jaeger over OpenTelemetry and run a sample orchestration. Any OpenTelemetry-compatible backend would work the same way. Jaeger is convenient because it is open-source and quick to run locally.

Figure 3. A single orchestration request as one correlated trace in Jaeger: the orchestrator span fans out to the retrieval, policy, and summarizer agents, and the selected model-call span carries the GenAI attributes, such as model name, provider, and token usage

Let’s see how a single orchestration trigger call gives rise to multiple spans, some of which contain calls to models (in this case, Claude Haiku).

The left column shows the trace tree. At the beginning is the incoming HTTP call, followed by the span related to the agent.orchestrator. It’s split into three child spans: chat-retrieval-agent, chat-policy-agent, and chat-summarizer-agent. Each of these is further split by Spring AI’s auto-instrumentation into a spring_ai_client call and an outbound call to the model. The timing bar shows the sequence of operations: retrieval, then policy application, then summarization.

Within each trace, we can view the captured attributes according to OpenTelemetry's conventional semantics. Here, gen_ai.system identifies Anthropic as the provider, while gen_ai.request.model and gen_ai.response.model give the requested and returned model names, and gen_ai.usage.input_tokens (140) along with gen_ai.usage.output_tokens (59) report the cost per span. Finally, gen_ai.response.finish_reason indicates that the call completed successfully without errors.

This trace shows which agent spent the tokens, which documents were retrieved to inform the policy decision, and where latency accumulated.

When The Classic Approach Still Makes Sense

This setup is not free, and it introduces dependencies: a collector to install, manage, and update; a backend to store traces; and a small amount of latency and compute overhead within each instrumented agent.

As noted earlier, a single agent with a single prompt call and a couple of tools is well served by structured logging, and the setup described here is not worth its cost there.

The same reasoning applies to internal prototypes with few users operating in purely experimental contexts. The issue is always complexity and the number of moving parts. The goal is to have observability strategies suited to the context, not ones that are complex because best practices dictate so.

The Alternative: Build on a Standard or Buy a Platform

OpenTelemetry is not an observability product, but rather a protocol and a vocabulary. The next decision is whether to build the tracing layer ourselves or buy an observability platform. The market is currently full of observability platforms that now support observability for interactions with LLMs. To make informed choices, it is necessary to understand the fundamental difference between value and vendor lock-in for each tool.

All of the platforms below interoperate with OpenTelemetry. What differs is how each engages with it; that difference shapes the lock-in.

• LangSmith, from the LangChain team, is the most ecosystem-native option. Within LangChain/LangGraph, it captures traces, prompts, and token usage, and adds built-in evaluation tools. Its OpenTelemetry support is a bridge layered on a proprietary SDK, since the native instrumentation is LangChain-specific, and OTLP import and export were added later. The trade-off is the dependency on a proprietary hosted platform, whose zero-setup convenience pays off mainly if we stay inside the LangChain ecosystem.
• Langfuse occupies a middle ground. It is an open-source, self-hostable product focused on LLM tracing, prompt management, and prompt evaluation. It engages with OpenTelemetry natively on the ingestion side. It accepts OTLP spans directly, so that spans instrumented with OpenTelemetry flow in without a separate SDK. This is a pragmatic choice when we want a graphical interface for LLM interactions, while staying on OpenTelemetry standards.
• Arize Phoenix is designed for evaluation and quality monitoring, including drift detection, hallucination detection, and embedding analysis. It is built directly on OpenTelemetry conventions via OpenInference, a set of OTel semantic conventions and auto-instrumentation packages for LLM applications. Therefore, its traces are standard OTLP. It fits when the goal is monitoring interaction quality rather than operational observability.
• General-purpose OTel backends (Grafana Tempo, Jaeger, Honeycomb, Datadog, New Relic) consume OTLP like any other trace, with no LLM-specific UI. Agent traces sit alongside everything else traversing the infrastructure, such as databases, queues, and HTTP services.

The table below summarizes all of these aspects.

Concern	LLM-specific platform (LangSmith / Langfuse / Phoenix)	General-purpose OTel Backend
Setup effort	Low: purpose-built SDKs and UIs	Higher: instrument manually and design the span model
LLM-aware views	Prompt diffs, eval runs, and token dashboards built in	None by default, we build them
Lock-in risk	Varies: stronger platform coupling for LangSmith; lower coupling when exporting standard OTLP traces.	Low: vendor-neutral by design
Full-stack correlation	Limited to the AI layer	Agents and all the infrastructure in one trace

The underlying principle remains the same in every case. Choose the preferred platform, but ensure instrumentation follows open standards.

If the spans that make up the trace follow conventional GenAI semantics and export metrics via OTLP, we can switch backends, change LLMs, or add specific tools to ensure observability without rewriting the core instrumentation model. The shared vocabulary is what makes this work. Once spans follow the GenAI conventions, any OTLP backend reads them identically.

Conclusion

A multi-agent system has many of the same observability problems as a distributed system, with the added variability of model outputs, tool behavior, and dynamic routing. These systems operate according to fan-out strategies that are unknown in advance and make decisions based on the outputs they receive. All of this makes it practically impossible to rely solely on a log for observability, no matter how well-formatted or well-written it is.

Therefore, the goal is to address this problem by collecting traces and metrics and storing them in tools that use a universal vocabulary.

Whether we choose an open-source backend or a complete SaaS platform, the principle to apply remains the same. Capture all relevant traces and maintain an open, interoperable vocabulary and standard.

The accompanying Java sample shows the trace setup, custom orchestration spans, and exported Jaeger trace.