Installing OpenTelemetry Won't Close Your AI Agent's Feedback Loop

I recently did a lightweight review of how OpenTelemetry (OTel) is being used across three repositories under development at my company:

• An AI agent application built primarily with Node.js and AWS Lambda
• An Electron-based desktop application
• A TypeScript SDK and compiler for developers

All three are works in progress. This article is a snapshot of the development environments and codebases at the time of the review, not an assessment of finished products. I used Claude Code to inspect dependencies, configuration, and selected execution paths, so the depth of the review also varied by repository.

I focused on three questions:

Are we actually using the OpenTelemetry mechanisms that exist in the codebase?
What makes adoption or operation difficult?
Can the resulting telemetry help Claude Code or Codex implement and verify changes?

The background: I have been experimenting with ways to make code, documentation, and runtime behavior easier for coding agents to understand, to get more reliable results from tools such as Claude Code and Codex. I understand OTel's value as a way to give telemetry from different services a shared meaning and a shared causal structure. What I have not yet developed is a confident sense of when to introduce it and how to wire it into the everyday development loop.

My conclusion up front is simple:

Installing OTel is not the goal. Telemetry becomes part of an AI-assisted development loop only when it is actually generated, accessible to the agent, correlated as one execution, and comparable before and after a change.

Along the way, I will propose six conditions for agent-usable observability — standardization, propagation, discoverability, controllability, accessibility, and comparability — that I now find more useful than asking "do we have OTel?"

"It exists," "it emits," and "the agent can use it" are different states

The three repositories were in very different states:

Repository	State at the time of review
AI agent application	No direct OTel usage; relies on structured logs, metrics, X-Ray, and audit records
Desktop application	A Collector and related environment variables exist, but the application emits little or no telemetry to them
SDK and compiler	OTel integration is implemented as an opt-in feature, but internal usage is limited

The most useful distinction was between three separate states:

1. An observability mechanism exists
2. It actually produces telemetry
3. An AI agent can use that telemetry to make a decision

A running Collector does not help if the application never emits spans. An integration API does not help if nothing calls it.

One repository showed the shape of the problem clearly. A tracing middleware existed — typed, exported, and documented — but the application's middleware pipeline never included it. Everything about the module looked "done," yet the system produced zero trace data. A coding agent grepping the repo would find a plausible-looking observability layer, and nothing to read at runtime.

This was not specific to OTel. A custom trace system in another repository had the same gap: trace types and debugging features existed, but the middleware was not connected to the runtime.

The underlying failure mode: we treat the implementation of an observability mechanism as completion, without verifying that it produces evidence and that someone — or something — actually consumes it.

OpenTelemetry's core value

OpenTelemetry is a vendor-neutral framework for generating, collecting, and exporting signals such as traces, metrics, and logs. The OpenTelemetry Collector provides a common pipeline for receiving, processing, and forwarding telemetry to one or more backends.

From the perspective of a coding agent, two parts are especially valuable.

Shared meaning

Semantic Conventions standardize not only attribute names, but also their types, meanings, and units across common domains such as HTTP, databases, RPC, and messaging. When the same concept is represented by the same attribute across repositories, Claude Code or Codex has less project-specific naming to infer.

Shared causality

Context propagation carries trace context across process and network boundaries so that distributed operations can be connected as one causal chain. Individual logs may be searchable, but if an API request, an async job, a worker execution, and an audit event cannot be associated with the same operation, it is hard to establish what actually caused a failure.

To me, the core value of OTel is not that it creates more telemetry. It gives telemetry from different execution environments a shared vocabulary and a common correlation key.

An agent can read a system without OTel

The first repository did not use OTel at all — yet it was the easiest for the agent to inspect.

It had structured logs, cloud metrics, X-Ray traces, audit records, and a data pipeline that allowed operational data to be re-queried with SQL. Some of the available datasets and analysis procedures were described as repository-level contracts.

In that kind of system, a coding agent can inspect runtime behavior just fine.

So in the repositories I reviewed, OTel was neither a necessary nor a sufficient condition for AI-readable observability. The real requirement was that telemetry be structured, actually generated, reachable by the agent, and joinable as one execution.

AI readability depends on where the data appears

The review also revealed a much more basic requirement:

Can Claude Code or Codex reach the data with the tools already available in the session — Read, Grep, and a shell?

In one repository, a lightweight trace written into the workspace as JSON and Markdown was the easiest runtime artifact for the agent to use. A richer trace existed in cloud storage, but reaching it required authentication, environment setup, network access, and waiting. That made it far less likely to be used in a loop measured in seconds.

For the development loops I reviewed, accessibility looked roughly like this:

Local files and stdout
  > Read-only MCP, CLI, or SQL access
  > An authenticated cloud UI

This is not a universal ranking — cloud telemetry is essential for production incidents. But when an agent edits and validates in a tight loop, immediate local access wins.

Fortunately, OTel makes the local path cheap. OTel SDKs commonly provide development exporters that write telemetry to the console or stdout — the Python SDK, for example. The Collector's debug exporter can print telemetry locally, while the Collector Contrib distribution also provides a file exporter. Third-party terminal viewers such as otel-tui let you inspect OTLP locally without any backend. When introducing OTel, wiring up a dev-mode path to stdout or local files first — before any remote backend — gives the agent something to read on day one.

Six conditions for agent-usable observability

This is the framework I now use instead of "is OTel installed?":

Condition	Question
Standardization	Are attribute meanings, types, and units defined?
Propagation	Can the same execution be followed across HTTP and async boundaries?
Discoverability	Can the available attributes and datasets be enumerated programmatically?
Controllability	Can time ranges, result limits, and permissions be safely constrained?
Accessibility	Can the agent read the data within its implementation loop?
Comparability	Can the state before and after a change be evaluated under the same conditions?

Different layers of the stack serve different conditions:

• OTel is particularly strong at standardization and propagation
• Collectors and backends support processing, storage, and controllability
• MCP servers, CLIs, SQL interfaces, and local files support discoverability and accessibility
• Queries, tests, and analytical models that evaluate changes support comparability

No single layer closes the development loop by itself.

OTel does not define domain meaning

Semantic Conventions primarily standardize infrastructure concerns. They do not fully express domain-specific events from an AI agent system, such as:

• A guard rejected an operation
• A hazardous action required approval
• The system returned a correctable error
• A particular next action became available
• A decision was based on a specific piece of evidence

Adopting OTel does not remove the need for custom attributes and repository-level schema contracts. Work is continuing on GenAI-related conventions in the Semantic Conventions repository, including conventions for GenAI clients and MCP — but that still does not mean every agent- or workflow-specific concept can or should be standardized.

A native execution trace may also carry richer information than an OTel span: correction codes, permitted next actions, references to evidence, state transitions, replay information. In such systems, OTel should not necessarily replace the native trace. A more natural separation:

• Keep domain-specific execution data in the native trace
• Project shared operational meaning and cross-system causality into OTel

The OTel representation may be a deliberately lossy projection of the richer native trace.

Do not put internal IDs directly into traceparent

The SDK and compiler repository exposed a concrete integration problem.

Suppose the system uses human-readable internal identifiers such as:

trace_0001
run_abc

Those values cannot be placed directly into the Trace ID or Parent ID fields of traceparent. Under the W3C Trace Context specification, the Trace ID must be 32 lowercase hex characters and the Parent ID 16, and all-zero values are invalid. A naive conversion produces something like:

00-0000000000000000000000trace_0001-run_abc000000000-01

A compliant receiver must ignore the invalid traceparent, so the downstream operation may start a new trace or remain uncorrelated — propagation silently fails.

The cleaner design:

• Let the tracer generate valid OTel Trace IDs and Span IDs
• Preserve the internal execution ID as a span attribute
• Make the two identifiers joinable at query time

And note: the existence of a propagation helper does not prove propagation works. Test it across a real service boundary and confirm the resulting spans land in the same trace.

OTel and audit evidence serve different purposes

Introducing OTel does not mean replacing existing logs, audit records, or native traces:

Data	Primary purpose
Spans	Latency, errors, and distributed causal analysis
Metrics	Aggregation, trends, and alerts
Logs	Detailed event and exception search
Audit records	Accountability, critical actions, and tamper evidence
Native traces	State transitions, corrections, and deterministic replay

Audit records that require tamper detection or deterministic replay should not be moved into sampled OTel spans. OTel is a common observability layer, not an audit ledger.

MCP can become the agent-facing query layer

Standardized telemetry is not enough; the agent also needs a safe interface for querying it.

At the time of writing, Grafana provides an official MCP server and documents connecting Claude Code and Codex CLI to observability data. Sentry provides an official MCP integration for exposing issue and error data to coding agents.

A useful division of responsibility:

• OTel provides shared meaning and causality
• MCP, CLI, and SQL interfaces provide the agent-facing query surface

But a query interface is not automatically safe or useful. It still needs read-only access, controllable time ranges and result limits, discoverable attributes, repeatable queries, and before-and-after comparison — the six conditions again, viewed from the interface side.

The coding agent is also a telemetry source

One more angle worth noting for readers using Claude Code: the agent is not only a reader of telemetry.

Claude Code supports OTel export of its own metrics and events — sessions, token usage, tool executions — enabled via environment variables such as CLAUDE_CODE_ENABLE_TELEMETRY=1 and OTEL_METRICS_EXPORTER / OTEL_LOGS_EXPORTER (with console available as an exporter for local debugging). Events emitted while processing a single user prompt share a prompt.id attribute, so a prompt's API calls and tool executions can be correlated in the backend. See the official monitoring documentation for details.

This closes an interesting circle: the same standardization and propagation that make your system readable to the agent can also make the agent's behavior readable to your team.

The hard part is often the data boundary, not instrumentation

When introducing OTel, configuring an SDK or Collector may not be the hardest part. The harder question is what should be recorded and where it should be sent.

URLs, SQL statements, exceptions, prompts, tool arguments, and user data can unintentionally appear in telemetry. The OpenTelemetry guidance on handling sensitive data recommends avoiding collection of sensitive information where possible, and describes removing or transforming data with Collector processors (attribute, filter, redaction, transform).

However, moving all redaction responsibility into the Collector can make critical policy invisible from the application repository. Collector configuration should be versioned, reviewed, and tested alongside the application — or the application should at least maintain a clear contract describing which attributes may be collected.

OTLP export is also a new egress path. In environments with restricted networking, connectivity to a Collector or remote backend becomes part of the architecture and security design. Adding observability can mean adding a communication boundary.

Cloud telemetry introduces a time gap

When an agent uses cloud telemetry to validate a change, there is an unavoidable sequence:

Change
→ Build
→ Deploy
→ Execute
→ Export telemetry
→ Ingest into the backend
→ Query

Depending on the architecture, stable query results may take tens of seconds to several minutes. A local loop measured in seconds and a cloud loop measured in minutes are different feedback cycles, and a practical setup combines both:

• Deterministic local tests
• Local trace files
• Smoke tests in a development environment
• Before-and-after comparisons using cloud telemetry

Requiring the agent to deploy and wait for ingestion after every change makes the everyday loop unnecessarily heavy.

The order I would use today

Based on this review, I would approach agent-readable observability in this order:

1. Connect instrumentation to a real execution path so telemetry is actually generated
2. Make development output readable through local files or stdout
3. Propagate trace context across HTTP, queues, and workers
4. Keep business and execution IDs as separate attributes, not inside standard trace IDs
5. Define Semantic Conventions and custom attributes as repository-level contracts
6. Establish policies for sensitive data, cardinality, and sampling
7. Provide read-only access through MCP, CLI, or SQL
8. Make before-and-after comparisons repeatable under the same conditions

I would not begin with comprehensive auto-instrumentation. It is easier to evaluate OTel by picking one important path and completing the entire chain:

Generate → Propagate → Store → Query → Compare

An AI agent does not need to stare at a large number of spans. It needs to determine what changed because of its own modification, and verify that conclusion again under the same conditions.

OTel is a common language, not the finished system

The three repositories in this review are still under development, and their implementations will change. Even so, one distinction became much clearer: having OTel, generating telemetry, and making that telemetry usable by an agent are three different stages — and only the last one closes the loop.

Turning telemetry into part of an AI-assisted development loop requires more than instrumentation: it also requires access paths, domain schemas, data boundaries, and repeatable before-and-after evaluation. The right starting point is therefore not comprehensive instrumentation, but one minimal path where telemetry is reliably generated, accessible to the agent, and comparable across a change.

OTel is not the finished form of observability for coding agents.

It is one of the strongest common languages available for building telemetry that is structured, correlated, and reproducible enough for them to use.