The LLM observability market hit $2.69 billion in 2026, yet Gartner reports only 15% of GenAI deployments were instrumented in early 2026, with most still blind to agent behavior. OpenTelemetry for AI agents promises a vendor-neutral escape from that gap, but the reality is more complicated than the marketing slides suggest. You'll find that tracing is largely solved, the vocabulary is still shifting, and the premium you pay for managed platforms buys evaluation workflows—not visibility—that still don't close the quality gap.
Why is OpenTelemetry becoming the default language for agent telemetry?
OpenTelemetry gives you two things that proprietary SDKs never could: a standard schema and a standard transport. The GenAI semantic conventions define common attribute names for model requests, agent steps, tool calls, and token usage, so a span from LangChain and a span from CrewAI describe the same operation in the same words. That portability matters more for agents than for traditional services, because agent stacks mix frameworks, runtimes, and cloud providers in ways that make vendor lock-in expensive.
The transport layer is OTLP—OpenTelemetry Protocol—exporting over HTTP or gRPC to any compatible collector or backend. You don't need a bespoke ingest pipeline per tool. Per AgentPing's analysis, this is genuinely useful: a trace of a run, with each model call and tool call as a nested span and token counts attached, is most of the raw material you need to understand what happened.
But "we emit OpenTelemetry" and "we can operate our agents" are different sentences. The conventions cover agent, workflow, tool, and model spans, plus required latency and token-usage metrics, according to the v1.41 spec documentation. Nearly all of those gen_ai.* attributes still carry Development stability badges, which means the vocabulary is standardized in principle but not frozen in practice.
Framework reach is already broad. LangChain, CrewAI, Strands, Microsoft Agent Framework, and the OpenAI Agents SDK emit OpenTelemetry GenAI spans natively or via instrumentation packages, exporting over OTLP per AgentPing's framework survey. That eliminates the need for a separate proprietary SDK for each framework—at least for the span tree.
If you're comparing full platform stacks, our guide to AI Agent Monitoring Tools Compared breaks down where each vendor sits on this portability axis.
What does the GenAI span tree actually capture?
A single agent run produces a tree, not one flat span. At the top sits an invoke_agent span for the full turn. Underneath it nest chat spans for every model call, execute_tool spans for every tool invocation, and subagent spans when one agent hands off to another. The span tree structure has held stable across framework releases: parent agent, child model calls, child tool calls, nested subagents.
That stability is the real product. Whether you're running LangGraph, the OpenAI Agents SDK, or Strands, the shape of the trace looks the same in your backend. You can replay the full execution path—planning, retrieval, reasoning, synthesis—without rewriting your dashboard queries for each framework.
The OpenTelemetry Demo 3.0 release from July 28, 2026 reinforces this by modeling agentic AI reference stacks with exactly this nested structure. The demo intentionally broke legacy app.* attributes and moved to demo.* to align with current conventions, showing that the span shape is the durable part while the labels keep changing.
For production debugging, that tree answers the critical question: why did the agent call get_weather three times instead of once? You can see the loop in the nested spans, measure latency per step, and attribute token cost to each model call. Without that granularity, you're grepping logs and guessing.
How unstable is the GenAI schema in practice?
Here's the honest answer: every gen_ai.* attribute remains in Development status, and the dedicated repository has no versioned release or finalized schema URL. As of July 17, 2026, John Hodge's assessment confirms that no GenAI-specific span, event, metric, or attribute in the open-telemetry/semantic-conventions-genai repository is marked Stable.
The rename timeline is relentless. gen_ai.usage.prompt_tokens and completion_tokens became input_tokens and output_tokens in v1.27. gen_ai.system became gen_ai.provider.name in v1.37, with per-message events replaced by structured message attributes. v1.38 added evaluation events. v1.40 brought retrieval spans and cache token fields. v1.41 split invoke_agent into client and internal spans, added reasoning-token fields, and introduced streaming latency metrics. The attribute rename history spans six releases with no sign of slowing.
Then v1.42.0 deprecated all GenAI content from the main repository, and v1.43.0 ships none. The conventions now live in their own repo with no tags. If you instrument today, you're pinning to a moving commit, not a versioned standard.
That instability has a practical cost. Your dashboards, cost-attribution queries, and evaluation pipelines all reference attribute names that may change in the next framework update. "Vendor-neutral" instrumentation today requires constant migration logic tomorrow. The portability promise is real for the transport layer; it's fragile for the vocabulary layer.
What does OpenTelemetry deliberately leave out?
OpenTelemetry is a transport and a schema, not an operations platform. Per AgentPing's breakdown, it deliberately excludes cost calculation, missing-agent detection, and output-quality evaluation. Those gaps are where the actual work lives.
The data shows this clearly. A 2025 LangChain survey found 89% of agent teams already run observability tools, yet one in three still cite quality as their top blocker to production. None of the twelve compared platforms deliver a real-time per-turn verdict under 90ms. Tracing records what the agent did; it doesn't judge whether the turn was correct, safe, or off-task.
That distinction defines what I call the Trace-Rich, Verdict-Poor pattern. Every platform captures nested agent runs well—eight of twelve have open-source cores built on OpenTelemetry—so the premium you pay for managed SaaS is not for visibility. It's for evaluation workflows, dataset management, and prompt versioning that still fail to produce the sub-90ms verdict layer every production team actually needs.
Cost attribution is another blind spot. A single agent turn with three model calls and three tool calls generates roughly seven billable spans on per-span billing platforms like Datadog, Arize, or Sentry, versus one request on proxy-based tools like Helicone. Per MorphLLM's pricing analysis, the sticker price and the real bill diverge sharply. OpenTelemetry gives you token counts on spans; it doesn't calculate your monthly invoice or flag which feature doubled spend.
If you're tracking hidden costs across platforms, our post on Tracing AI Agents: The Real Cost Is Not in the Trace covers the billing-unit fragmentation in detail.
How do the major platforms compare on cost and portability?
The market is consolidating fast. ClickHouse acquired Langfuse in January 2026. Mintlify acquired Helicone in March and moved it to maintenance mode. Cisco is acquiring Galileo. Per Respan's market analysis, choosing a platform in 2026 means choosing its roadmap, and several of those roadmaps just changed hands.
Here's how the key tools line up on deployment model, pricing, and portability:
| Tool | Deployment | Pricing (Public) | OTel Support | Best For |
|---|---|---|---|---|
| Langfuse | Self-host (MIT) / Cloud | Free OSS; cloud 50k units/mo free, then $29+/mo | Native v3 SDK, OTLP ingestion | Data residency, open-source control |
| LangSmith | Managed cloud / Enterprise K8s | Free 5k base traces/mo; Plus $39/seat + $2.50/1k over 10k; ~$2,514/mo at 1M | Full two-way OTel | LangChain/LangGraph shops needing evals |
| Helicone | Proxy / Self-host | Free tier + usage-based; maintenance mode post-acquisition | Proxy-first, not core ingestion | Fast LLM cost tracking, minimal code change |
| Arize Phoenix | Self-host OSS / Managed | Phoenix OSS free; managed/enterprise for AX | OpenInference-first, OTel-compatible | Evals + tracing in one pipeline |
| Cloudflare Agents | Managed (Workers) | Free during beta; priced under Workers Observability from Oct 1, 2026 | Native agent spans | Edge-deployed agents on Workers |
Self-hosted open-source cores—Langfuse MIT, Arize Phoenix OSS—support data residency and OTel-native ingestion. Managed SaaS platforms—LangSmith, Braintrust—integrate evaluation workflows but rely on proprietary SDKs or cloud-only deployment. The tradeoff is clear: portability versus integrated feature depth.
For a deeper comparison of evaluation depth versus monitoring breadth, see Prompt Tracing: Cost, Portability, Governance Shift.
Should you instrument with native SDKs or proxy gateways?
Your instrumentation choice determines your switching cost. OTel-native SDKs mean your code exports standard spans over OTLP with gen_ai.* semantic conventions. When you migrate backends, you re-tag spans and rebuild dashboards, but you don't rewrite capture logic. Proprietary SDKs—LangSmith's primary path, for example—tie you to vendor-specific attributes and ingestion formats.
Proxy gateways like Helicone sit between your application and the model API. They capture tokens, latency, and cost per request with almost no code change, but they never see the reasoning between calls or the nested agent structure. Per MorphLLM's comparison, a proxy sees one request; an SDK sees seven spans for the same turn. If you need to debug why a subagent looped, the proxy won't help.
Framework coverage varies. First-class support for LangGraph, CrewAI, the OpenAI Agents SDK, AutoGen, Pydantic AI, or the Vercel AI SDK saves weeks of instrumentation. OpenTelemetry-native tools accept spans from any framework that emits them, but deeper feature integration—prompt versioning, dataset management—often requires vendor-specific extensions.
Amazon's approach illustrates the deployment-model split. Bedrock AgentCore Observability natively traces only agents running on the AgentCore runtime in AWS. For EKS, ECS, Lambda, on-premises, GCP, or Azure deployments, you must configure AWS Distro for OpenTelemetry (ADOT) with OTLP export and IAM authentication, per AWS's multi-cloud guide. The native path is simpler; the portable path requires more setup.
Arize AX takes a normalization approach: it maps incoming gen_ai.* spans into structured OpenInference fields—operation type, messages, token counts, provider/model metadata, tool inputs/outputs—so evaluations and cost analysis work regardless of which framework emitted the telemetry. Per Arize's announcement, this removes the need for custom processors in client code.
What's the practical path forward?
Stop comparing tracing dashboards. The data is clear: 89% of teams already have traces, one-third still can't ship due to quality blind spots, and not one of the twelve compared platforms closes that gap. The premium for managed SaaS buys evaluation workflows and faster setup, not visibility. Given the consolidation risk—Helicone in maintenance mode, Langfuse under ClickHouse, Galileo under Cisco—betting on a proprietary managed roadmap is unstable regardless of schema portability.
My recommendation: self-host an open-source OTel core—Langfuse MIT or Arize Phoenix OSS—to own your data and avoid acquisition-driven roadmap changes. Then redirect budget from SaaS tracing premiums toward custom evaluation pipelines that produce the per-turn verdict layer no vendor provides. Instrument with native SDKs for portable span trees, but treat the gen_ai.* vocabulary as temporary: build your queries with abstraction layers so renames don't break your dashboards.
The MCP specification revision released July 28, 2026 eliminates session handshakes and replaces proprietary logging with OpenTelemetry, making servers stateless HTTP services. Per New Relic's analysis, that aligns the protocol layer with the same portability goals. Your instrumentation should match that direction: stateless, portable, and independent of any single vendor's future.
If you're deciding between build and buy, ask not which dashboard looks best, but which failure mode costs you more: a broken evaluation pipeline you control, or a tracing platform whose roadmap just changed hands. What's your team's tolerance for migration logic every quarter?
Originally published at SaaS with Alex
Top comments (0)