DEV Community

Cover image for A simple typo swallowed our telemetry: Why our AI agent spans vanished into the void
S M Tahosin
S M Tahosin Subscriber

Posted on

A simple typo swallowed our telemetry: Why our AI agent spans vanished into the void

Summer Bug Smash: Clear the Lineup 🐛🛹

This is a submission for DEV's Summer Bug Smash: Clear the Lineup.

Project Overview

In the modern era of autonomous AI agents, observability is everything. If an agent hallucinates or takes an unexpected path, you need to know exactly which model was invoked, what the system prompt was, and how long it took.

The Sentry Python SDK provides a fantastic integration (OpenAIAgentsIntegration) specifically designed to trace the internal execution of the openai Swarm/Agents framework. It hooks into the framework's core loops to automatically generate beautiful, hierarchical tracing spans for every tool execution and model invocation.

Bug Fix or Performance Improvement

While reviewing the integration for version 0.8.0 of the openai-agents SDK, something felt off. The tool execution spans were working perfectly. However, the model invocation spans, which are the most critical pieces of the telemetry, were completely missing.

The Sentry dashboard was virtually blind to when the agent actually reached out to the LLM.

My Improvements

I started digging into the SDK's source code at sentry_sdk/integrations/openai_agents/__init__.py.

In the 0.8.0 release of the openai-agents framework, the internal architecture had changed. The method responsible for resolving models was refactored and moved from AgentRunner._get_model() to a new location: agents.run_internal.turn_preparation.get_model().

The Sentry maintainers were aware of this change. They wrote the exact code needed to wrap the new turn_preparation.get_model function:

@wraps(turn_preparation.get_model)
def new_wrapped_get_model(
    agent: "agents.Agent", run_config: "agents.RunConfig"
) -> "agents.Model":
    return _get_model(turn_preparation.get_model, agent, run_config)
Enter fullscreen mode Exit fullscreen mode

The wrap was perfectly executed. The telemetry injection was flawless. So why wasn't it working?

I looked at the very next line of code, where the wrapped function is injected back into the SDK:

agents.run_internal.run_loop.get_model = new_wrapped_get_model
Enter fullscreen mode Exit fullscreen mode

There it was. A simple, silent typo.

They correctly wrapped turn_preparation.get_model, but they assigned the result to run_loop.get_model. Because the SDK internally uses turn_preparation to resolve the model, the patched function sitting inside run_loop was completely ignored.

The patch never installed. The model invocation spans vanished into the void.

Code

GitHub PR: getsentry/sentry-python#7227

The fix was as simple as correcting the assignment target. I submitted a patch to ensure the wrapped function overwrites the actual method the SDK uses:

- agents.run_internal.run_loop.get_model = new_wrapped_get_model
+ agents.run_internal.turn_preparation.get_model = new_wrapped_get_model
Enter fullscreen mode Exit fullscreen mode

By changing run_loop to turn_preparation, the SDK now correctly intercepts the model resolution step.

Sentry Traces Restored
Accurate telemetry restored!

In software engineering, some of the most critical bugs that entirely disable core features often come down to a single line of misdirected code.

Top comments (0)