For the last decade, observability has been built around a fairly simple mental model: a request comes in, it moves through a handful of services, and somewhere in that path, something goes wrong.
Traces show you the path. Metrics show you the trend. Logs show you the details. It's worked well enough because the systems we were watching were, fundamentally, deterministic; the same input mostly produced the same path through the same code.
Agentic AI breaks that assumption, and most of our tooling hasn't caught up yet.
The Problem: Agents Don't Behave Like Services
An AI agent doesn't execute a fixed code path. It reasons, calls tools, sometimes calls the same tool multiple times with different arguments, backtracks, retries with a different strategy, and occasionally does something you didn't anticipate at all. Two identical requests to the same agent can produce two completely different execution graphs.
This creates a few very real problems for anyone trying to run these systems in production:
- Traditional tracing assumes a stable shape. A trace waterfall works great when a request flows through auth → service A → database → response. It's much less useful when the "service" in the middle might call itself recursively, spawn three parallel sub-tasks, or make a judgment call that changes the entire downstream path.
- Logs lose their anchor. A log line saying "retrying operation" means something very different when a human wrote the retry logic versus when an LLM decided, based on its own reasoning, that a retry was the right move. You now need to capture why the agent made that call, not just that it did.
- Metrics stop being purely technical. Latency and error rate still matter, but now you also care about things like "how many tool calls did it take to complete this task," "did the agent's plan match its final action," or "how often does it need human intervention." These are semi-qualitative signals that don't fit cleanly into a time-series database built for CPU and memory graphs.
What Actually Needs to Change
None of this means the three pillars: logs, metrics, and traces, are obsolete. It means they need a fourth layer sitting on top: reasoning context.
A few patterns are starting to emerge across teams building and running agentic systems:
1. Treat the agent's decision trace as a first-class signal.
Instead of just logging the final action an agent took, capture the intermediate reasoning steps, the plan it formed, the tools it considered, and why it picked one path over another. This doesn't need to be a full chain-of-thought dump (which is expensive and sometimes unreliable), but a structured record of "decision points" makes debugging dramatically easier when an agent goes off the rails.
2. Correlate tool calls the same way you'd correlate microservice calls.
If your agent calls a search API, then a database, then a code execution sandbox, that's functionally a distributed trace; treat it like one. Propagate a trace ID through every tool call so you can reconstruct the full path after the fact, exactly the way you would for a request bouncing across five microservices.
3. Track task-level outcomes, not just request-level ones.
An agent might make twelve individual tool calls to complete one task. Individually, every call could return 200 OK. But if the task itself failed, the wrong file got modified, the wrong ticket got closed, none of your existing dashboards will show that. You need an outcome signal that sits above the level of individual calls.
4. Budget for non-determinism in your alerting.
If you alert on "error rate > 1%" the way you would for a REST API, you're going to get paged constantly, because agents fail differently, a wrong answer isn't a 500, it's a semantic failure your monitoring stack was never built to detect. Some teams are experimenting with using a second, smaller model purely to evaluate whether an agent's output looks reasonable, essentially building an automated eval into the observability pipeline itself.
The Uncomfortable Part
Here's the part that's genuinely hard and worth being honest about: a lot of "agent observability" right now is still catching up to the problem. Tracing standards for agentic workflows are still being figured out industry-wide, and plenty of teams are hand-rolling their own logging conventions because there isn't yet a settled standard the way OpenTelemetry became one for distributed tracing.
If you're building or operating agentic systems today, you're probably going to end up designing at least part of your own observability layer, because the tooling ecosystem hasn't fully caught up to the problem yet. That's not a fun answer, but it's an honest one, and it's also, frankly, a pretty interesting problem to be working on right now.
Are you running agentic systems in production? Curious what you've built to get visibility into them? Drop your approach in the comments.
Top comments (0)