DEV Community: Kasper Borg Nissen

Observing Goose on Ollama with OpenTelemetry

Kasper Borg Nissen — Thu, 27 Aug 2026 06:22:00 +0000

Disclosure: I work at Dash0, which is why the screenshots are from there. Nothing in this post depends on it. goose speaks OTLP, so point the exporter at whatever you prefer, whether that is Jaeger, a local Collector writing to a file, or another vendor. Every number below came out of a plain OpenTelemetry Collector before it reached any UI.

Coding agents are easy to run and hard to evaluate. A session either moved your work forward or burned tokens going in circles, and once you are past a handful of runs, those two look identical. You have a bill, a pile of sessions, and no way to sort one from the other.

Answering that starts with a more basic question: what does a coding agent actually report about itself? So I put one under a microscope. goose driven by open-weight models on Ollama is a good subject, because the entire loop runs on your own machine. Nothing is proxied, summarized, or hidden behind a vendor dashboard, so every span, metric, and log record is there to read.

Getting at them is genuinely easy. goose speaks OpenTelemetry natively, so it takes three environment variables and no code changes. What the telemetry contains is more interesting, let's dive into it.

Why goose and Ollama

goose is an open source coding agent now hosted under the Agentic AI Foundation. It runs in your terminal, calls tools, edits files, and speaks MCP. Two things make it a good subject: it is provider agnostic, so pointing it at a local model is a config change rather than a fork, and it already links the OpenTelemetry SDK, so no plugin is needed to get telemetry out.

Ollama runs open-weight models on your own hardware behind an OpenAI-compatible API on localhost:11434. goose treats it like any other provider.

Running the pair locally means no API keys, no per-token bill, and nothing leaving your machine, which matters more than usual once you see what the agent puts in its telemetry. The cost is quality. A 30B model on a laptop is not a frontier model, and the gap shows up exactly where it hurts agents most, in reliable tool calling.

Setting up goose with Ollama

1. Install goose

On macOS:

brew install block-goose-cli

Watch the formula name. The goose formula in homebrew-core is an unrelated Go database migration tool, and the two conflict because both install a goose binary.

goose --version should report 1.46.0, which is what everything below is measured against.

2. Install Ollama and pull a model

Install Ollama from ollama.com, then pull a model. The hard requirement is native tool calling. A model that cannot emit tool calls will flail no matter how good its prose is.

ollama pull qwen3.6:35b-a3b-q4_K_M

Two that work well:

Model	Size	Context	Notes
`qwen3.6:35b-a3b-q4_K_M`	23 GB	256K	Mixture of experts, roughly 3B active parameters, so it is quick on Apple Silicon despite its size
`devstral-small-2:24b`	15 GB	384K	Built for tool-driven codebase work, lighter on memory

ollama list confirms what you have locally.

**Keep Ollama on the host.* On a Mac, Docker Desktop cannot pass the GPU through to a container, so a containerized Ollama falls back to CPU inference and gets several times slower. Every number in this post comes from Ollama running natively.*

3. Connect goose to Ollama

Three environment variables, no config file editing:

export GOOSE_PROVIDER=ollama
export GOOSE_MODEL=qwen3.6:35b-a3b-q4_K_M
export OLLAMA_HOST=localhost:11434

If you would rather persist it, goose configure writes the same settings to ~/.config/goose/config.yaml.

4. Check that it works

goose run -t "Reply with exactly the word: PONG. Nothing else."

    __( O)>  ● new session · ollama qwen3.6:35b-a3b-q4_K_M
   ____)    20260810_1 · /tmp/demo
     L L     goose is ready
PONG

The header line confirms the provider and model. Get the model name wrong and goose says so plainly, listing what is actually available, which is a nicer failure than most.

Now give it real work:

goose run -t "Create a Node.js Express app in app.js exposing GET /rolldice that returns a random integer from 1 to 6 as plain text. Also create a package.json with express as a dependency. Do not run npm install."

 ────────────────────────────────────────
  ▸ write
    path ~/dash0/demos/dash0-examples/goose/app.js
    content: const express = require("express");
const app = express();
const PORT = proce...

    Created ../goose/package.json (12 lines)
    Created ../goose/app.js (12 lines)
Done. Here's what I created:

### `package.json`
- Defines the `rolldice` app with a `"start"` script (`node app.js`)
- Lists `express: ^4.21.0` as a dependency (no `npm install` run)

### `app.js`
- Creates an Express server on port 3000 (or `$PORT`)
- Exposes **GET /rolldice** which returns a random integer from **1 to 6** as plain text

That is a local coding agent doing local work, with no account anywhere.

Why bother observing it?

Watching one session in a terminal, you barely need telemetry. You can see the agent working and read the files it produced.

That stops being true almost immediately. Agents run headless, in CI, on other people's machines, and across a team. Coding agents also move on three fronts at once: the agent itself, the harness and configuration around it, and the underlying model. Change any one and both your spend and your output shift. Without a common set of signals, you cannot tell which change did what.

Instrumenting the agent gives you the raw material for those comparisons:

Where does the time go? In the run below, 15.5 of 28.5 seconds sat inside two inference calls. For a local agent, "why is this slow" is nearly always the model, and now you can prove it.
What does a task cost? 8,694 tokens for two files. That is the number you compare models on.
Which tools does it reach for, and how often do they fail? Rarely the ones you assume.
How do two models compare on the same job? This turns "are open models good enough yet" from a debate into a measurement.

The one thing this list does not give you is whether the work was any good. Hold that thought.

Why OpenTelemetry, and how to turn it on

The case for OpenTelemetry here is that goose already speaks it. It links the OpenTelemetry Rust SDK and reads the standard OTEL_* variables at startup, initializing traces, metrics, and logs if it finds an endpoint. No plugin, no wrapper, no code changes.

export OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318
export OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf
export OTEL_SERVICE_NAME=goose

That is the whole integration.

To see what comes out, run a collector with a debug exporter. This config also writes each signal to disk, which is how every table in this post was produced:

receivers:
  otlp:
    protocols:
      http:
        endpoint: 0.0.0.0:4318

exporters:
  debug:
    verbosity: detailed
  file/traces:
    path: /out/traces.jsonl
  file/metrics:
    path: /out/metrics.jsonl
  file/logs:
    path: /out/logs.jsonl

service:
  pipelines:
    traces:  { receivers: [otlp], exporters: [debug, file/traces] }
    metrics: { receivers: [otlp], exporters: [debug, file/metrics] }
    logs:    { receivers: [otlp], exporters: [debug, file/logs] }

touch out/traces.jsonl out/metrics.jsonl out/logs.jsonl

docker run -d --name goose-collector -p 4318:4318 \
  -v "$PWD/collector.yaml:/etc/otelcol-contrib/config.yaml" \
  -v "$PWD/out:/out" \
  otel/opentelemetry-collector-contrib:0.158.0

Run goose again and spans start arriving.

NB: goose has no gRPC exporter. It is compiled without the grpc-tonic feature, so the endpoint has to be OTLP over HTTP on 4318, not 4317. Setting OTEL_EXPORTER_OTLP_PROTOCOL=grpc produces no spans, no metrics, no logs, and exit status 0, with nothing on the terminal to say so. Leaving the protocol unset is fine, since the spec default is already http/protobuf.

GOOSE_TELEMETRY_ENABLED is not the setting you want. That flag controls goose's own product analytics, which by its own description collects error types and never collects your conversations, code, or tool arguments. It has nothing to do with OTLP export.

What goose 1.46.0 emits

Here is the rolldice run, captured. 28.5 seconds, 2 tool calls, one whole trace.

The resource attributes are clean and conventional:

service.name            goose
service.namespace       goose
service.version         1.46.0
host.name               Mac.localdomain
user.name               kaspernissen
telemetry.sdk.language  rust
telemetry.sdk.name      opentelemetry
telemetry.sdk.version   0.32.1

Traces

Six spans in a single trace, properly nested, each carrying a conventional operation name:

reply                                 invoke_agent      63.5 ms
  └── reply_stream                    invoke_agent   28,273.5 ms
        ├── stream_response_from_provider   chat      14,272.4 ms
        ├── dispatch_tool_call        execute_tool         1.2 ms
        ├── dispatch_tool_call        execute_tool         0.1 ms
        └── stream_response_from_provider   chat       1,195.8 ms

One trace, properly nested. Because the attributes are conventional, the backend renders a dedicated Gen AI view without anyone writing a query.

All three conventional operation names are there: invoke_agent on the agent spans, execute_tool on the tool calls, and chat on the two inference spans. Those inference spans also carry the model and the provider as first-class attributes, rather than buried in a log line.

The inference call: stream_response_from_provider, tagged gen_ai.operation.name = chat, carrying the provider, the model, and the real per-call token split.

That matters more than tidiness. goose can run different models for different roles through GOOSE_SUBAGENT_MODEL and GOOSE_PLANNER_MODEL, and attributing cost or latency to the model that incurred it needs the model on the span.

Tokens arrive per call, and the input grows as the conversation accumulates:

stream_response_from_provider   7,944 in / 448 out
stream_response_from_provider   8,470 in / 224 out

Tool calls are described through gen_ai.tool.name, gen_ai.tool.call.id, and the session is grouped by gen_ai.conversation.id.

One trap for queries. Both reply and reply_stream are tagged invoke_agent, and both carry the same aggregated token counts, 16,414 in and 672 out. Read the stream_response_from_provider spans instead, the ones tagged chat.

Metrics

The metrics did not move. Seven instruments, every one a monotonic cumulative sum with no unit set, all still custom-named:

goose.session_tokens         8694   message="Session tokens"
goose.session_duration_ms   28474   message="Session duration"
goose.tool_calls                2   tool_name=write
goose.tool_completions          2   tool_name=write, result=success
goose.session_starts            1   interactive=False
goose.session_completions       1   exit_type=normal, duration_ms=28474, total_tokens=8694, message_count=6
goose.cli_commands              1   command=run

Every instrument a SUM, and an entirely empty Unit column.

None of the GenAI metric instruments exist. The conventions specify gen_ai.client.token.usage and gen_ai.client.operation.duration as histograms; goose has sums, so distributions cannot be recovered. There is no p95 inference latency at any price.

Three things break OpenTelemetry's own rules, independent of the GenAI conventions.

The unit is in the name. goose.session_duration_ms should be a duration with a unit of seconds, as the metric naming guidance spells out.
message is a description, not a dimension. Every metric carries one, holding text like "Session tokens". That belongs in the instrument description.
goose.session_completions puts measurements in attributes. duration_ms, total_tokens, and message_count are values, not dimensions. As attributes they have unbounded cardinality, so every session creates its own time series.

There is a fourth thing, and it will cost you money if you trust it. goose.session_tokens is not the session total. It reported 8,694 for this run. The two inference calls actually consumed 7,944 + 448 + 8,470 + 224 = 17,086 tokens. The metric matches the last call exactly, because each call resends the growing conversation, so what it really measures is final context size. Useful for context-budget work, wrong for spend.

Logs, emitted twice

Ten records, all INFO, with bodies like Tool call started and Session completed.

Look at their attributes and you find keys named monotonic_counter.goose.session_tokens and monotonic_counter.goose.tool_calls. goose produces its metrics through the Rust tracing crate's metrics-via-logs convention, and the logs pipeline exports those same events again as log records. Every counter increment is billed twice, once as a data point and once as a log record, for no added information.

None of the ten records carry a trace ID, so there is still no log-to-trace correlation.

Where it still diverges

The attribute story is nearly complete. Four things are not.

Span names are internal. The convention asks for {operation} {target}, so chat qwen3.6:35b-a3b-q4_K_M and execute_tool write. goose keeps stream_response_from_provider and dispatch_tool_call and layers the attributes on top.

Span kind is Internal everywhere, including the stream_response_from_provider spans that wrap an HTTP call to Ollama. Those are Client spans on their own terms, before any GenAI convention applies.

gen_ai.agent.name is absent, and agent-aware tooling keys off it.

The metrics are untouched, as above.

If you are still on 1.45.0, none of this applies, because none of it is there. The GenAI work merged in #10700 and #10816 and shipped in 1.46.0 on 2026-08-12. On 1.45.0 the string gen_ai does not appear in the binary at all, the trace arrives as two unrelated fragments, and the model name is recorded once, inside a log body. Upgrade rather than work around it.

Message content is opt-in, and the gate leaks

1.46.0 puts message content behind an opt-in, using the variable the conventions themselves suggest:

export OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT=true

It defaults to off, and there is a test in the codebase named content_capture_requires_explicit_opt_in. One detail worth knowing: it accepts only the literal string true, compared case-insensitively. 1 silently does nothing, which will catch anyone who reaches for =1 out of habit.

Turn it on and the conversation arrives properly structured, prompt and response side by side:

With capture enabled, gen_ai.input.messages and gen_ai.output.messages populate a readable conversation view.

This is the right shape. Content sits behind a flag, under the conventional attribute names, which is what redaction features and shared collector configs know to look for.

Now run the same task twice, once with the gate at its default and once enabled, and compare what is actually on the wire:

Attribute	Capture off	Capture on
`gen_ai.input.messages`	absent	present
`gen_ai.output.messages`	absent	present
`gen_ai.tool.call.arguments`	absent	present
`gen_ai.tool.call.result`	absent	present
`output`	absent	present
`user_message`	present	present
`trace_input`	present	present
`input`	present	present

The conventional attributes obey the flag exactly. Three legacy attributes ignore it.

With capture off, this still leaves the machine:

[span=reply]              user_message: Create a Node.js Express app in app.js exposing GET /rolldice ...
[span=reply]              trace_input:  Create a Node.js Express app in app.js exposing GET /rolldice ...
[span=dispatch_tool_call] input:        {"tool":"write","arguments":{"path":".../app.js",
                                         "content":"const express = require(\"express\");\n..."}}

The full prompt, twice, and the complete contents of every file the agent writes.

This shipped. goose 1.46.0 advertises opt-in message capture, and with capture at its default the prompt and the file contents go out anyway. It looks like the legacy attributes were kept for backward compatibility without the gate being extended to cover them, which is easy to miss in review. I have reported it upstream.

Until it is fixed, strip them at the collector. Three lines:

processors:
  transform:
    trace_statements:
      - delete_key(span.attributes, "trace_input")
      - delete_key(span.attributes, "user_message")
      - delete_key(span.attributes, "input")

Sending it somewhere permanent

A file of JSONL is fine for investigation, not for keeping. The same variables that pointed at a local collector point at a hosted endpoint. Using Dash0, because that is what I had running:

export OTEL_EXPORTER_OTLP_ENDPOINT="https://ingress.eu-west-1.aws.dash0.com"
export OTEL_EXPORTER_OTLP_PROTOCOL="http/protobuf"
export OTEL_EXPORTER_OTLP_HEADERS="Authorization=Bearer%20${DASH0_AUTH_TOKEN},Dash0-Dataset=goose"
export OTEL_SERVICE_NAME="goose"

The %20 is the percent-encoded space in Bearer <token>. goose decodes it correctly and appends /v1/traces, /v1/metrics, and /v1/logs itself, so leave the path off. Note this is the HTTPS ingress on 443, not the gRPC endpoint.

One trick worth knowing for comparing runs. gen_ai.request.model tells you which model served a call, but it lives on spans, not on metrics, so comparing sessions by model still means tagging them yourself. goose honors OTEL_RESOURCE_ATTRIBUTES on all three signals, so you can tag runs yourself and compare them:

export OTEL_RESOURCE_ATTRIBUTES="demo.model=qwen3.6-35b-a3b,demo.run=1"

Every comparison in this post was produced that way.

goose.session_tokens split by demo_run. Three runs, each separable: the same task with content capture off and on, and the small model that failed.

The run that failed and said it succeeded

Back to where I started.

I ran the identical task on llama3.2:3b, a much smaller model. It asked for two files and produced one, missing its app.listen() call, so the server never starts. The second write degenerated into malformed JSON and unicode garbage, and package.json was never created. Half the task silently dropped.

Here is what goose reported:

goose.tool_calls{tool_name="write"}                         1
goose.tool_completions{tool_name="write", result="success"} 1
goose.session_completions{exit_type="normal"}               1
process exit code                                           0

All five spans carry status UNSET. Not one error, anywhere. And note the trap: the tool call that did happen genuinely succeeded. The file got written. It was just wrong, and the half that failed never registered as an attempt at all.

So result=success means the tool returned without raising, not that it did anything useful. exit_type=normal means the session ended without crashing, not that the task was accomplished. I had assumed tool failure rate would be a decent proxy for whether a model is competent at agentic work. It is not.

You could object that I was sitting right there and could see one broken file. True, and irrelevant the moment nobody is watching: headless runs, CI, a team reading dashboards instead of terminals. There, these are the signals you have.

The only hints were indirect: 5,064 tokens against 8,694 for the same task, 4 messages against 6, 16.0 seconds against 28.5. The model gave up early. That is an anomaly signal, not a correctness signal, and it means nothing without a known-good baseline.

Which points at where this has to end up. Tool-level signals tell you the agent ran. Whether it delivered is a question about outcomes: did the branch merge, did the tests pass, did the change survive review. Those live outside the agent, and joining them to the session is the only way to separate a run that moved work forward from one that burned the same tokens for nothing.

Final thoughts

goose answers real questions for the price of three environment variables, and 1.46.0 brings the GenAI conventions far enough that a backend can read its traces without help. Two things it does not touch are the metrics, which stay custom-named and unit-less, and log-to-trace correlation, which still does not exist.

None of those is the one that matters most, which is telling a successful run from a confidently wrong one. That is not goose's problem to solve, and it is the one worth designing around, because an agent that fails loudly is easy and an agent that fails quietly is expensive.

So instrument the outcome, not just the agent. Start with a task you can verify independently, then check whether the telemetry agrees with reality.

Useful next steps:

The goose documentation and repository
The GenAI semantic conventions, now in their own repository
The OpenTelemetry Collector docs, particularly the debug and file exporters
Observing vLLM with OpenTelemetry, which covers the inference layer underneath an agent

That is the same lesson this experiment kept teaching at small scale. Tokens are easy to count. Whether they delivered anything is the part worth measuring.

In Search of Observability's Rails Moment

Kasper Borg Nissen — Fri, 14 Aug 2026 10:01:24 +0000

Most organizations already have observability. Agents are deployed, dashboards exist, alerts fire, and there is a budget line that grows every year. What they don't have is an observability product: something with defined users, opinionated defaults, documentation, and a lifecycle.

That sounds like semantics. It isn't. Capability describes what exists somewhere in your organization. Product describes how it reaches the people who need it, and what happens when it doesn't fit their case. Observability nearly always exists as a capability. It rarely exists as a product, and that gap explains why so many teams collect more telemetry every year and understand their systems no better.

In 2019, Bryan Liles stood on a KubeCon keynote stage and asked the community to go and find Kubernetes' Rails moment. We did, more or less. It was called platform engineering. Observability is still looking for its own, and I have spent the past year arguing on stage about where to look.

Somewhere in there the argument stopped being mine. The industry gave it a name, ran a survey, and turned it into a position platform teams now claim to hold. The name is shift down. Nearly half say it describes their strategy. Very few have built it. That gap between the strategy and the build is what the rest of this post is actually about.

The decade of shifting left

For about ten years, the answer to any capability gap was to shift it left. Shift testing left. Shift security left. Shift observability left. Catch problems earlier, closer to the person who wrote the code.

The cost landed on developers.

I watched this from inside a bank, where I was a platform engineer while we moved onto Kubernetes. We did what everyone did. We handed developers Dockerfiles, manifests, ingress rules and a wiki page, and called it empowerment. What we had actually done was take a set of genuinely hard infrastructure problems and distribute them across every team in the company, so each one could solve them slightly differently and slightly wrong.

Daniel Bryant has a way of visualizing what that did, and I have been reusing his framing in talks ever since. Plot developer cognitive load against time and you don't get a straight line. Monoliths in the early 2000s kept the boundaries clear: developers wrote code, operations ran infrastructure. Service-oriented architectures pushed the line up, because now the network was the developer's problem too. Then Heroku arrived and the curve dropped hard. git push heroku master, and your application was running on somebody else's problem.

Then containers, microservices and Kubernetes showed up, and the curve went almost vertical.

Developer Cognitive load over time, roughly 2000 to today. Credits to Daniel Bryant, whose talk the visualization is based on.

Kubernetes earned that complexity, and this isn't a complaint about it. Bryan Liles made the point better than I can in his 2019 KubeCon keynote, In Search of the Kubernetes "Rails" Moment. Kubernetes runs on bare metal, in VMs, in other people's clouds, on a Raspberry Pi, across multiple containers and networking and storage runtimes. "Kubernetes complexity is necessary complexity," he said, and people who call it too hard are usually only thinking about their own problem.

His argument was that Rails didn't win by shipping more primitives than anyone else. It won on convention over configuration: good defaults, easy to override, and an opinionated path from empty directory to running application. The line I keep coming back to is his description of the job itself. You are, he said, the principal architect of defaults.

He also said something that has stuck with me ever since: Kubernetes is a vehicle taking us to our destination, not the destination itself. We keep treating it as a product when it isn't one. It's a fancy car.

So the problem was never too much power. Ship primitives without defaults and you haven't removed the assembly work, you've moved it onto the user. That is the gap platform engineering stepped into.

Platform engineering was the correction

Platform teams didn't remove Kubernetes. The raw power stayed exactly where it was. A layer went in between, and that layer absorbed the complexity instead of distributing it. Developers stopped writing manifests and started consuming self-service APIs, templates, golden paths and guardrails. GitOps gave a lot of organizations something close to the Heroku experience again, on infrastructure they controlled.

Evan Bottcher's definition is the one I keep coming back to:

A digital platform is a foundation of self-service APIs, tools, services, knowledge and support arranged as a compelling internal product.

Compelling. Not adequate, not mandated. A developer who could bypass the platform chooses not to, because what they get is better than what they would build themselves. Team Topologies makes the same point from the org side: a platform team exists to provide a compelling internal product that accelerates stream-aligned teams.

So platform teams learned to think in products. They started conducting user research with their own developers. Documentation. Versioned interfaces, feedback loops, adoption treated as a signal rather than a mandate. Backstage gave a lot of organizations a front door. Golden paths became standard vocabulary.

That work is a decade deep for infrastructure and CI/CD. It has barely started for observability.

Observability is repeating the mistake

The pattern rhymes uncomfortably well with early Kubernetes. Powerful primitives, an enormous ecosystem of genuinely good tools, total flexibility, no default experience.

The promise was faster root cause analysis, lower MTTR, real understanding of system behavior including the failure modes nobody predicted. Those outcomes are achievable. I have seen teams get there. But what most engineers actually experience is a different set of questions. Where do I start? Which of these tools is the right one? Why is this metric spiking, and what shipped this component?

Then something breaks. One browser tab for metrics, another for logs, a third for traces. Maybe a fourth for cloud monitoring and a fifth for RUM if the problem might be in the browser. You copy an ID out of one system and paste it into another. You translate between three query languages and juggle between three mental models of what the data even is. You reconstruct the sequence of events by hand.

That's human correlation. We made engineers the integration layer between observability tools, and it doesn't scale. It gets worse as the system grows and worse again at three in the morning.

The same causes come up every time I ask platform teams why. Every signal type ships its own query language, so during an incident you debug syntax alongside the system. Instrumentation was historically vendor-specific, so lock-in starts at the deepest layer and migrating means touching every codebase. Metadata drifts, so one system calls it service.name, another calls it app, and the join quietly fails. And when all that gets expensive enough, teams skip the deeper instrumentation, add a few logs, and call it done.

The easy conclusion is that the tools are bad. They're not. Most are excellent within their scope. We don't have a metrics problem, or a logging problem, or a tracing problem. We have systems problems, and each signal only ever shows one partial view.

That is why I stopped saying three pillars. Three was never the right number, and profiling and real user monitoring are already here. The bigger issue is that the metaphor implies the signals stand apart, when all of the value is in how they connect. A metric with no trace to jump to is a shape on a screen. A trace with no logs underneath is a shape at higher resolution.

The industry has a name for it now

Weave Intelligence, the analyst firm behind PlatformCon, published a market guide on observability trends in platform engineering built on a survey of 105 platform engineers, SREs and operations leaders, fielded in May and June 2026. They state that no vendor commissioned, sponsored or influenced it, and that a sample this size is directional rather than statistically representative. Both caveats are worth carrying.

Asked which statement best describes their observability philosophy for 2026, 47.5% chose shift down: observability as a transparent capability of the platform, with auto-injected OpenTelemetry agents, default dashboards and managed telemetry pipelines. 36.6% chose shift left, where developers instrument their own code. 12.9% are still traditional, with a central reliability team building all the dashboards.

Their definition is the cleanest statement of the idea I have seen written down:

Observability is no longer only a capability that platform teams provide to developers. It has become a capability that platform teams embed into the infrastructure so that developers no longer have to proactively take care of it, but get it by design.

That's the correction platform engineering made for Kubernetes, applied one layer up. Absorb the complexity centrally. Deliver simplicity at the edges. Developers shouldn't assemble telemetry, they should inherit it.

Almost nobody has actually built it

A name is not an achievement.

In the same survey, 18% of teams have automated more than half their observability work, counting things like dashboard and alert setup. 39% say adding new metrics or traces still costs them more than 24 hours of feature-to-production time, and 11% say more than a week. 58% named the instrumentation skill gap between developers and SREs as their biggest cultural blocker. Asked whether their setup gives people relevant, actionable insight, 42.6% said yes. The other 57% split between too much noise and alerts that fire without pointing anywhere useful.

Nearly half of platform teams describe shift down as their strategy. Under a fifth have automated the work that strategy implies. Most still have developers hand-instrumenting, losing a day or more per feature, and getting paged by something that doesn't say what broke.

The report's authors put it politely: the vision is widely shared, the execution is not yet widespread. I'd put it less politely. A lot of us are describing a platform we haven't finished building, and that includes earlier versions of my own talks.

None of this is new, either. The CNCF's 2025 annual survey found 47% of organizations naming "cultural changes with the development team" as their number one adoption challenge, ahead of lack of training at 36%. The substrate has been settled for years. The practices around it haven't.

What shifting down actually requires

Closing that gap isn't a strategy problem, it's a build problem. The technical foundation is no longer speculative. We know how to get there. Most of it is mature, and the parts that aren't are visibly improving.

OpenTelemetry is the settled part

OpenTelemetry reaches 49% production use with another 26% evaluating, according to the same CNCF survey. It graduated in May 2026. And in CNCF's 2025 project velocity data it is the second-largest project in the ecosystem, logging 46,707 pull requests and issues across the year against Kubernetes' 40,559, though Kubernetes still leads comfortably on raw commits. This is mainstream infrastructure now.

What matters for shifting down is the separation it creates. Instrumentation is decoupled from destination. You instrument once and export wherever you choose. The Collector becomes the control point where enrichment, filtering, redaction, sampling and routing live, owned by the platform team instead of copy-pasted across a hundred repositories.

That's what makes it plausibly the last observability agent you install. Changing backend becomes a routing decision instead of a migration that touches every service. You still have to move dashboards, alerts and historical data, so the switch isn't free. But the invasive part, ripping instrumentation out of application code, stops being part of the equation.

One thing OpenTelemetry deliberately does not do is define the product experience. It standardizes how telemetry is produced and transported. It is not storage, not a query language, not a UI. Adopting the standard does not by itself give developers anything they'd call usable, which is exactly what happened with Kubernetes.

Semantic conventions are the shared vocabulary

Engineers don't debug in signals. They debug in layers of context. Who owns this? What service is it? Which container, which runtime, which build? Which cluster and namespace? Which region?

Correlation works when telemetry encodes those layers consistently and breaks when it doesn't. Semantic conventions are what make it consistent: service.name, deployment.environment.name, k8s.cluster.name, cloud.region, http.request.method, http.response.status_code. Not magic, and not AI. Structured metadata that means the same thing everywhere.

One gap is worth being honest about, because it's the layer people ask me about most and the one I can't point them to. The conventions cover infrastructure, compute, platform and architecture well. The organizational layer, who owns this service and who you call at 2am, still isn't covered by stable conventions. Work is underway. Until it lands, OpenTelemetry Weaver lets you define and enforce your own, which beats waiting for the spec.

Operators are the delivery mechanism

Conventions on their own are documentation, and documentation that says "please remember to enable tracing" is not a product. Something has to make the default happen without anyone choosing it.

In Kubernetes that something is an operator. The OpenTelemetry Operator gives you an Instrumentation custom resource where the platform team defines where telemetry goes and how it's configured. Annotate a pod and the operator injects zero-code instrumentation at deploy time for .NET, Java, Node.js, Python or Go. Zero-code is the term the project now uses for instrumenting without touching application source, and it is the whole ballgame for a platform team, because it is the only approach that scales to services the platform team has never seen.

I demo this at conferences because the shape lands better than the description. A developer scaffolds a service from a template, writes the business logic, deploys. There's nothing observability-related in the code at all. The service appears in the distributed trace immediately, parented correctly to the calls around it, carrying cluster and namespace and deployment attributes, context propagating across the boundary. Nobody did anything to make that happen except deploy.

Coverage isn't uniform and I won't pretend otherwise. Some language ecosystems are considerably better instrumented than others, and auto-instrumentation occasionally conflicts with application frameworks. Treat it as a strong default with known edges.

Developers still own the part only they can write

None of this says developers should stop caring about observability. It says the opposite, by clearing space for the part that needs them.

Adriana Villela gave a talk at Cloud Native Bergen last year called Observability Is a Team Sport, and the line I took away from it is one she credits to Josh Lee: don't make the developers do everything, but make everyone do observability. Her point is that an observability team operating in a silo defeats its own purpose, because observability touches every part of the development lifecycle.

The way I think about it is a football team. Different positions, different jobs, and the team fails when everyone crowds the same part of the field.

The platform can produce HTTP spans, database calls, queue operations, resource attributes and context propagation, because those are identical everywhere and should be solved once. The platform cannot produce the span that says a settlement completed, or the attribute explaining which fraud rule rejected a payout, or the SLO capturing what this team promises its callers. Write those on a developer's behalf and you get spans that are empty or spans that lie.

The platform provides the scaffolding. The developer provides the meaning. When a developer stops asking how to set up tracing and starts asking what business context to capture, the product is working.

Why this matters more now that agents are involved

I'd make this argument regardless of what's happening in AI. Reducing cognitive load and correlating signals were worth doing in 2019. But the timeline has compressed.

Agents consume telemetry differently than we do. A human investigates serially, one hypothesis at a time. Agentic tooling investigates in parallel, ten or more hypotheses at once, each hitting observability endpoints independently. The Weave guide puts a single LLM query at two to five times the telemetry of a standard log event, with parallel investigation generating query loads an order of magnitude or two beyond what existing architectures were sized for.

The volume isn't the interesting part. Agents reason over structure.

If service.name means the same thing everywhere, an agent can navigate. If cloud.region is consistent, it can scope. If trace context propagates across boundaries, it can follow a request through a system it has never seen. Without conventions, correlation fails. Without correlation, an agent guesses, and a confident guess in fluent prose is worse than no answer.

Garbage in, garbage out, same as it ever was. The survey found 20% of respondents still running proprietary instrumentation exclusively, which the analysts call AI readiness debt. That framing is right. Every investment in telemetry structure pays twice, once in human debugging and once in whatever you connect to it later.

So AI doesn’t remove the need to think about observability at the platform level. It raises the cost of skipping it.

Final thoughts

Observability isn't failing. It's at the point Kubernetes was at in 2019: powerful primitives, an enormous ecosystem, real innovation, no coherent default experience for the people expected to use it. Still in search of its Rails moment.

We solved that for infrastructure by treating the platform as a product, absorbing complexity centrally and leaving documented escape hatches for the cases the defaults don't fit. The same move works here, and most platform teams already believe it. What's left is closing the distance between believing it and having shipped it.

If you want somewhere to start this quarter: deploy the OpenTelemetry Collector as the single place telemetry policy lives, enforce semantic conventions there rather than asking every team to remember them, then put the OpenTelemetry Operator in front so deploying onto your platform is the only thing a developer does to get instrumented. That order is deliberate. Each step makes the next one cheaper.

I work at Dash0, so weigh the next paragraph accordingly. We built OpenTelemetry-native rather than OpenTelemetry-compatible, which means the conventions your platform team enforces are the ones the product reasons about directly, with no proprietary translation layer in between quietly losing things. Agent0, our AI control plane for production, scans the environment continuously, answers questions across telemetry and code in plain language with citations you can check, correlates across signals, and generates dashboards, alerts, runbooks and pull requests that are validated against live telemetry before they ship. It works better when the telemetry underneath has structure, which is this whole post applied to our own software.

If you're earlier in this, we wrote OpenTelemetry for Dummies as a short introduction, and there's more on semantic conventions and on why logs, metrics and traces work better together in our knowledge base.

Build the foundation now. The agents are already showing up, and they'll only ever be as good as the structure you hand them.