OpenTelemetry span exceptions: record them in your traces

It is 3 a.m., the alert says 4% of payments are failing, and the dashboard is a wall of red bars. What you are missing is OpenTelemetry span exceptions: without them you know a span failed but never why. With them, the stack trace sits in the same trace as the error.

I start from that scene and work back along the chain with the Python SDK: what to write in code, what the span really stores, the call to make about the announced deprecation, and then triage in Dynatrace SaaS. The SRE goal is plain: cut MTTR by going from an error rate to a cause in two queries.

What is checked and what is not. I ran the Python code with opentelemetry-sdk 1.45.2 and show the real output. I did not run the DQL queries: they come from the Dynatrace documentation and serve as examples. Everything applies to Dynatrace SaaS (Grail, DQL); Dynatrace Managed is out of scope.

Two facts, two fields: status and exception

A span that ends silently has the status Unset, which the OpenTelemetry documentation reads as « no error ». To count it in an SLI you must set ERROR. That is the first field: you filter and count on it.

The second field answers the 3 a.m. question. The exception lives in an event attached to the failing span, hence to the right service, the right operation and the right trace_id. No manual matching against a log file is needed.

Keep this reading rule: status is for counting, the event is for understanding. Neither is enough alone.

Writing OpenTelemetry span exceptions in code

In its Python examples, OpenTelemetry pairs a set_status with a record_exception. Nothing on the page I read says an exception crossing a with block triggers both automatically. So I write them myself, in one place.

My test rig: the opentelemetry-sdk package, a console exporter to read the span without a backend, and a payment function that fails on purpose.

Two details matter in this file. Line 18 copies the message into the status description, so it is visible without opening the event. Line 19 creates the event. And if the caller must know about the failure, finish the block with a raise: otherwise parent spans stay green while the child failed.

Reading the span: what the run produced

The specification fixes the event name exception and three attributes: exception.type, exception.message and exception.stacktrace. Type and message are « conditionally required » (at least one), the stack trace is only recommended. This is what my console printed.

• Status and event sit in the same object, yet they are independent pieces of data.
• My SDK also adds exception.escaped. The specification calls it deprecated, and asks you to stop recording handled exceptions that never leave the span.
• The message travels to your backend untouched, and the specification warns it may hold sensitive data. A customer number or a token has no place in exception text.

Should the deprecation worry you? A three-step decision

The specification page on exceptions in spans shows « Deprecated » and points to the log conventions. Yet the SDK I ran still emits the event. Both facts hold at once, and the arbitration is yours.

To steer the transition, the specification offers the variable OTEL_SEMCONV_EXCEPTION_SIGNAL_OPT_IN:

• absent: the instrumentation keeps the span event;
• logs: exceptions leave only as logs;
• logs/dup: both signals, for a gradual rollout.

On the log side, severity follows gravity: ERROR (17) for an unhandled exception that does not stop the application, WARN (13) for one the code is meant to handle. When the instrumentation also creates spans, the log must carry the context of the span concerned.

Here is how I decide. Step one: while your SDK emits the event and your queries read it, change nothing. Step two: try logs/dup on a low-risk service. Step three: switch the event off only after rewriting your queries.

Beware, each SDK chooses whether to honour this variable. With logs and Python SDK 1.45.2, the event stayed in the span. I did not test other languages or auto-instrumentation agents.

The same switchover logic exists for HTTP: see OpenTelemetry semantic conventions and their stability. For the log side, read stable OpenTelemetry logs.

From error rate to cause in Dynatrace SaaS

Spans arrive over OTLP, directly or through a Collector. The setup is covered in the OpenTelemetry Collector to Dynatrace. I did not test ingestion end to end.

The Dynatrace documentation page « Use traces, DQL, and logs to spot patterns » offers a triage query, which I annotate here in my own words.

Read top to bottom, it does four things. First, filter iAny(…) drops every span without an exception event. Next, expand span.events unfolds the list into one row per event. Then fieldsFlatten turns exception.type and exception.message into columns. Finally, summarize counts by service and message: at the top sits the likely culprit.

To see when the trouble began, the same page suggests makeTimeseries count(), by: {exception.type}, time:start_time. Once you have the service, open a failing trace and read the stack. If the trace does not exist, go through diagnosing missing OpenTelemetry traces first.

Not verified: the exact name of the status field in Grail spans. I will not invent it. Check it in a notebook before turning it into an alert.

Five traps I met

• Exception without status: it shows in the trace but never enters your error rate.
• Status without exception: a red span, no stack.
• Handled exceptions: if the code catches them and carries on, do not record them; you would drown the real errors.
• Double counting: with logs/dup the same exception exists twice. Deduplicate before counting.
• Misread deprecation: it targets the span convention. The record_exception call still appears in the official Python example I read.

Best practices

• Status and exception in the same place, once per span.
• A raise after recording, so parents see the failure.
• Migrate with logs/dup, measuring before any cut-off.
• Exception messages free of personal data.
• Logs tied to traces, as in OpenTelemetry logs with trace_id.

Frequently asked questions

Do I still need to call record_exception?

Yes, if your SDK produces the event and your backend reads it. The transition variable shows the default remains the span event. Test your own SDK.

ERROR status or exception event: which one?

Both. The status lets you filter and count failures; the event gives the type, message and stack trace.

How do I rank services by number of exceptions?

Adapt the documentation query: filter iAny on the event, expand, fieldsFlatten, then summarize count() by service.name. These queries apply to Dynatrace SaaS.

Conclusion

Keep three habits: set the status, record the event, plan the exit towards logs. Today the span event works with the SDK I tested, and the official direction is logs. Evolve your queries with your SDKs, and verify each step on your own system.

span.observer is an independent community blog, not affiliated with Dynatrace LLC. Trademarks belong to their owners. Tools change quickly: check commands and options in the official documentation. Sources: specification of exceptions in spans (page read on 8 November 2026).

---

Originally published on span.observer. French version: https://span.observer/fr/exceptions-opentelemetry-traces/.