How Distributed Tracing Really Works

It's relatively simple these days to spin up an OpenTelemetry demo and / or auto-instrument services with OpenTelemetry.

That's fine - when it works - but it's when something breaks that perhaps you realise you don't quite understand what's going on under the covers. I've lost count of the number of times customers have reported "broken traces" to me as they flow through an uninstrumented tier, a load balancer or a queue.

In the following video I show precisely how distributed tracing using OpenTelemetry is able to do what it does (for HTTP) by spinning up two Python FastAPI applications, instrumenting them using OpenTelemetry and demonstrating that the traces are "broken" in Jaeger. Then I solve the issue (hint: it's all about propagation).

The code for all of this can be found here: https://github.com/agardnerIT/python-fastapi-requests-traceparent