A Beginner’s Guide to Distributed Tracing with OpenTelemetry and Jaeger

Distributed systems are complex, with requests flowing through multiple services. Debugging issues in such systems can be challenging. Distributed tracing, using tools like OpenTelemetry and Jaeger, provides visibility into these interactions, helping us identify performance bottlenecks and troubleshoot effectively.

In this blog, we'll explore how to instrument a simple Python application with OpenTelemetry and send trace data to Jaeger for visualization.

---

Why Distributed Tracing?

Distributed tracing captures the lifecycle of a request across multiple services. Key benefits include:

• End-to-End Visibility: Understand how a request flows through your system.
• Performance Insights: Identify slow components and bottlenecks.
• Error Tracking: Pinpoint where failures occur in your application stack.

---

Overview of the Setup

We'll create a simple Python application and configure OpenTelemetry to send trace data to Jaeger. Here's what we’ll cover:

1. Setting Up Jaeger
2. Instrumenting a Python App with OpenTelemetry
3. Visualizing Traces in Jaeger
4. Code Walkthrough
5. Demo: Step-by-Step Execution

---

1. Setting Up Jaeger

Jaeger is an open-source tool for tracing and monitoring distributed systems. We’ll use the all-in-one Docker image for simplicity.

Command to Run Jaeger

docker run -d --name jaeger \
  -e COLLECTOR_ZIPKIN_HTTP_PORT=9411 \
  -p 5775:5775 -p 6831:6831/udp -p 6832:6832/udp \
  -p 5778:5778 -p 16686:16686 -p 14250:14250 -p 14268:14268 \
  jaegertracing/all-in-one:1.31

• 16686: Jaeger UI (View traces).
• 14250: GRPC endpoint (Receive traces from OpenTelemetry).
• 9411: Zipkin-compatible endpoint.

Why Jaeger? It provides a comprehensive solution for distributed tracing with a simple UI for visualizing trace data.

---

2. Instrumenting a Python App with OpenTelemetry

OpenTelemetry is a set of APIs and tools for collecting telemetry data like traces and metrics. We'll use it to instrument a Python app.

Install OpenTelemetry Libraries

pip install flask opentelemetry-api opentelemetry-sdk \
    opentelemetry-exporter-jaeger opentelemetry-instrumentation-flask

Code: A Simple Flask Application

File: app.py

from flask import Flask, jsonify
from opentelemetry import trace
from opentelemetry.exporter.jaeger.proto.grpc import JaegerExporter
from opentelemetry.sdk.resources import Resource
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor

# Initialize Flask app
app = Flask(__name__)

# Configure OpenTelemetry
trace.set_tracer_provider(
    TracerProvider(resource=Resource.create({"service.name": "python_app"}))
)
tracer = trace.get_tracer_provider().get_tracer(__name__)

# Export traces to Jaeger
jaeger_exporter = JaegerExporter(
    collector_endpoint="http://localhost:14250", insecure=True
)
span_processor = BatchSpanProcessor(jaeger_exporter)
trace.get_tracer_provider().add_span_processor(span_processor)

@app.route('/')
def home():
    with tracer.start_as_current_span("home-span"):
        return jsonify(message="Welcome to the home page!")

@app.route('/process')
def process():
    with tracer.start_as_current_span("process-span"):
        return jsonify(message="Processing done!")

if __name__ == '__main__':
    app.run(host='0.0.0.0', port=5000)

• Trace Provider: Defines the application as python_app for Jaeger.
• Jaeger Exporter: Sends traces to Jaeger’s GRPC endpoint.
• Spans: Represent units of work, like handling a request.

---

3. Visualizing Traces in Jaeger

After instrumenting the application:

1. Access the Jaeger UI: http://localhost:16686.
2. Select the service python_app from the dropdown.
3. Click Find Traces to view recent traces.

---

4. Dockerizing the Application

To simplify deployment, we’ll use Docker Compose to run both Jaeger and the Flask app.

Docker Compose File

File: docker-compose.yml

version: '3.7'

services:
  jaeger:
    image: jaegertracing/all-in-one:1.31
    ports:
      - "5775:5775/udp"
      - "6831:6831/udp"
      - "6832:6832/udp"
      - "5778:5778"
      - "16686:16686"
      - "14250:14250"
      - "14268:14268"
      - "9411:9411"

  python_app:
    build:
      context: .
    ports:
      - "5000:5000"
    environment:
      - OTEL_EXPORTER_JAEGER_ENDPOINT=http://jaeger:14250

Dockerfile for Flask App

File: Dockerfile

FROM python:3.9-slim

WORKDIR /app

COPY requirements.txt requirements.txt
RUN pip install -r requirements.txt

COPY . .

CMD ["python", "app.py"]

File: requirements.txt

flask
opentelemetry-api
opentelemetry-sdk
opentelemetry-exporter-jaeger
opentelemetry-instrumentation-flask

---

5. Running the Demo

1. Build and Start the Stack:

   docker-compose up --build

1. Access the App:

   • Visit http://localhost:5000.
   • Try / and /process routes to generate traces.

2. View Traces:

   • Open http://localhost:16686.
   • Select python_app and analyze the traces.

---

Code Explanation

• Spans: Represent individual operations in your app (e.g., home-span, process-span).
• Trace Attributes: Metadata added to spans for better debugging.
• Jaeger Exporter: Sends trace data to the Jaeger instance.

---

Conclusion

In this tutorial, we learned to:

1. Set up Jaeger for distributed tracing.
2. Instrument a Python app using OpenTelemetry.
3. Use Docker Compose to simplify the setup.

With this foundation, you can extend the project by adding custom spans, simulating failures, or tracing across microservices. Distributed tracing is a vital skill for debugging and optimizing modern applications.