🚀 Getting Started with OpenTelemetry for Python Microservices on Kubernetes

Shubham Chavan — Sun, 04 May 2025 18:38:00 +0000

Here’s a clean, step-by-step guide to help you get up and running with OpenTelemetry in a real-world, containerized setup ideal for Python-based services running in Kubernetes.

🆚 Auto-Instrumentation vs Manual-Instrumentation in OpenTelemetry

Feature	Auto-Instrumentation	Manual Instrumentation
Definition	Automatically instruments supported libraries and frameworks using OpenTelemetry wrappers.	Requires you to explicitly define spans in code using the OpenTelemetry API.
Setup	Minimal changes — just install dependencies and run via `opentelemetry-instrument` CLI.	Requires importing OTEL SDK and wrapping logic with spans manually.
Use Cases	Quick observability setup, ideal for supported web frameworks like Flask, Django, FastAPI, SQLAlchemy, etc.	Fine-grained control — custom logic, background jobs, async flows, or business-critical code.
Example	`opentelemetry-instrument python app.py`	`python with tracer.start_as_current_span("process-order"): do_work()`
Pros	Easy, fast, low effort. Captures most useful traces out of the box.	Complete control over what is traced, how it’s labeled, and where context is passed.
Cons	Limited to what the SDK supports; may miss custom logic or async paths.	Requires effort, code changes, and understanding of OTEL trace context.

🧩 Architecture Overview

We’ll use:

OpenTelemetry SDK in a Python app
OTEL Collector Agent (sidecar or DaemonSet on app clusters)
OTEL Collector Gateway (centralized, on a GitOps or management cluster)
SigNoz as the visualization backend (optional – you can swap with Jaeger, Datadog, etc.)

📦 Step 1: Instrument Your Python Application

Install OpenTelemetry dependencies:

pip install opentelemetry-sdk \
            opentelemetry-exporter-otlp \
            opentelemetry-instrumentation \
            opentelemetry-instrumentation-flask  # or your framework

Initialize OTEL in your code:

from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter

trace.set_tracer_provider(TracerProvider())
tracer = trace.get_tracer(__name__)

otlp_exporter = OTLPSpanExporter(endpoint="http://otel-collector:4318/v1/traces")
span_processor = BatchSpanProcessor(otlp_exporter)
trace.get_tracer_provider().add_span_processor(span_processor)

Instrument libraries:

opentelemetry-instrument python app.py

🏗️ Step 2: Deploy OpenTelemetry Collector Agent

Use the official Helm chart or a custom otel-agent.yaml.

Basic example (DaemonSet mode):

receivers:
  otlp:
    protocols:
      http:
      grpc:

exporters:
  otlp:
    endpoint: <gateway-collector>:4317
    tls:
      insecure: true

service:
  pipelines:
    traces:
      receivers: [otlp]
      exporters: [otlp]

🌐 Step 3: Deploy OTEL Collector Gateway

This collector aggregates all telemetry. Example config:

receivers:
  otlp:
    protocols:
      grpc:
      http:

exporters:
  signoz:
    endpoint: http://signoz-otel-collector:4317

service:
  pipelines:
    traces:
      receivers: [otlp]
      exporters: [signoz]

📊 Step 4: Install SigNoz or Jaeger as Your Observability Backend

helm repo add signoz https://charts.signoz.io helm install signoz signoz/signoz

🛠 Tips

Use resource attributes to tag services (service.name, env, etc.).
Add custom spans around critical code blocks.
Use batch processors to reduce export overhead.

DEV Community: Shubham Chavan

🚀 Getting Started with OpenTelemetry for Python Microservices on Kubernetes