DEV Community

Naveen Karasu
Naveen Karasu

Posted on

Day 13/60: Distributed Tracing with Zipkin -- Production Engineering

#sre #observability #distributedtracing #tutorial

Day 13/60: Distributed Tracing with Zipkin -- Production Engineering

60 Day Production Engineering Challenge

When a request crosses 15 service boundaries and times out, logs from individual services tell you almost nothing useful. Zipkin gives you a single trace view spanning every hop.

Zipkin Architecture

Four components: the collector receives spans (HTTP or Kafka transport), storage persists them, the query service exposes a retrieval API, and the web UI renders waterfall diagrams plus a dependency graph.

Deploy with Elasticsearch storage for production-grade tag searches:

version: '3.8'
services:
  zipkin:
    image: openzipkin/zipkin:3.1
    environment:
      STORAGE_TYPE: elasticsearch
      ES_HOSTS: http://elasticsearch:9200
      COLLECTOR_SAMPLE_RATE: '0.1'
    ports:
      - '9411:9411'
    depends_on:
      elasticsearch:
        condition: service_healthy

  elasticsearch:
    image: elasticsearch:8.12.0
    environment:
      discovery.type: single-node
      xpack.security.enabled: 'false'
      ES_JAVA_OPTS: -Xms1g -Xmx1g
    healthcheck:
      test: ['CMD', 'curl', '-f', 'http://localhost:9200/_cluster/health']
      interval: 10s
      retries: 10
    volumes:
      - es_data:/usr/share/elasticsearch/data

volumes:
  es_data:
Enter fullscreen mode Exit fullscreen mode

B3 Propagation and Brave Instrumentation

B3 headers (X-B3-TraceId, X-B3-SpanId, X-B3-ParentSpanId, X-B3-Sampled) carry trace context between services. With Spring Boot 3 and Micrometer, add these dependencies:

// build.gradle.kts
dependencies {
    implementation("io.micrometer:micrometer-tracing-bridge-brave")
    implementation("io.zipkin.reporter2:zipkin-reporter-brave")
}
Enter fullscreen mode Exit fullscreen mode

Configure in application.yml:

management:
  tracing:
    sampling:
      probability: 0.1
  zipkin:
    tracing:
      endpoint: http://zipkin:9411/api/v2/spans
spring:
  application:
    name: order-service
Enter fullscreen mode Exit fullscreen mode

Brave automatically instruments RestTemplate and WebClient calls with B3 headers. Every inbound request gets a span, every outbound call propagates the trace.

Storage Backend Comparison

Backend Throughput Query Speed Best For
Elasticsearch High Fast tag search Most production deployments
Cassandra Very high Good by trace ID Teams already running Cassandra
MySQL Low (~10k spans/s) Slow at scale Development only

Zipkin vs Jaeger

Jaeger has adaptive sampling (automatic per-endpoint rate adjustment) and a Kubernetes Operator. Zipkin has a broader client library ecosystem via Brave and simpler operations. For Java/Spring Boot stacks, Zipkin. For Kubernetes-native with adaptive sampling needs, Jaeger. For new projects, use OpenTelemetry SDK with a Zipkin exporter and defer the backend choice.

Day 13/60 of the 60 Day Production Engineering Challenge

Top comments (0)