DEV Community

Alex Spinov
Alex Spinov

Posted on

Grafana Tempo Has a Free Distributed Tracing Backend You Should Know About

#tracing #grafana #microservices #devops

Distributed tracing is essential for microservices debugging. But Jaeger needs Elasticsearch, and commercial solutions cost thousands per month.

Grafana Tempo is different: it stores traces in object storage (S3, GCS, MinIO) with no indexing database required. The result? 10-100x cheaper than alternatives.

Why Tempo Is Different

No Index Required

Traditional tracing backends (Jaeger, Zipkin) store traces in Elasticsearch or Cassandra. Tempo stores them directly in object storage and finds them via trace ID lookup or integration with Loki/Prometheus.

Traditional: Trace → Index (Elasticsearch) → Query
Tempo:       Trace → Object Storage → Lookup by ID
                                    → Search via metrics/logs correlation
Enter fullscreen mode Exit fullscreen mode

Cost Comparison

Jaeger + ES Tempo + S3
Storage cost (1TB) $200-500/mo $23/mo
Index overhead 2-3x data size None
Operational burden High Low

Setup in 5 Minutes 

# docker-compose.yml
services:
  tempo:
    image: grafana/tempo:latest
    command: ["-config.file=/etc/tempo.yaml"]
    volumes:
      - ./tempo.yaml:/etc/tempo.yaml
    ports:
      - "3200:3200"   # tempo API
      - "4317:4317"   # OTLP gRPC
      - "4318:4318"   # OTLP HTTP
Enter fullscreen mode Exit fullscreen mode 
# tempo.yaml
server:
  http_listen_port: 3200
distributor:
  receivers:
    otlp:
      protocols:
        grpc:
        http:
storage:
  trace:
    backend: local
    local:
      path: /tmp/tempo/blocks
Enter fullscreen mode Exit fullscreen mode

Send Traces via OpenTelemetry 

const { NodeSDK } = require("@opentelemetry/sdk-node");
const { OTLPTraceExporter } = require("@opentelemetry/exporter-trace-otlp-grpc");
const { getNodeAutoInstrumentations } = require("@opentelemetry/auto-instrumentations-node");

const sdk = new NodeSDK({
  traceExporter: new OTLPTraceExporter({
    url: "http://localhost:4317",
  }),
  instrumentations: [getNodeAutoInstrumentations()],
});
sdk.start();
Enter fullscreen mode Exit fullscreen mode

Every HTTP request, DB query, and external call is now traced and stored in Tempo.

TraceQL — Query Language for Traces 

// Find slow database spans
{ span.db.system = "postgresql" && duration > 500ms }

// Find errored spans in payment service
{ resource.service.name = "payment" && status = error }

// Structural queries — find traces with specific patterns
{ span.http.method = "POST" } >> { span.db.statement =~ "INSERT.*orders" }
Enter fullscreen mode Exit fullscreen mode

The Killer Feature: Logs-to-Traces

With Loki + Tempo in Grafana, you can jump from a log line directly to its trace:

  1. See error in Loki: {app="api"} |= "timeout"
  2. Click the trace ID in the log line
  3. Grafana opens the full distributed trace in Tempo
  4. See exactly which service/query caused the timeout

This correlation is free — just configure derived fields in Grafana.

Building microservices and need observability? I help teams set up production tracing and monitoring. Email spinov001@gmail.com or explore my developer tools.

Top comments (0)