DEV Community

Cover image for Monitoring Java Microservices with OpenTelemetry and OpenObserve
OpenObserve profile image Manas Sharma
Manas Sharma for OpenObserve

Posted on Edited on Originally published at openobserve.ai

Monitoring Java Microservices with OpenTelemetry and OpenObserve

#java #springboot #opentelemetry #observability

Monitoring microservices is hard.

When a user request fans out across multiple services, each with its own database, logs, and failure modes, traditional monitoring tools often give you a fragmented picture. You can tell something is slow, but not exactly where or why.

Distributed tracing solves this.

In this tutorial, we'll implement distributed tracing for a Java Spring Boot microservices application using two open-source tools: OpenTelemetry and OpenObserve.

If your stack includes other languages, check out these guides too:

What you'll build

By the end of this guide, you'll have:

  • A working Spring Boot microservices setup with cross-service HTTP calls
  • Zero-code instrumentation using the OpenTelemetry Java Agent
  • End-to-end traces in OpenObserve with flamegraph and Gantt chart views

What is distributed tracing?

In microservices, one user action can trigger a chain of calls across many services. If a request takes 3 seconds, tracing helps answer:

  • Which service caused the delay?
  • Which operation failed?
  • Where exactly time was spent?

Distributed tracing works by attaching context (trace_id, span_id) at request entry and propagating it across service boundaries (usually with traceparent headers). This gives you one complete request journey.

A trace is made up of spans. Each span records:

  • Service + operation
  • Start time + duration
  • HTTP details (method, URL, status)
  • DB query metadata
  • Errors/exceptions
  • Parent-child relationships

Key components of distributed tracing in an e-commerce example

For deeper fundamentals: Distributed Tracing Basics to Beyond

Why OpenTelemetry + OpenObserve?

OpenTelemetry

OpenTelemetry is a CNCF standard for traces, metrics, and logs.

For Java, the OpenTelemetry Java Agent can auto-instrument Spring Boot, JDBC, and HTTP clients with no code changes.

OpenObserve

OpenObserve is an open-source backend for logs, metrics, and traces.

  • OTLP-native ingest
  • SQL-powered analytics
  • Unified observability in one interface
  • Lightweight and storage-efficient

Architecture used in this tutorial

We'll run four services:

Service Port Responsibility
discovery-service 8761 Eureka registry
user-service 8081 User CRUD (MySQL)
order-service 8082 Order management; calls user-service
payment-service 8083 Payment processing; calls order-service

The key trace path is:

payment-service -> order-service -> user-service -> MySQL

Payment Processing Workflow

Prerequisites

  • Java 17+
  • Maven 3.8+
  • Docker + Docker Compose
  • MySQL 8 (or use Dockerized MySQL from compose)

Step 1: Clone the project 

git clone https://github.com/openobserve/java-distributed-tracing.git
cd java-distributed-tracing
Enter fullscreen mode Exit fullscreen mode

Step 2: Start OpenObserve and MySQL 

docker-compose up -d
Enter fullscreen mode Exit fullscreen mode

Docker Compose startup

This starts:

  • OpenObserve: http://localhost:5080
  • MySQL: localhost:3306 (tracingdb)

Login to OpenObserve with:

  • Email: admin@example.com
  • Password: Admin123!

OpenObserve dashboard

Step 3: Download OpenTelemetry Java Agent 

mkdir agents
curl -L https://github.com/open-telemetry/opentelemetry-java-instrumentation/releases/latest/download/opentelemetry-javaagent.jar \
  -o agents/opentelemetry-javaagent.jar
Enter fullscreen mode Exit fullscreen mode

Download OpenTelemetry Java Agent

Step 4: Configure agent export to OpenObserve

Example from user-service/scripts/start.sh:

export OTEL_SERVICE_NAME=user-service
export OTEL_RESOURCE_ATTRIBUTES=service.name=user-service,deployment.environment=dev
export OTEL_TRACES_EXPORTER=otlp
export OTEL_METRICS_EXPORTER=none
export OTEL_LOGS_EXPORTER=none
export OTEL_EXPORTER_OTLP_TRACES_ENDPOINT=http://localhost:5080/api/default/traces
export OTEL_EXPORTER_OTLP_TRACES_PROTOCOL=http/protobuf
export OTEL_EXPORTER_OTLP_TRACES_HEADERS="Authorization=Basic {token}"

java \
  -Xms256m \
  -Xmx512m \
  -javaagent:../agents/opentelemetry-javaagent.jar \
  -jar target/user-service-0.0.1-SNAPSHOT.jar
Enter fullscreen mode Exit fullscreen mode

Get {token} from OpenObserve UI:

OpenTelemetry token location

Step 5: Start discovery-service 

cd discovery-service
mvn clean install -Dmaven.test.skip
sh scripts/start.sh
Enter fullscreen mode Exit fullscreen mode

Open: http://localhost:8761

Discovery service

Step 6: Start user/order/payment services

Run each in a separate terminal.

cd user-service
mvn clean install -Dmaven.test.skip
sh scripts/start.sh
Enter fullscreen mode Exit fullscreen mode 
cd order-service
mvn clean install -Dmaven.test.skip
sh scripts/start.sh
Enter fullscreen mode Exit fullscreen mode 
cd payment-service
mvn clean install -Dmaven.test.skip
sh scripts/start.sh
Enter fullscreen mode Exit fullscreen mode

Verify registration in Eureka:

Eureka dashboard

Step 7: Generate traces

1) Create user 

curl -X POST http://localhost:8081/api/users \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Priya Sharma",
    "email": "priya@example.com",
    "phone": "+91-9876543210"
  }'
Enter fullscreen mode Exit fullscreen mode

Create user API

2) Create order 

curl -X POST http://localhost:8082/api/orders \
  -H "Content-Type: application/json" \
  -d '{
    "userId": 1,
    "productName": "Mechanical Keyboard",
    "quantity": 1,
    "totalAmount": 4999.00
  }'
Enter fullscreen mode Exit fullscreen mode

Create order API

3) Process payment (full distributed trace) 

curl -X POST http://localhost:8083/api/payments/process \
  -H "Content-Type: application/json" \
  -d '{
    "userId": 1,
    "orderId": 1,
    "amount": 4999.00,
    "currency": "INR",
    "paymentMethod": "UPI"
  }'
Enter fullscreen mode Exit fullscreen mode

Process payment API

4) Trigger an error trace 

curl -X POST http://localhost:8082/api/orders \
  -H "Content-Type: application/json" \
  -d '{
    "userId": 9999,
    "productName": "Test Product",
    "quantity": 1,
    "totalAmount": 100.00
  }'
Enter fullscreen mode Exit fullscreen mode

Expected: 400 Bad Request

Error test API

Visualize in OpenObserve

Go to http://localhost:5080 -> Traces

Trace Explorer

You'll see:

  • Trace ID
  • Root span
  • Service
  • Duration
  • Span count
  • Status

Trace explorer

Filter examples

  • service_name = payment-service
  • status = ERROR
  • Duration range
  • operation_name for specific endpoints

Filter by service
Filter by error

Flamegraph + Gantt chart

Click a POST /api/payments/process trace.

  • Flamegraph: nested span timing hierarchy
  • Gantt: timeline-aligned span bars

Flamegraph view
Gantt chart view

Query traces with SQL

OpenObserve supports SQL over trace data.

Slowest payment traces 

SELECT trace_id, duration, service_name, operation_name
FROM "default"
WHERE service_name = 'payment-service'
  AND operation_name LIKE '%payments/process%'
ORDER BY duration DESC
LIMIT 10;
Enter fullscreen mode Exit fullscreen mode

Slow traces SQL

Error count by service 

SELECT service_name, COUNT(*) as error_count
FROM "default"
WHERE span_status = 'ERROR'
GROUP BY service_name
ORDER BY error_count DESC;
Enter fullscreen mode Exit fullscreen mode

Error count SQL

Avg/max latency by service 

SELECT service_name,
       AVG(duration) as avg_duration_us,
       MAX(duration) as max_duration_us,
       COUNT(*) as request_count
FROM "default"
GROUP BY service_name;
Enter fullscreen mode Exit fullscreen mode

Latency SQL

What the Java agent captured automatically

Without adding tracing code, the OpenTelemetry Java Agent instrumented:

  • Spring Web incoming HTTP requests
  • RestTemplate outbound calls (traceparent injected)
  • JDBC/MySQL queries
  • Context propagation across service boundaries

See supported libraries: OpenTelemetry Java Instrumentation

Final takeaway

You now have end-to-end distributed tracing for a Java microservices app with:

  • Zero-code instrumentation
  • Full request path visibility
  • Visual root-cause analysis (flamegraph/Gantt)
  • SQL-based troubleshooting in OpenObserve
  • A path to production scaling without vendor lock-in

Further reading

Top comments (0)