A single click in my app — "create ticket" — quietly touches four services and crosses a Kafka topic before anything reaches the user's inbox. When that click got slow, I had no idea which hop to blame. Running docker logs on six containers, one at a time, is not observability. It's guessing with extra steps.
So over one long weekend I finally fixed it: I added real distributed tracing to my microservices with SigNoz and OpenTelemetry — without touching a single line of my application code. Here's exactly how it went, including the part where my laptop tapped out.
The app: six services, zero visibility
The project is a side build of mine called TicketFlow — a small support-ticket system, but structured the "real" way:
- an API Gateway (JWT auth + routing)
- Eureka for service discovery
- Auth, User, and Ticket services, each with its own Postgres database
- a Notification service that emails users
- an Angular 17 frontend
- Kafka connecting the ticket flow to notifications
It looks clean in a diagram. But the "create ticket" request travels Gateway → Ticket service → Kafka → Notification service, and I had no way to actually see that journey. If it took four seconds, I couldn't tell you where those seconds went.
Why the OpenTelemetry Java agent
The thing that sold me was the OpenTelemetry Java agent. You attach one .jar with -javaagent, and it auto-instruments Spring MVC, Spring Cloud Gateway, JDBC (Postgres), and — the part I cared about most — Kafka producers and consumers. No SDK wiring in every service, no annotations. For six services, that's the difference between an evening and a week.
SigNoz is the backend that receives it all and lets you look at your traces, metrics, and logs in one place.
Self-hosting SigNoz (in WSL, the right way)
I'm on Windows, so everything runs in WSL2 (Ubuntu). First lesson, learned the annoying way: run Docker Engine inside WSL, not Docker Desktop. SigNoz's ClickHouse Keeper doesn't play nice with the Desktop backend on Windows.
Second lesson: the old install.sh most tutorials mention is deprecated now. The current self-host path is Foundry:
curl -fsSL https://signoz.io/foundry.sh | bash
A tiny casting.yaml:
apiVersion: v1alpha1
kind: Installation
metadata:
name: signoz
spec:
deployment:
flavor: compose
mode: docker
Then:
foundryctl cast -f casting.yaml
A few minutes of image pulls later, SigNoz was live on http://localhost:8080, with its collector listening on :4317.

SigNoz up and running — logs, traces, and metrics ingestion all active.
Instrumenting all six services with zero code changes
Because my services run as containers, I didn't open a single Java file. I downloaded the agent once into the project:
mkdir -p otel
wget -O otel/opentelemetry-javaagent.jar \
https://github.com/open-telemetry/opentelemetry-java-instrumentation/releases/latest/download/opentelemetry-javaagent.jar
Then, in docker-compose.yml, I merged this into each Java service's existing environment: block, changing OTEL_SERVICE_NAME for each one:
environment:
# ...my existing env vars...
JAVA_TOOL_OPTIONS: "-javaagent:/otel/opentelemetry-javaagent.jar"
OTEL_SERVICE_NAME: "ticket-service"
OTEL_EXPORTER_OTLP_ENDPOINT: "http://host.docker.internal:4317"
OTEL_EXPORTER_OTLP_PROTOCOL: "grpc"
OTEL_LOGS_EXPORTER: "otlp"
volumes:
- ./otel/opentelemetry-javaagent.jar:/otel/opentelemetry-javaagent.jar
extra_hosts:
- "host.docker.internal:host-gateway"
That extra_hosts line is the bit that trips people up: my app containers sit on their own Docker network, but SigNoz's collector is published on the host at :4317. host.docker.internal:host-gateway is how a WSL container reaches back to the host.
One conflict I hit immediately: my API Gateway published host port 8080 — the exact port SigNoz's UI uses. They can't both bind it, so the gateway refused to start. One-line fix: map the gateway to 8090:8080 instead. Internal routing (Angular → gateway over the Docker network) uses container names, so nothing else broke — I just hit the gateway at localhost:8090 after that.
The part where my laptop said no
First docker compose up --build, and the Angular build died on a TypeScript error — Cannot find name 'BuiltinIteratorReturn', a newer @types/node wanting a newer TypeScript than my project pins. A failed build aborts the whole up, so nothing started. I pinned @types/node and set skipLibCheck: true to get moving.
Then the real wall. I tried to boot all nine JVMs at once plus SigNoz's ClickHouse stack — on a laptop with about 6 GB of RAM total, where WSL was quietly capped at ~2.8 GB. The VM started swapping, then swap-died, and froze so hard the terminal stopped responding.
Not gonna lie — terminal frozen, laptop fans screaming, well past midnight — I seriously thought about giving up and going back to grepping logs the old way. But quitting over a memory setting felt too dumb, so I pushed on.
The fix was three things:
-
Heap caps —
-Xmx256mon each app'sJAVA_TOOL_OPTIONS, plus modestKAFKA_HEAP_OPTSfor Kafka and Zookeeper. -
More room for WSL — a
C:\Users\<me>\.wslconfigfile:
[wsl2]
memory=4GB
swap=6GB
then wsl --shutdown to apply it.
- A staggered bring-up instead of a boot storm:
docker compose up -d postgres zookeeper kafka
docker compose up -d eureka-server # wait ~30s until healthy
docker compose up -d auth-service user-service ticket-service notification-service api-gateway
After that, all five Spring Boot apps started, registered with Eureka, and the gateway reported {"status":"UP"}.
The payoff: watching a request cross Kafka
I generated some real traffic through the gateway — register, login, create a ticket — and opened SigNoz. All five services showed up right away.

All five services reporting in. Notice the multi-second P99 latency — that was my first hint the environment, not the code, was the bottleneck.
Then came the moment that made the whole exercise worth it — the trace for one create-ticket call.
I'd sketched this exact request flow on paper more times than I can count. Seeing it actually render — every hop in order, with real timings attached — was oddly satisfying, like the whiteboard diagram had finally come to life.

One trace, ten spans: api-gateway → ticket-service → INSERT ticket_db → ticket-events publish (Kafka producer) → ticket-events process on notification-service (Kafka consumer).
This is the feature I like most in SigNoz: the trace context propagates across the Kafka topic. The producer span in the ticket service and the consumer span in the notification service are stitched into the same trace — even though those two services never call each other and only meet through a topic named ticket-events. Two fully decoupled services, one continuous story. And the Postgres INSERT and SELECT calls sit right there as child spans, so I can see the DB time too.
There was a bonus insight hiding in the timings. The API call itself returned in about 3.6 seconds — but the trace didn't stop there. The notification service picked up the Kafka event and kept working for another ~10 seconds, long after the user already had their response. Because SigNoz keeps both halves in a single trace, I could actually see that asynchronous background work happening — something that's completely invisible if you only stare at the API's response time. That's the whole promise of Kafka — fire an event and move on — finally made visible.
It flagged failures cleanly too. A mistyped login showed up as its own trace, marked 401, with the error landing exactly on the auth service's Postgres lookup — no log-diving required.

A failed login, caught as a 401 — SigNoz pointed straight at where it broke.
What the traces actually taught me
Here's the twist. My requests were slow — multi-second spans everywhere. My first instinct was "my code is doing something dumb." But the traces told a different story: the time wasn't in my business logic. It was the environment — memory pressure and swapping dragging everything down. On a properly-resourced box, those same spans would collapse to milliseconds.
That's the real lesson of distributed tracing: it doesn't lie about where time goes. Without it, I'd have spent an evening "optimizing" ticket-service code that was never the problem. With it, one look at the waterfall pointed straight at the actual bottleneck.
A few things I'm taking away:
- The OpenTelemetry Java agent is the fastest way to instrument an existing Spring Boot fleet — Spring, JDBC, and Kafka, all with zero code changes.
- Kafka context propagation turns a pile of decoupled services back into one readable request flow. In an event-driven system, this alone justifies tracing.
- Self-hosting SigNoz is realistic, but budget your RAM honestly. On a small machine, cap the JVM heaps and give WSL room, or it will swap-die.
- Observability is a mindset shift: stop guessing where the time went, and go look.
If you're building anything where more than two services talk to each other, this is worth an afternoon. Mine fought back a little because of the hardware — but watching that one trace stitch itself across Kafka made it completely worth it.
Built with Spring Boot, Kafka, Angular, PostgreSQL, OpenTelemetry, and SigNoz.
Top comments (0)