End-to-End Observability to AI Application Using OpenTelemetry and SigNoz

Abhishek Ugare — Sun, 26 Jul 2026 07:04:15 +0000

End-to-End Observability for AI Applications Using OpenTelemetry and SigNoz

How I implemented distributed tracing, monitored an AI application end-to-end, and transformed debugging with OpenTelemetry and SigNoz.

Introduction

Building an AI application is exciting. Debugging it when requests travel through multiple services is an entirely different challenge.

When I started building Yaatra, my AI-powered travel assistant, the architecture looked simple.

A user would:

Enter travel preferences.
The backend would process the request.
The AI engine would generate a personalized itinerary.
The response would be displayed on the frontend.

At least, that's how it looked on paper.

Behind the scenes, a single request was traveling through multiple independent services. When something became slow—or worse, failed—I had no easy way to identify the root cause.

Questions started appearing almost immediately:

Was the backend slow?
Was the AI engine taking longer than expected?
Was the external LLM causing delays?
Did the request even reach the AI service?

Traditional logging only provided fragments of the story. I needed a complete picture.

That's when I decided to implement end-to-end observability using OpenTelemetry and SigNoz.

In this blog, I'll share my journey of instrumenting an AI-native application, the challenges I encountered, the lessons I learned, and how observability completely changed the way I debug distributed systems.

The Problem

Yaatra isn't a monolithic application.

Although users interact with a single interface, every request passes through multiple services before a response is generated.

Initially, my debugging strategy looked something like this:

console.log("Request received");
console.log("Calling AI Engine...");
console.log("Response generated");

This worked during early development.

As the project grew, the logs quickly became insufficient.

I couldn't answer questions like:

Which service is responsible for high latency?
How long does the AI engine spend processing?
Are requests successfully reaching the AI service?
Which request generated this log?
How can I follow one request across multiple services?

I needed more than logs.

I needed visibility into the complete lifecycle of every request.

Why OpenTelemetry?

I wanted an observability solution that wasn't tied to a single vendor.

OpenTelemetry stood out because it has become the industry standard for collecting telemetry data.

Instead of manually adding monitoring logic throughout my codebase, OpenTelemetry automatically instruments supported libraries and exports telemetry to any compatible backend.

For visualization, I chose SigNoz, an open-source observability platform that supports:

Distributed Traces
Metrics
Logs
Dashboards

Together, they provided everything I needed:

Distributed tracing
Automatic instrumentation
Performance monitoring
Request latency analysis
Error tracking
Custom dashboards

My Tech Stack

Gateway Service

Node.js
Express.js
OpenTelemetry SDK
OTLP Exporter
HTTP Instrumentation

AI Engine

FastAPI
Groq LLM
Python OpenTelemetry SDK

Observability Stack

OpenTelemetry
SigNoz
OTLP Protocol
Docker

Understanding Distributed Tracing

Before implementing observability, I had heard about distributed tracing, but never fully understood its value.

Once I started using it, everything clicked.

Every incoming request creates a Trace.

Every operation performed while processing that request becomes a Span.

Instead of viewing every service independently, I could now follow a request throughout its entire journey.

Frontend
    │
    ▼
Gateway Service
    │
    ▼
AI Engine
    │
    ▼
LLM (Groq)
    │
    ▼
Response

Each step became visible.

This was probably the biggest mindset shift during the project.

Instrumenting the Gateway

The gateway receives requests from the frontend and forwards them to the AI engine.

To instrument it, I added the OpenTelemetry SDK to my Node.js application.

The SDK included:

Node SDK
OTLP Trace Exporter
HTTP Instrumentation
Resource Configuration

One issue confused me for quite some time.

No traces were appearing inside SigNoz.

After debugging, I discovered that OpenTelemetry must be initialized before importing Express.

Changing the initialization order immediately solved the issue.

A tiny implementation detail—but one that consumed several hours.

Instrumenting the AI Engine

The AI engine is built using FastAPI.

Unlike Node.js, it required the Python OpenTelemetry SDK.

I instrumented:

FastAPI
Requests library
OTLP Exporter

Once configured, every request reaching the AI engine automatically became a child span of the gateway request.

This was the moment distributed tracing finally made sense.

Instead of seeing unrelated requests, SigNoz displayed a complete waterfall showing the entire request journey.

Context Propagation Was the Missing Piece

One concept that initially confused me was context propagation.

Generating spans alone isn't enough.

The trace context must travel alongside every request.

Without context propagation:

Gateway creates one trace
AI Engine creates another trace

There is no relationship between them.

With propagation enabled:

Single Trace

Gateway
   │
   ├── AI Engine
   │      │
   │      └── LLM Request
   │
   └── Response

Everything belongs to one trace.

This became one of the most valuable lessons from the entire implementation.

Sending Telemetry to SigNoz

Both services exported telemetry using the OTLP protocol.

Once telemetry reached SigNoz, I could immediately visualize:

Active services
Incoming requests
Distributed traces
Span duration
Service dependencies
Errors

Instead of scrolling through thousands of log lines, I could simply click a trace and inspect the complete execution path.

Building Meaningful Dashboards

Collecting traces wasn't my end goal.

I wanted dashboards that answered real operational questions.

Request Count

Tracks incoming requests over time.

Useful for identifying traffic spikes.

Average Request Latency

Measures response times.

Latency increases become immediately visible.

AI Processing Time

A custom metric showing how much time the AI engine spends generating responses.

Request Volume

A bar chart comparing request activity across different time periods.

Error Monitoring

Displays failed requests and helps identify problems before users report them.

Challenges I Faced

Not everything worked on the first attempt.

Missing Traces

Initially, no traces appeared.

The problem turned out to be the SDK initialization order.

Broken Context Propagation

The gateway and AI engine were generating completely independent traces.

Proper propagation fixed the issue.

Force Flush Issues

During development, telemetry wasn't always exported immediately.

Understanding how and when telemetry is flushed made debugging much easier.

Dashboard Design

Creating dashboards is easy.

Creating useful dashboards is much harder.

I realized dashboards should answer questions—not simply display numbers.

What I Learned

Logs Are Not Enough

Logs tell you what happened.

Traces tell you:

Where it happened
How long it took
What happened before
What happened after

Observability Should Be Added Early

Adding instrumentation later is possible.

Adding it early makes development and debugging significantly easier.

Distributed Systems Need Distributed Tracing

As applications become more service-oriented, console logs become increasingly difficult to manage.

Distributed tracing provides context that logs alone cannot.

Dashboards Save Time

Instead of manually inspecting logs, dashboards immediately reveal:

Increased latency
Higher request volume
Failed requests
Slow downstream services

Architecture Overview

               Frontend
                   │
                   ▼
        Gateway Service (Node.js)
                   │
         OpenTelemetry SDK
                   │
          OTLP Trace Exporter
                   │
                   ▼
          AI Engine (FastAPI)
                   │
         OpenTelemetry SDK
                   │
                   ▼
              Groq LLM
                   │
                   ▼
          Response to Client

────────────────────────────────

Telemetry Flow

Gateway
      ─────────►
                OTLP
      ◄─────────

AI Engine
      ─────────►

                ▼

             SigNoz
      (Traces • Metrics • Logs)

Screenshots

Dashboard Overview

Docker Containers

Yaatra System Architecture

SigNoz Deployment

OpenTelemetry Implementation

What I'd Improve Next

Now that end-to-end observability is working, there are several improvements I'd like to make.

Instrument the React frontend for browser tracing.
Add structured logging alongside traces.
Configure alerts for high latency and increased error rates.
Track business metrics such as itinerary generation time.
Monitor AI token usage.
Measure user experience from browser interaction to AI response.

Final Thoughts

Before this project, I believed observability was primarily a production concern.

After implementing OpenTelemetry and SigNoz, I realized it's equally valuable during development.

Instead of relying on scattered console logs, I can now:

Follow every request across the application.
Identify bottlenecks within seconds.
Understand how each service contributes to response time.
Debug distributed systems with confidence.

The biggest takeaway wasn't learning another tool.

It was learning a completely new way to think about debugging.

Modern applications are distributed.

Our debugging approach should be distributed too.

If you're building AI applications, microservices, or any system where requests travel across multiple components, I highly recommend investing in observability early.

It will save countless hours of debugging and provide confidence that every request tells a complete story.

Thanks for reading!

References

OpenTelemetry Documentation
SigNoz Documentation
YouTube Tutorials
ChatGPT (for brainstorming, explanations, and documentation assistance)

DEV Community: Abhishek Ugare