Building a Fullstack E-Commerce Platform with Node.js, React, and Grafana Observability

senthilkumar sugumar — Tue, 24 Jun 2025 10:52:42 +0000

In this post, I’ll walk you through the design and implementation of a full-fledged e-commerce platform built using Node.js microservices and a ReactJS frontend, complete with distributed tracing and observability via Grafana Tempo.

🛠️ Tech Stack

Backend (Microservices)

Node.js + Express for individual microservices
MongoDB for persistent storage
Redis for session and cart caching
Kafka for inter-service communication
OpenTelemetry for distributed tracing

Frontend

ReactJS
Redux Toolkit + RTK Query for state management and API calls

Observability

OpenTelemetry SDK for instrumentation
Grafana Tempo for tracing
Grafana Dashboard for visualization

🧩 Microservices Architecture

We follow a modular architecture using Docker Compose:

1. Auth Service

Handles user signup, signin, and signout
JWT-based authentication
Redis for refresh token/session store

2. User Service

Stores and updates user profiles
Triggered by Kafka event user.registered

3. Product Service

Manages product catalog with image uploads
Stores metadata in MongoDB, images in AWS S3
Emits events like product.created

4. Cart Service

Stores temporary carts in Redis
Emits cart.checkedout event

5. Order Service

Consumes cart.checkedout, processes orders
Persists in MongoDB

6. Frontend

Built with React
Login/Register, product listing, cart view, order history
Uses Redux slices and RTK Query

🔎 Observability with Grafana Tempo

To make our system observable, we integrated OpenTelemetry into each Node.js service.

Instrumentation Includes:

@opentelemetry/sdk-node
@opentelemetry/instrumentation-express
@opentelemetry/exporter-otlp-http

Export Path

Service → OTEL SDK → OTEL Collector → Tempo → Grafana

Grafana Dashboard

View traces by service, span duration, and errors
Filter by endpoint or user action

Docker Compose

We use two Compose files:

kafka-docker-compose.yml: for app services
docker-compose.yml: for Grafana, Tempo, OTEL Collector

To run everything together:

docker-compose -f kafka-docker-compose.yml -f docker-compose.yml up --build

⚙️ Development Flow

Build and start the backend microservices
Start the frontend React app
Add OTEL instrumentation to each backend
Verify trace data in Grafana after making API requests

🧵 Repository

https://github.com/senthilkumar3282/base-setup-nodejs-microservices-reactjs

✅ Conclusion

This project demonstrates how to build a modular and scalable e-commerce app using Node.js and React, while maintaining full observability using OpenTelemetry and Grafana Tempo.

Observability Journey with OpenTelemetry

senthilkumar sugumar — Fri, 06 Jun 2025 16:17:03 +0000

By Senthilkumar Sugumar

As a full-stack developer working on distributed systems and cloud-native applications, I recently embarked on a journey to master OpenTelemetry — an open-source observability framework that unifies logs, metrics, and traces in a vendor-neutral format.

From local Node.js development to Kubernetes deployment on Amazon EKS, and from exporting telemetry to Grafana Cloud and Elastic Cloud, I explored how OpenTelemetry can provide end-to-end observability across an entire system.

🔍 What is OpenTelemetry?

OpenTelemetry (OTel) is a standardized framework for instrumenting, generating, collecting, and exporting telemetry data:

Traces: Follow the flow of requests across microservices
Metrics: Monitor application and system health
Logs: Capture contextual outputs and events

🛠️ Manual vs Automatic Instrumentation

🔧 Manual Instrumentation

Using SDKs like @opentelemetry/sdk-node, developers explicitly define spans, metrics, and logs:

const span = tracer.startSpan('get-user');
// Your business logic here
span.end();

✅ Pros: More control over what gets tracked
❌ Cons: More boilerplate and maintenance

⚙️ Automatic Instrumentation

Use auto-instrumentation tools for popular frameworks like Express:

node --require @opentelemetry/auto-instrumentations-node/register app.js

✅ Pros: Quick setup, zero code changes
❌ Cons: Limited customization options

🔁 Telemetry Lifecycle

Layer	Method	Telemetry Types
Application	SDKs, Auto-instrumentation	Traces, Metrics, Logs
Host/Node	OTEL Agent (hostmetrics, filelog)	CPU, Memory, Disk, Logs
Kubernetes	OTEL Operator, Sidecars	Pod, Node, Container
Cloud Providers	AWS, GCP, Azure Receivers	Cloud-specific metrics

All of this data is centralized and processed using the OpenTelemetry Collector.

🧰 OpenTelemetry Collector: The Heart of Observability

The OTEL Collector receives, processes, and exports telemetry data.

Component	Role
Receivers	Ingest telemetry (OTLP, Prometheus, Jaeger, etc.)
Processors	Enrich/filter data (batching, resource attribution)
Exporters	Send to backends (Grafana, Elastic, etc.)
Extensions	Add health checks, authentication, etc.

📦 Collector Distributions

Type	Description
Core	Basic exporters like OTLP, Prometheus, Jaeger
Contrib	Extras: hostmetrics, filelog, Kafka, AWS, Elastic, etc.

📈 My Learning Path – Real-World Projects

🧪 1. Basic OTEL SDK with Node.js

🔗 GitHub: opentelemetry-nodejs

Used @opentelemetry/sdk-node for manual instrumentation
Exported traces to Jaeger, metrics to Prometheus
Local OTEL Collector setup

✅ Takeaway: Learned core SDK usage, span creation, and OTLP export setup

🌐 2. Grafana Cloud Integration with Node.js

🔗 GitHub: OpenTelemetry-Node.js-App-with-Grafana-Cloud-Integration

Exported traces to Grafana Tempo, metrics to Grafana Cloud
Used OTLP HTTP exporter with basic auth
Configured OTEL Collector for secure pipeline

✅ Takeaway: Experience with remote observability and secure telemetry flows

📦 3. Full Observability with Elastic APM + ELK

🔗 GitHub: openTelemetry-elastic-APM-nodejs

Sent traces to Elastic APM
Used Logstash for log ingestion into Elasticsearch
Visualized telemetry in Kibana

✅ Takeaway: Full-stack telemetry system using Elastic APM + ELK

☁️ 4. OpenTelemetry on Kubernetes with FastAPI

🔗 GitHub: fastapi-otel-eks

Deployed FastAPI on Amazon EKS
Used OTEL SDK + opentelemetry-instrument
Enabled auto-instrumentation with OTEL Kubernetes Operator
Exported to Elastic Cloud via OTLP and API key

✅ Takeaway: Production-grade observability in Kubernetes

🧩 Common Challenges I Faced

YAML syntax errors and pipeline misconfigurations in the Collector
Missing telemetry due to incorrect API keys or OTLP endpoints
While setting up Grafana, Prometheus, Jaeger, and the ELK stack locally with Docker for a Node.js app, I faced several dependency issues and ELK authentication problems—especially with the APM setup.
However, when I switched to cloud services, both auto and manual instrumentation worked smoothly with Elastic and Grafana Cloud using Docker-based Node.js applications.
Later, I explored OpenTelemetry on Kubernetes by deploying a Python (FastAPI) application integrated with Elastic Cloud. Initially, I was able to collect server-level metrics and logs using Elastic Agents, but I didn’t receive application-level traces. To fix this, I added OpenTelemetry API-based instrumentation in the Python app, which enabled me to capture application-level metrics, traces, and logs successfully.

🧵 Repository Recap

Use Case	GitHub Repo
Basic Node.js OTEL SDK	`opentelemetry-nodejs`
Grafana Cloud Integration	`OpenTelemetry-Node.js-App-with-Grafana-Cloud-Integration`
Elastic APM + Logs	`openTelemetry-elastic-APM-nodejs`
FastAPI + EKS + Elastic	`fastapi-otel-eks`

💡 Final Thoughts

OpenTelemetry has transformed how I monitor, debug, and operate distributed apps. Whether it's a monolith or microservices on Kubernetes, OTEL gives me the visibility I need — across any stack.

Observability isn’t just about tools — it’s about empowering developers to deeply understand their systems.

📫 Let’s Connect

🔗 GitHub
💼 LinkedIn

Thanks for reading! 🌟

DEV Community: senthilkumar sugumar