Instrumenting Your Node.js Apps with OpenTelemetry

When I first joined New Relic, I really didn't understand the importance of observability, because I came from a frontend background. As I began learning about why observability was valuable for developers, I started digging deeper into the open source ecosystem and learning what makes it possible for modern apps to maintain uptime. I learned more about OpenTelemetry, a popular open source tool for monitoring your apps and websites, but it was intimidating because I couldn't find any introductory tutorials online guiding me through the process of instrumentation.

It wasn't until I began instrumenting my own apps using the OpenTelemetry documentation that I realized how easy it was to get started. I collaborated with freeCodeCamp.org to create a beginner-friendly resource for anyone to begin instrumenting apps with OpenTelemetry. I worked with an amazing technical content creator named Ania Kubów to bring this one-hour video course to life. This course teaches how to use OpenTelemetry, including microservices, observability, tracing, and more.

Instrumenting your Node.js apps with OpenTelemetry

As systems become increasingly complex, it’s increasingly important to get visibility into the inner workings of systems to increase performance and reliability. Distributed tracing shows how each request passes through the application, giving developers context to resolve incidents, showing what parts of their system are slow or broken.

A single trace shows the path a request makes, from the browser or mobile device down to the database. By looking at traces as a whole, developers can quickly discover which parts of their application is having the biggest impact on performance as it affects your users’ experiences.

That’s pretty abstract, right? So let’s zero in on a specific example to help clarify things. We’ll use OpenTelemetry to generate and view traces from a small sample application.

Spinning up our Movies App

We have written a simple application consisting of two microservices, movies and dashboard. The movies service provides the name of movies and their genre in JSON format, while the dashboard service returns the results from the movies service.

👉 Clone the repo

To spin up the app, run

$ npm i
$ node dashboard.js
$ node movies.js

Notice the variable delay, built into the movies microservice that causes random delays returning the JSON.

const express = require('express')
const app = express()
const port = 3000

app.get('/movies', async function (req, res) {
   res.type('json')
+  var delay = Math.floor( ( Math.random() * 2000 ) + 100);
+  setTimeout((() => {
      res.send(({movies: [
         { name: 'Jaws', genre: 'Thriller'},
         { name: 'Annie', genre: 'Family'},
         { name: 'Jurassic Park', genre: 'Action'},
      ]}))
+  }), delay)
})

Tracing HTTP Requests with Open Telemetry

OpenTelemetry traces incoming and outgoing HTTP requests by attaching IDs. To do this, we need to

• Instantiate a trace provider to get data flowing.
• Configure that trace provider with an exporter to send telemetry data to another system where you can view, store, and analyze it.
• Install OpenTelemetry plugins to instrument specific node module(s) to automatically instrument various frameworks

You need to have Docker on your machine to run a Zipkin instance. If you don't have Docker yet, it's easy to install. As for Zipkin, it's an open-source distributed tracing system created by Twitter that helps gather timing data needed to troubleshoot latency problems in service architectures. The OpenZipkin volunteer organization currently runs it. Finally, if you want to export your OpenTelemetry data to New Relic in Step 4, sign up to analyze, store, and use your telemetry data for free, forever.

Step 1: Create our trace provider and configuring it with an exporter

To create a trace provider, you need to install the following tool:

$ npm install @opentelemetry/node

OpenTelemetry auto instrumentation package for NodeJS

The @opentelemetry/node module provides auto-instrumentation for Node.js applications, which automatically identifies frameworks (Express), common protocols (HTTP), databases, and other libraries within your application. This module uses other community-contributed plugins to automatically instrument your application to automatically produce spans and provide end-to-end tracing with just a few lines of code.

OpenTelemetry Plugins

Install the plugins:

$ npm install @opentelemetry/plugin-http
$ npm install @opentelemetry/plugin-express

When NodeJS’s HTTP module handles any API requests, the @opentelemetry/plugin-http plugin generates trace data. The @opentelemetry/plugin-express plugin generates trace data from requests sent through the Express framework.

Step 2: Adding the Trace Provider and the Span Processor

After tracers are implemented into applications, they record timing and metadata about operations that take place (for example, when a web server records exactly when it receives a request, and when it sends a response).

Add this code snippet to

• create a trace provider
• adds a span processor to the trace provider

This code gets data from your local application and prints it to the terminal:

const { NodeTracerProvider } = require('@opentelemetry/node');
const { ConsoleSpanExporter, SimpleSpanProcessor } = require('@opentelemetry/tracing');

const provider = new NodeTracerProvider();
const consoleExporter = new ConsoleSpanExporter();
const spanProcessor = new SimpleSpanProcessor(consoleExporter);
provider.addSpanProcessor(spanProcessor);
provider.register()

If you want to learn more about this code, check out the OpenTelemetry docs on tracers.

Once we add this code snippet, whenever we reload http://localhost:3001/dashboard, we should get something like this - beautiful things on the terminal.

Step 3: Use Docker to install Zipkin and start tracing your application

You instrumented OpenTelemetry in the previous step. Now you move the data that you collected to a running Zipkin instance.

Let's spin up a Zipkin instance with the Docker Hub Image

$ docker run -d -p 9411:9411 openzipkin/zipkin

and you’ll have a Zipkin instance up and running. You’ll be able to load it by pointing your web browser to http://localhost:9411. You’ll see something like this

Exporting to Zipkin

Although neat, spans in a terminal window are a poor way to gain visibility into a service. You’re not going to want to scroll through JSON data in your terminal. Instead, it’s a lot easier to see a visualization in a dashboard. Let's work on that now. In the previous step, we added a console exporter to the system. Now you ship this data to Zipkin.

In this code snippet, we are instantiating a Zipkin exporter, and then adding it to the trace provider.

const { NodeTracerProvider } = require('@opentelemetry/node')
const { ConsoleSpanExporter, SimpleSpanProcessor } = require('@opentelemetry/tracing')
+ const { ZipkinExporter } = require('@opentelemetry/exporter-zipkin')
const provider = new NodeTracerProvider()
const consoleExporter = new ConsoleSpanExporter()
const spanProcessor = new SimpleSpanProcessor(consoleExporter)
provider.addSpanProcessor(spanProcessor)
provider.register()

+ const zipkinExporter = new ZipkinExporter({
+  url: 'http://localhost:9411/api/v2/spans',
+  serviceName: 'movies-service'
})

+ const zipkinProcessor = new SimpleSpanProcessor(zipkinExporter)
+ provider.addSpanProcessor(zipkinProcessor)

After you make these changes, let's visit our Zipkin instance at localhost:9411, start our application back up and request some URLs.

Step 4: Using the OpenTelemetry Collector to export the data into New Relic

What happens if we want to send the OpenTelemetry data to another backend where you didn't have to manage all of your own telemetry data?

Well, the amazing contributors to OpenTelemetry have come up with a solution to fix this!

The OpenTelemetry Collector is a way for developers to receive, process, and export telemetry data to multiple backends. This collector acts as the intermediary, getting data from the instrumentation and sending it to multiple backends to store, process, and analyze the data.

It supports multiple open source observability data formats like Zipkin, Jaeger, Prometheus, and Fluent Bit, sending it to one or more open source or commercial backends.

New Relic

New Relic is a platform for you to analyze, store, and use your telemetry data for Free, forever. Sign up now!

Configuring the OpenTelemetry Collector with New Relic

Clone the OpenTelemetry Collector with New Relic Exporter and spin up the docker container, making sure to export the New Relic API key.

To get a key, go to the New Relic one dashboard and choose API keys from the dropdown menu in the upper right.

Then, from the API keys window, click the Create a key button.

When creating the key, make sure you choose the Ingest - License key type. Then click Create a key to generate the key.

After you have an API key, you need to replace <INSERT-API-KEY-HERE> in the code snippet below with your API key.

export NEW_RELIC_API_KEY=<INSERT-API-KEY-HERE>
docker-compose -f docker-compose.yaml up

💡 Make sure to change the reporting URL from http://localhost:9411/api/v2/spans to http://localhost:9411/ in both dashboard.js and movies.js

const zipkinExporter = new ZipkinExporter({
- url: 'http://localhost:9411/api/v2/spans',
+ url: 'http://localhost:9411',
  serviceName: 'movies-service'
})

Step 5: Look at your ✨ beautiful data ✨

Navigate to the "Explorer" tab on New Relic One.

When you click on the service, you should be able to see some ✨beautiful✨ traces!

The trace in the dashboard is transmitting data about the random delay that was added to the API calls:

Final Thoughts

Instrumenting your app with Open Telemetry makes it easy to figure out what is going wrong when parts of your application is slow, broken, or both. With the collector, you can forward your data anywhere, so you are never locked into a vendor. You can choose to spin up an open source backend, use a proprietary backend like New Relic, or just roll your own backend! Whatever you choose, I wish you well you in journey to instrument EVERYTHING!

Next Steps

You can try out New Relic One with OpenTelemetry by signing up for our always free tier today.

To learn more about OpenTelemetry, look for our upcoming Understand OpenTelemetry blog series.

Comments

[Manny]

@lazyplatypus 👋 Thanks for making this post!
I am curious, does this still hold true in 2023?

On the OTel Quick Start Page for Node.js they have the following:

/*instrumentation.ts*/
import { NodeSDK } from '@opentelemetry/sdk-node';
import { ConsoleSpanExporter } from '@opentelemetry/sdk-trace-node';
import { getNodeAutoInstrumentations } from '@opentelemetry/auto-instrumentations-node';
import { PeriodicExportingMetricReader, ConsoleMetricExporter } from '@opentelemetry/sdk-metrics';

const sdk = new NodeSDK({
  traceExporter: new ConsoleSpanExporter(),
  metricReader: new PeriodicExportingMetricReader({
    exporter: new ConsoleMetricExporter()
  }),
  instrumentations: [getNodeAutoInstrumentations()]
});

sdk
  .start()

Which differs slightly from the tracer.js/ts file pattern you shared above.

I worked with OTel-JS in earlier iterations and it required more configurations as shared by SumoLogic's Guide

'use strict';

const opentelemetry = require('@opentelemetry/api');
const { NodeTracerProvider } = require('@opentelemetry/sdk-trace-node');
const { BatchSpanProcessor } = require("@opentelemetry/sdk-trace-base");
const { OTLPTraceExporter } = require('@opentelemetry/exporter-trace-otlp-proto');
const { registerInstrumentations } = require('@opentelemetry/instrumentation');
const { HttpInstrumentation } = require('@opentelemetry/instrumentation-http');
const { Resource } = require('@opentelemetry/resources');


module.exports = () => {
 const resources = new Resource({
    'service.name': 'YOUR_SERVICE_NAME',
    'application': 'YOUR_APPLICATION_NAME',
    //'ANY_OTHER_ATTRIBUTE_KEY': 'ANY_OTHER_ATTRIBUTE_VALUE',
 });

 const provider = new NodeTracerProvider({ resource: resources });

 const exporterOptions = {
  url: 'http://collection-sumologic-otelcol.sumologic:4318/v1/traces',
 }

 const exporter = new OTLPTraceExporter(exporterOptions);
 provider.addSpanProcessor(new BatchSpanProcessor(exporter));
 provider.register();

 registerInstrumentations({
  instrumentations: [
    new HttpInstrumentation(),
  ],
 })
 return opentelemetry.trace.getTracer("instrumentation-example");
}

And I am trying to decipher what holds true at this point.
I hadn't worked with OTel-JS since 2021 and during and since then there's been quite a few updates/renames/repatterning and I am questioning what I use to do before and what is the current landscape.

I'd love to hear from you!

[Shreya]

This was awesome!!

[bruce-x-wu]

Awesome!

[Maria Lucena]

Thanks for breaking this down!