DEV Community

Cover image for OpenTelemetry KafkaJS Instrumentation for Node.js
Amir Blum for Aspecto

Posted on • Originally published at Medium

6 2

OpenTelemetry KafkaJS Instrumentation for Node.js

TL;DR — Our JS OpenTelemetry plugin for kafkajs, is available here.

This article describes this plugin for the kafkajs package and our thought process behind it.

Some Background

OpenTelemetry is a CNCF project, which, among other things, enables the collection of distributed traces.

At Aspecto, we use OpenTelemetry at the core of our product.

While implementing it in our backend, we found a few plugins that were missing, especially when dealing with asynchronous communication.

One of them was kafkaJS.

We took this opportunity to give back to the community and developed it ourselves.

The Plugin

This plugin allows you to track all Kafka interactions in your collected traces, which means, you get a more comprehensive view of your application behavior when using Kafka as a message broker.

The kafkajs plugin captures producer and consumer operations and creates spans according to the semantic conventions for Messaging Systems.

  • Each message being produced and consumed is represented by a span with attributes such as messaging.destination(topic name).

  • Context is propagated from producers to consumers. When a message is sent to Kafka, the trace will reveal which services consume it and what other cascading operations happen down the pipe.

  • Batch operations can aggregate multiple messages into a single batch and receive\process them together. This is handled in the plugin according to the specification.

  • The plugin can be extended with hooks, which enable users to run custom logic for adding span attributes depending on the Kafka message.

simple-trace-example-in-Jaeger-view-Aspecto-opentelemetry-kafkajs-plugin-1024x270.png

The above screenshot shows an example of a producer application named kafka-producer, that exposes an HTTP endpoint (first line), route it in express (second line), and produce two messages to a Kafka topic named test, which are then consumed by another application called kafka-consumer .

As mentioned above, kafkajs was one of the missing plugins we found and as you are reading this we are working to add more plugins.

Feel free to reach out to us with any questions as we are very much invested in OpenTelemetry and the OpenTelementry community.

Image of Timescale

🚀 pgai Vectorizer: SQLAlchemy and LiteLLM Make Vector Search Simple

We built pgai Vectorizer to simplify embedding management for AI applications—without needing a separate database or complex infrastructure. Since launch, developers have created over 3,000 vectorizers on Timescale Cloud, with many more self-hosted.

Read more

Top comments (0)

Postmark Image

Speedy emails, satisfied customers

Are delayed transactional emails costing you user satisfaction? Postmark delivers your emails almost instantly, keeping your customers happy and connected.

Sign up