TL;DR — Our JS OpenTelemetry plugin for
kafkajs, is available here.
This article describes this plugin for the
kafkajs package and our thought process behind it.
OpenTelemetry is a CNCF project, which, among other things, enables the collection of distributed traces.
At Aspecto, we use OpenTelemetry at the core of our product.
While implementing it in our backend, we found a few plugins that were missing, especially when dealing with asynchronous communication.
One of them was kafkaJS.
We took this opportunity to give back to the community and developed it ourselves.
This plugin allows you to track all Kafka interactions in your collected traces, which means, you get a more comprehensive view of your application behavior when using Kafka as a message broker.
kafkajs plugin captures
consumer operations and creates spans according to the semantic conventions for Messaging Systems.
Each message being produced and consumed is represented by a span with attributes such as
Context is propagated from producers to consumers. When a message is sent to Kafka, the trace will reveal which services consume it and what other cascading operations happen down the pipe.
Batch operations can aggregate multiple messages into a single batch and receive\process them together. This is handled in the plugin according to the specification.
The plugin can be extended with hooks, which enable users to run custom logic for adding span attributes depending on the Kafka message.
The above screenshot shows an example of a producer application named
kafka-producer, that exposes an HTTP endpoint (first line), route it in express (second line), and produce two messages to a Kafka topic named
test, which are then consumed by another application called
As mentioned above, kafkajs was one of the missing plugins we found and as you are reading this we are working to add more plugins.
Feel free to reach out to us with any questions as we are very much invested in OpenTelemetry and the OpenTelementry community.
Top comments (0)