DEV Community

Cover image for Installing node-rdkafka on M1 for use with SASL
Lucia Cerchie
Lucia Cerchie

Posted on

16

Installing node-rdkafka on M1 for use with SASL

If you're using Kafka in a Node.js app, it's likely that you'll need node-rdkafka. This is a library that wraps the librdkafka library and makes it available in Node.js. According to the project's README, "All the complexity of balancing writes across partitions and managing (possibly ever-changing) brokers should be encapsulated in the library."

Installation steps may vary depending on the machine you're using, and whether you need SASL/SSL (Simple Authentication and Security Layer) support. You might need this, for example, if you're connecting to Confluent Cloud, as it is a required protocol to protect resources (see docs).

Here's a short, step by step guide on installing the node-rdkafka client specifically on M1 (the first chip released by Apple specifically for Mac, article here).

Currently, as of September 30 '22, you have to do it manually. You can find the reason why in the confluent-kafka Python client README.md:

NOTE: The pre-built Linux wheels do NOT contain SASL Kerberos/GSSAPI support. If you need SASL Kerberos/GSSAPI support you must install librdkafka and its dependencies using the repositories below and then build confluent-kafka using the instructions in the "Install from source" section below.

Here are the steps you'll need to take for SSL/SASL support (run in your home directory):

git clone https://github.com/edenhill/librdkafka.git
cd ./librdkafka
./configure --install-deps --source-deps-only
make
sudo make install
Enter fullscreen mode Exit fullscreen mode

Then,

  1. Use homebrew to install openssl.

  2. Navigate out of ./librdkafka and into your Node.js project directory.

  3. Export these variables with these commands, to make sure that the path to openssl is correct:

    export CPPFLAGS=-I/opt/homebrew/opt/openssl@3/include
    export LDFLAGS=-L/opt/homebrew/opt/openssl@3/lib
    
  4. install openssl with npm install openssl

  5. Next, install node-rdkafka -- now you've got SSL support for your Node.js/Kafka project!

Hopefully this helps! If you're still running into issues, you can post a request for help in Confluent Community.

Billboard image

The Next Generation Developer Platform

Coherence is the first Platform-as-a-Service you can control. Unlike "black-box" platforms that are opinionated about the infra you can deploy, Coherence is powered by CNC, the open-source IaC framework, which offers limitless customization.

Learn more

Top comments (0)

Sentry image

See why 4M developers consider Sentry, “not bad.”

Fixing code doesn’t have to be the worst part of your day. Learn how Sentry can help.

Learn more

👋 Kindness is contagious

Dive into an ocean of knowledge with this thought-provoking post, revered deeply within the supportive DEV Community. Developers of all levels are welcome to join and enhance our collective intelligence.

Saying a simple "thank you" can brighten someone's day. Share your gratitude in the comments below!

On DEV, sharing ideas eases our path and fortifies our community connections. Found this helpful? Sending a quick thanks to the author can be profoundly valued.

Okay