DEV Community

Cover image for Install Kafa to Raspberry Pi 4
Nimrod Resulta
Nimrod Resulta

Posted on • Updated on

Install Kafa to Raspberry Pi 4

I wanted to run kafka on my raspberry pi 4 as i wanted to do a POC on an IoT project. Here are the steps on how to install it. I use SSH to login to my Rpi.

1. Install Java:

  • Run the following command to install it:
$ sudo apt update
$ sudo apt install default-jdk
Enter fullscreen mode Exit fullscreen mode
  • Check version to test if the installation is successful. It should show you the version installed as shown below.
$ java --version
openjdk 11.0.8 2020-07-14
OpenJDK Runtime Environment (build 11.0.8+10-post-Raspbian-1deb10u1)
OpenJDK Server VM (build 11.0.8+10-post-Raspbian-1deb10u1, mixed mode)
Enter fullscreen mode Exit fullscreen mode

2. Install Kafka:

$ wget https://apachemirror.sg.wuchna.com/kafka/2.6.0/kafka_2.12-2.6.0.tgz
Enter fullscreen mode Exit fullscreen mode
  • Extract Kafka. Using the tar command. Once extracted, go inside the directory and create /data folder with /kafka and /zookeeper subfolders.
$ tar -xzf kafka_2.12-2.6.0.tgz
$ cd kafka_2.12-2.6.0
~/kafka_2.12-2.6.0 $ mkdir data
~/kafka_2.12-2.6.0/data $ mkdir kafka
~/kafka_2.12-2.6.0/data $ mkdir zookeeper
Enter fullscreen mode Exit fullscreen mode
  • Edit config/zookeeper.properties
$ sudo nano kafka_2.12-2.6.0/config/zookeeper.properties
Enter fullscreen mode Exit fullscreen mode

Find and change dataDir value to /home/pi/kafka_2.12-2.6.0/data/zookeeper.

dataDir=/home/pi/kafka_2.12-2.6.0/data/zookeeper

Press Ctrl+X then Y to save it.

  • Edit config/server.properties
$ sudo nano kafka_2.12-2.6.0/config/server.properties
Enter fullscreen mode Exit fullscreen mode

Find and Change listeners value to PLAINTEXT://{your_ip_address}:9092. Ensure to replace {your_ip_address} with your device ip address.

listeners=PLAINTEXT://{your_ip_address}:9092

Next is to locate and update log.dirs value to /home/pi/kafka_2.12-2.6.0/data/kafka.

log.dirs=/home/pi/kafka_2.12-2.6.0/data/kafka

3. Test Kafka

  • Start Zookeeper
~/kafka_2.12-2.6.0 $ bin/zookeeper-server-start.sh config/zookeeper.properties
Enter fullscreen mode Exit fullscreen mode
  • Start Kafka
~/kafka_2.12-2.6.0 $ bin/kafka-server-start.sh config/server.properties
Enter fullscreen mode Exit fullscreen mode
  • Create a topic
~/kafka_2.12-2.6.0 $ bin/kafka-topics.sh --create --bootstrap-server {your_ip_address}:9092 --replication-factor 1 --partitions 1 --topic TestTopic
Enter fullscreen mode Exit fullscreen mode

It will show you the message below as reponse for topic created.

Created topic TestTopic.

  • Check for the created topic
~/kafka_2.12-2.6.0 $ bin/kafka-topics.sh --list --bootstrap-server {your_ip_address}:9092
Enter fullscreen mode Exit fullscreen mode

This will show you the newly created topic

TestTopic

  • Start Kafka Producer
~/kafka_2.12-2.6.0 $ bin/kafka-console-producer.sh --broker-list {your_ip_address}:9092 --topic TestTopic
Enter fullscreen mode Exit fullscreen mode
  • Start Kafka Consumer
~/kafka_2.12-2.6.0 $ bin/kafka-console-consumer.sh --bootstrap-server {your_ip_address}:9092 --topic TestTopic
Enter fullscreen mode Exit fullscreen mode

Once both Producer and Consumer is up and running start typing on the producer window. Each message should show in the consumer window.

Alt Text

Discussion (2)

Collapse
moonraker727 profile image
Alex

Hello from 2021. I created an account here just to say thanks... but also Start Kafka Consumer should listen to the same topic as the producer, you've got TestTopic and NewTopic. Thanks again, this really helped.

Collapse
nimrodresulta profile image
Nimrod Resulta Author • Edited

Hey, thanks man, appreciate it. I have updated my post.