Setup Apache Kafka Environment on Windows

In this article, I will be walking you through the process of installing Apache Kafka successfully on Windows. Let's dive into it.

Why use Apache Kafka?

Apache Kafka is an open-source distributed event streaming platform used by thousands of companies for high-performance data pipelines, streaming analytics, data integration, and mission-critical applications. You can use Kafka for real-time data streaming, big data collection, and real-time data analysis. It can be used with in-memory microservices ensuring durability. It will allow you to feed events to complex event streaming systems. It is one of the most commonly used fault-tolerant messaging systems.

Prerequisites

• Download and Install the Java Runtime Environment (JRE8). This can be done in two different ways which are by:
  • Download from the Oracle Java Official Downloads Page here
  • Install using Chocolatey by pasting this script on your windows command line.

choco install jdk8

After installing, confirm your Java installation

java --version

• Download and Install Apache Zookeeper

Apache Zookeeper helps keeps track of the status of the Kafka cluster nodes and it also keeps track of Kafka topics, partitions.

Get Started with Apache Kafka

For the installation process, please follow the steps given below:

Step 1: Visit Apache Kafka Official Download Page

• Download the latest stable release of Apache Kafka Binaries - https://kafka.apache.org/downloads

Step 2: Extract Apache Folder

• Visit your downloads folder
• Extract the downloaded file and move it to any folder of your choice. For this tutorial, I recommend moving to the C:/ folder.

Step 3: Create directories for storing Kafka/zookeeper snapshots and logs

After moving to the C:/ folder ;

• Create a data folder in the Kafka directory
• Create Kafka and zookeeper directories inside the data folder

Step 4: Update configuration file for Zookeeper and Kafka Logs

• In C:/kafka, open the config folder in a text editor and update the zookeeper.properties file. Change the dataDir variable to the path where your zookeeper snapshots will be stored as shown in the image below.

• In the same config folder, update the server.properties file. Update the log.dirs variable to the path where your Kafka logs will be stored.

Step 5: Start Apache Zookeeper and Kafka

• Start Apache zookeeper, Open windows command prompt, change your directory to c:/ and run the following scripts

cd kafka/bin/windows

zookeeper-server-start.bat ../../config/zookeeper.properties

• Start Apache Kafka, open up a new command prompt

Run the following command from C:\kafka\bin\windows directory

kafka-server-start.bat  ../../config/server.properties

Apache Kafka should start running successfully on your machine 🎉🎉🎉🎉

Conclusion

Apache Kafka is now successfully installed on your machine, you can start making use of Kafka for event and data streaming. Learn more about Kafka Use cases and applications.

If you have any questions, don't hesitate to reach out me on Twitter: @adeileri