DEV Community

Cover image for Fast apache kafka local deployment
LehaUchicha
LehaUchicha

Posted on

 

Fast apache kafka local deployment

Hey, dev.to!

If you are using apache kafka, you know that local deployment required a lot of things. You need to download zookeeper, apache kafka, edit configs and then run them. And, of cause, it takes time. The detailed description you can find in this article https://dev.to/lehauchicha/spring-boot-application-with-apache-kafka-299n. But is it possible to speed up the local deployment process of apache kafka ?

The answer is yes!
In this article we will deploy apache kafka locally using docker-compose. All you need is download and install docker from official site: https://www.docker.com/get-started

Lets create docker-compose.yml file:

version: '2'
services:
  zookeeper:
    image: confluentinc/cp-zookeeper:latest
    environment:
      ZOOKEEPER_CLIENT_PORT: 2181
      ZOOKEEPER_TICK_TIME: 2000
    ports:
      - 22181:2181

  kafka:
    image: confluentinc/cp-kafka:latest
    depends_on:
      - zookeeper
    ports:
      - 29092:29092
    environment:
      KAFKA_BROKER_ID: 1
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka:9092,PLAINTEXT_HOST://localhost:29092
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT
      KAFKA_INTER_BROKER_LISTENER_NAME: PLAINTEXT
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
Enter fullscreen mode Exit fullscreen mode

As you can see here we specify images for zookeeper and kafka with ports. That's it!

Lets run kafka, using docker-compose:

docker-compose up -d
Enter fullscreen mode Exit fullscreen mode

you will see such result:
Image description

Now kafka ready for usage in your applications!

One more thing, for convenient configuring kafka topics, I would like to suggest UI Tool for Kafka - Offset Explorer, which you can find by link https://kafkatool.com/download.html.

For connection to deployed kafka you need install tool and add connection parameters here on properties tab:

Image description

and here on advanced tab:

Image description

That's it. Thank you for reading!

Top comments (7)

Collapse
 
inzagio profile image
Trym

Another UI tool which is completely web based.
github.com/obsidiandynamics/kafdrop

Collapse
 
saurabh619 profile image
Saurabh Bomble

This is the best oneπŸ™Œ

Collapse
 
lehauchicha profile image
LehaUchicha

Hey, Trym!
Looks interesting, thank you!

Collapse
 
demir profile image
Mehmet

Why don't you try upstash? The free version will suffice.

upstash.com/

Collapse
 
lehauchicha profile image
LehaUchicha

Hey, Mehmet!
I just don't heard about upstash, so thank you for mention, I will try)

Collapse
 
vivekera profile image
vivekEra

Can we connect kafka connector with any db to read or write the data?(Not via producer
)

Collapse
 
lehauchicha profile image
LehaUchicha

Yes, sure. Here you can read some information about connectors: docs.confluent.io/platform/current....
You can configure source connector(for example jdbs source connector) and sink connector(For example, elastic search sink connector)