If you are using apache kafka, you know that local deployment required a lot of things. You need to download zookeeper, apache kafka, edit configs and then run them. And, of cause, it takes time. The detailed description you can find in this article https://dev.to/lehauchicha/spring-boot-application-with-apache-kafka-299n. But is it possible to speed up the local deployment process of apache kafka ?
The answer is yes!
In this article we will deploy apache kafka locally using docker-compose. All you need is download and install docker from official site: https://www.docker.com/get-started
Lets create docker-compose.yml file:
version: '2' services: zookeeper: image: confluentinc/cp-zookeeper:latest environment: ZOOKEEPER_CLIENT_PORT: 2181 ZOOKEEPER_TICK_TIME: 2000 ports: - 22181:2181 kafka: image: confluentinc/cp-kafka:latest depends_on: - zookeeper ports: - 29092:29092 environment: KAFKA_BROKER_ID: 1 KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181 KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka:9092,PLAINTEXT_HOST://localhost:29092 KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT KAFKA_INTER_BROKER_LISTENER_NAME: PLAINTEXT KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
As you can see here we specify images for zookeeper and kafka with ports. That's it!
Lets run kafka, using docker-compose:
docker-compose up -d
Now kafka ready for usage in your applications!
One more thing, for convenient configuring kafka topics, I would like to suggest UI Tool for Kafka - Offset Explorer, which you can find by link https://kafkatool.com/download.html.
For connection to deployed kafka you need install tool and add connection parameters here on properties tab:
and here on advanced tab:
That's it. Thank you for reading!
Top comments (7)
Another UI tool which is completely web based.
This is the best one🙌
Looks interesting, thank you!
Why don't you try upstash? The free version will suffice.
I just don't heard about upstash, so thank you for mention, I will try)
Can we connect kafka connector with any db to read or write the data?(Not via producer
Yes, sure. Here you can read some information about connectors: docs.confluent.io/platform/current....
You can configure source connector(for example jdbs source connector) and sink connector(For example, elastic search sink connector)