Hello 🖐️ !
Following the mechanisms described here to send CSV files into Kafka in JSON using kcat through shell pipelines, we packed 📦 all of that in a new home-made tool named producer-file
. Here are the features :
- 💨 Sending line by line
- 💪 CSV to JSON conversion
- 🔦 Support message headers by file and/or by line
- ⚡️ Delta-changes mode (only send changes occurred after last send)
☝️ More about the shell way :


⌨️ Pipe xlsx files into/from Kafka... From cli with (k)cat 🙀
adriens for opt-nc ・ Oct 10 '22
Motivations
Using jq
+ kcat
with shell pipelines do the job well, but this doesn't fit the company's requirements today about software packaging and logging capabilities 🤷🏻♂️.
📽️ Demo
Short video demonstrating how to use this tool to send a CSV file in JSON into Kafka within a Docker environnement :
Top comments (2)
Here is a related and worhtwhile content :
Projet d'exemple d'utilisation de la lib `org.springframework.kafka:spring-kafka`
Ce projet explique comment développer un producer/consumer sans la lib interne historique au profit de celle développée et maintenue par Spring.
Projet d'exemple d'utilisation de la lib
org.springframework.kafka:spring-k...
kafka
docker
&docker-compose
(si utilisation dekafka.yml
)Avant de passer à la suite, prendre soin de consulter les ressources ci-dessous:
demo.user
Entrer dans le container
kafka
:Créer le producer :
Copier les lignes :
demo.sms
Initialiser…