Hi Marcos Maia,
Loved your post. I have recently started working with Kafka and docker. Your post was a saviour for me. Post is very well written.
I just want to know is it possible with the kafka image I have to write to a http api as sink. I want a sink connector and process with which we write the data from the topic to an api. Could you help me out with that?
I am an experienced Developer, Trainer, and eventual Speaker. Lifelong learner, eager to learn and share knowledge. A bit introspective. Keep coding. Be humble. Help others.
Usually for this use cases where you want to get data from kafka and send using HTTP I would most likely go for a traditional consumer where you have more control over the data you can also control offset commits back to kafka if you need delivery garantees (i.e that the endpoint calls 200 on the call) which would be simpler with a standard kafka consumer.
I have articles on kafka consumers I would recommend you to take a look at them, also you will find tutorials on building http clients which you could connect to your consumer to achieve what you want.
Best,
For further actions, you may consider blocking this person and/or reporting abuse
We're a place where coders share, stay up-to-date and grow their careers.
Hi Marcos Maia,
Loved your post. I have recently started working with Kafka and docker. Your post was a saviour for me. Post is very well written.
I just want to know is it possible with the kafka image I have to write to a http api as sink. I want a sink connector and process with which we write the data from the topic to an api. Could you help me out with that?
Hi,
Well, you could, Confluent even has a connector for that. docs.confluent.io/5.0.4/connect/ka... but unfortunatelly it's a paid connector.
Usually for this use cases where you want to get data from kafka and send using HTTP I would most likely go for a traditional consumer where you have more control over the data you can also control offset commits back to kafka if you need delivery garantees (i.e that the endpoint calls 200 on the call) which would be simpler with a standard kafka consumer.
I have articles on kafka consumers I would recommend you to take a look at them, also you will find tutorials on building http clients which you could connect to your consumer to achieve what you want.
Best,