You might be correct. But my understanding for best practice with Kafka is to specifically commit offsets when you're certain you've finished processing a message.
For example, if Sarama auto commits the offset but I've failed to persist the message, I'll have to manually seek the missed offset. However, if I don't commit until I know I've persisted it, it will be replayed to the consumer by Kafka within the topic's retention period. I've worked on a couple Kafka implementations and this seems to be how it's done.
Would like to know if that's actually the best way to go though
For further actions, you may consider blocking this person and/or reporting abuse
We're a place where coders share, stay up-to-date and grow their careers.
Hi :) As I know sarama has offset auto commit by default, so you don't have to do consumer.MarkOffset by yourself. Or am I wrong? :)
You might be correct. But my understanding for best practice with Kafka is to specifically commit offsets when you're certain you've finished processing a message.
For example, if Sarama auto commits the offset but I've failed to persist the message, I'll have to manually seek the missed offset. However, if I don't commit until I know I've persisted it, it will be replayed to the consumer by Kafka within the topic's retention period. I've worked on a couple Kafka implementations and this seems to be how it's done.
Would like to know if that's actually the best way to go though