I was reading about offset retention and was mentally prepared for losing consumer offsets when the consumer has been offline for 7+ days. But I realized that it’s 7+ days since the last update of offsets. This means if a topic did not receive any new message for 7+ days, there is no reason to update the offsets and they will be lost as well.
WTF, Kafka? How about the Principle of least astonishment?
It’s So Over.
Ok, ok. We had changes for consumer groups but not the simple consumers. Then we had updates for consumer groups to delete some offsets. Which lead to a bug during rebalance that made my brain spin for almost a month.
Top comments (0)