The KafkaProducer configuration has a option(retries property) for doing automatic retries in case of RetriableExceptions like NoLeaderForPartition, NetworkException, TimeoutException etc.
The retries property has the default value as Integer.MAX as per https://kafka.apache.org/documentation/#producerconfigs.
I was using spring-kafka 2.2.8-RELEASE in the application and tried to test the retry behavior by bringing one of the broker down from the Kafka cluster. It thrown the NoLeaderForPartition exception as the leader broker for a partition went down and it took 1-2 seconds for a new leader to be assigned to that partition.
As per the documentation, the producer should’ve retried on this exception, but to the surprise it didn’t happen. Digging further, found out that the spring-kafka 2.2.8-RELEASE (Aug 2019, latest in central repo) is using 2.0.1 version of kafka-clients library whereas the kafka-clients library’s latest version is 2.3.0. In kafka-clients 2.0.1 version, the default value for retries is 0.
Learning I take away from this is that the latest spring library version need not be using the latest version of the actual client library. Also we need to understand from which version of the client library, the documentation applies to.
Top comments (0)