Great article! You've given really easy to understand example. However, I was wondering something I didn't see you mention. In the first case, when you don't use the polling principle upon failing you see which record exactly has failed. Is it true that with the polling you can only get the batch of records in which a failing record appeared and not the single record itself? And another question, could you explain what would happen if for example the 3 of 10 records in the batch fails? Does this fail that the whole batch of 10 records?
with the polling you can get the single record by setting the config max.poll.records to 1, but it's not good for performances.
With polling you get the a batch of messages, but you can process it one at a time, so, it's possible to know which is failing, the advice is to avoid commit when at least one message fails.
So yes, with this approach one or more fail = all batch fails.
For further actions, you may consider blocking this person and/or reporting abuse
We're a place where coders share, stay up-to-date and grow their careers.
Great article! You've given really easy to understand example. However, I was wondering something I didn't see you mention. In the first case, when you don't use the polling principle upon failing you see which record exactly has failed. Is it true that with the polling you can only get the batch of records in which a failing record appeared and not the single record itself? And another question, could you explain what would happen if for example the 3 of 10 records in the batch fails? Does this fail that the whole batch of 10 records?
with the polling you can get the single record by setting the config
max.poll.recordsto 1, but it's not good for performances.With polling you get the a batch of messages, but you can process it one at a time, so, it's possible to know which is failing, the advice is to avoid commit when at least one message fails.
So yes, with this approach one or more fail = all batch fails.