Kafka's API provides poll(timer) to read messages from queue; let's call that a pull. Do websockets rely on this method (under the covers)? If it's truly event driven then don't we need push semantics exposed within Kafka's APIs to get the 'real-time' eventing that's being shown here? Make sense?
Hi! Sorry for such a late response.
Your comment raises a good point, it is true that Kafka does not provide something such as JMS' MessageListener or WebSockets' MessageHandler.Whole and hence, one needs to explicitly perform the periodic pulling in order to retrieve messages from Kafka.
But this is implemented this way because of performance reasons (so that the Kafka broker does not need to keep track of what messages have been received by which client) and one important detail to note is that a single pull/poll operation may grab more than one message from more than one Kafka topic in one go.
Also, the polling interval is supposed to be very short when querying Kafka, much shorter than the traditional polling we see being used in other systems whose original use case was synchronous communication (e.g. HTTP request/response).
Hence, because of the two points above (the ability to receive messages in bulk and the short polling interval), Kafka is considered a real-time system, despite not providing explicit push (asynchronous receiving) semantics.
I hope this makes sense!
For further actions, you may consider blocking this person and/or reporting abuse
We're a place where coders share, stay up-to-date and grow their careers.
Kafka's API provides poll(timer) to read messages from queue; let's call that a pull. Do websockets rely on this method (under the covers)? If it's truly event driven then don't we need push semantics exposed within Kafka's APIs to get the 'real-time' eventing that's being shown here? Make sense?
Thanks a bunch!
Hi! Sorry for such a late response.
Your comment raises a good point, it is true that Kafka does not provide something such as JMS' MessageListener or WebSockets'
MessageHandler.Whole and hence, one needs to explicitly perform the periodic pulling in order to retrieve messages from Kafka.
But this is implemented this way because of performance reasons (so that the Kafka broker does not need to keep track of what messages have been received by which client) and one important detail to note is that a single pull/poll operation may grab more than one message from more than one Kafka topic in one go.
Also, the polling interval is supposed to be very short when querying Kafka, much shorter than the traditional polling we see being used in other systems whose original use case was synchronous communication (e.g. HTTP request/response).
Hence, because of the two points above (the ability to receive messages in bulk and the short polling interval), Kafka is considered a real-time system, despite not providing explicit push (asynchronous receiving) semantics.
I hope this makes sense!