In the the initial post of the Event-driven microservices with Kafka series (see here), I talked about the advantages of using event-driven communi...
For further actions, you may consider blocking this person and/or reporting abuse
Kafka's API provides poll(timer) to read messages from queue; let's call that a pull. Do websockets rely on this method (under the covers)? If it's truly event driven then don't we need push semantics exposed within Kafka's APIs to get the 'real-time' eventing that's being shown here? Make sense?
Thanks a bunch!
Hi! Sorry for such a late response.
Your comment raises a good point, it is true that Kafka does not provide something such as JMS' MessageListener or WebSockets'
MessageHandler.Whole and hence, one needs to explicitly perform the periodic pulling in order to retrieve messages from Kafka.
But this is implemented this way because of performance reasons (so that the Kafka broker does not need to keep track of what messages have been received by which client) and one important detail to note is that a single pull/poll operation may grab more than one message from more than one Kafka topic in one go.
Also, the polling interval is supposed to be very short when querying Kafka, much shorter than the traditional polling we see being used in other systems whose original use case was synchronous communication (e.g. HTTP request/response).
Hence, because of the two points above (the ability to receive messages in bulk and the short polling interval), Kafka is considered a real-time system, despite not providing explicit push (asynchronous receiving) semantics.
I hope this makes sense!
Is there any way to send messages directly from KTable to websocket?
I have data that are constantly updating. I keep them in Kafka compacted topic. When new websocket customer arrive I would like to send him a current state of my data, and then I want to send him automatically every change of the data. I believe that the best way it would be to somehow bind KTable (or GlobalKTable, or some state store maybe) with websocket. Is something like that possible?
Hi, sorry for the late response.
Well, since a KTable is basically (please allow me the rough simplification) a view based on a Kafka topic, I guess what you are really asking is whether out-of-the-box integration with Kafka using Websockets exists.
And unfortunately the answer is no, it does not exist.
As per this post, it looks like they experimented with the idea at Confluent, but I could not find anything more about websockets in their blog nor documentation.
I also found this (random) repository on Github but it is 5 years old and looks abandoned.
I even considered to try building a Kafka connector for Websockets myself but I never started it.
Is the github link valid? I am not able to check the source code?
Hi, sorry for the late response.
Yes, the Github link works --> github.com/VictorGil/transfers_web...
It is a publicly accessible repository and in the
README.mdfile there are links to other related repos.