DEV Community

Discussion on: Microservices communications. Why you should switch to message queues.

Collapse
 
aldex32 profile image
Aldo Sinanaj

Hi Matteo, nice article! I have a question about correlation id. Where do you store it so you can keep track of it once you get a response on the response topic? I was wondering if you store it in a memory cache it will not be reliable if the producer will crash, or if you have more running instances of this component then you need a shared cache?! Can you share your experience how you have deal with this scenario? Thanks, Aldo

Collapse
 
matteojoliveau profile image
Matteo Joliveau

Hi Aldo, thanks for the question, it's actually a very good point I missed in the article!
Correlation IDs are only useful if someone is actually waiting for a response in particular. Since most microservices are stateless and routing is taken care of by the broker itself, the Correlation ID can be kept in the message headers and passed around (and logged, always log CorIDs!) between services without them caring much about it. The only one that needs to keep track of it is the request initiator (e.g. the HTTP API Gateway). It will create an ID and assign it to the request message, then wait on its response topic for a message with the same ID.
The other services will not need to care about it since the topic on which to send responses to is provided by the request message itself (in the reply_to RabbitMQ property for example) so every req/res is self-contained.
You receive a request, you process it and you send it to the provided topic.

On the other hand, if you need to contact another service in order to answer a request you just received, you become an initiator yourself, so when you send your secondary request you can use the same Correlation ID when waiting for the response.

Now to actually answer your question, the IDs you are traking can be kept in memory (normally you use a thread pool or an event loop to deal with requests/responses so the ID can be maintained there) or you can use a cache like Redis, but by no means, it must be shared. Remember that microservices are independent and must only care about themselves.

I will update the article to add this and other enhancements :)

Collapse
 
aldex32 profile image
Aldo Sinanaj

Thanks for the reply! Waiting for the update :-)