There are definitely some subtleties in different patterns. One of the reasons I wanted to share a little bit more about patterns is I think too often folks assume shared understanding. Patterns are not algorithms. They are ways we talk about the architecture we are building.
It's really hard to keep descriptions about a single pattern specific and clear but the example with financial transactions you are giving is why I brought up idempotency. I could have gone into greater detail there but it's a delicate balance of info loading and providing enough information at appropriate times. With transactions, they are definitely not idempotent in nature. If you keep applying a -$5 charge for coffee against an account, you'll quickly rack up a lot of credit charges. So it's a balance between retries, fast fails, and complete roll backs.
I'm curious though; why bring up the message queue pattern here? Retry to me is a lower level pattern. How do you communicate to a message queue? What compromises do you make when sending events into the message queue?
You're absolutely right, patterns are just like a way of communication.
why bring up the message queue pattern here?
How do you communicate to a message queue?
What compromises do you make when sending events into the message queue?
The retry pattern is meant for resilience.
Task queues (like Celery) already have the implementation of the retry pattern, and task queues require MQs (or brokers) to do their job.
So, to achieve a greater level or resilience, both a task queue and an MQ are necessary.
Actually, this pattern goes with couple of other design decisions (like microservices and event-driven-architecture).
Because of that, the biggest compromise when using such architecture is that your program would often need a complete rewrite.
Say for the previous example (of using a transaction to place an order), you would instead create an event "order_created".
This will trigger other services like shippingService & chargingService to do the next steps.
Suppose the chargingService failed in the middle, it will retry again (cuz the message of "charging_credit_card" won't leave the queue for couple of retries). If it ultimately fails, it will trigger another event for rolling back across multiple services.
For further actions, you may consider blocking this person and/or reporting abuse
We're a place where coders share, stay up-to-date and grow their careers.
There are definitely some subtleties in different patterns. One of the reasons I wanted to share a little bit more about patterns is I think too often folks assume shared understanding. Patterns are not algorithms. They are ways we talk about the architecture we are building.
It's really hard to keep descriptions about a single pattern specific and clear but the example with financial transactions you are giving is why I brought up idempotency. I could have gone into greater detail there but it's a delicate balance of info loading and providing enough information at appropriate times. With transactions, they are definitely not idempotent in nature. If you keep applying a -$5 charge for coffee against an account, you'll quickly rack up a lot of credit charges. So it's a balance between retries, fast fails, and complete roll backs.
I'm curious though; why bring up the message queue pattern here? Retry to me is a lower level pattern. How do you communicate to a message queue? What compromises do you make when sending events into the message queue?
You're absolutely right, patterns are just like a way of communication.
The retry pattern is meant for resilience.
Task queues (like Celery) already have the implementation of the retry pattern, and task queues require MQs (or brokers) to do their job.
So, to achieve a greater level or resilience, both a task queue and an MQ are necessary.
Actually, this pattern goes with couple of other design decisions (like microservices and event-driven-architecture).
Because of that, the biggest compromise when using such architecture is that your program would often need a complete rewrite.
Say for the previous example (of using a transaction to place an order), you would instead create an event "order_created".
This will trigger other services like shippingService & chargingService to do the next steps.
Suppose the chargingService failed in the middle, it will retry again (cuz the message of "charging_credit_card" won't leave the queue for couple of retries). If it ultimately fails, it will trigger another event for rolling back across multiple services.