Outbox Pattern

The problem

I'm working on an OMS (Order Management System) that listens to a message queue for orders that are placed and then processes them. There are a few steps involved, as follows:

1. Get the order placement message from the message queue.
2. Process the order placement depending on business needs.
3. Save the state of the order to the database.
4. Publish a new message for the next step to the message queue.

There is a problem since we need to save the state of the order and publish a message for the next step, and these two operations are not atomic, which means one can fail and the other will not be able to revert. We could run in a situation where the order's state is updated successfully in the database, but no message for the next step is published to the message queue due to an error or network timeout, causing the order to be stuck and not progress any further. This could make the data in our system to be inconsistent and unreliable.

We can try to have a condition here that if the publishing to the message queue fails, we can retry to deliver the message or revert the order back to its previous status. But what if the system crashed while trying to publish the message or that the retry/revert also failed, how can we revert the state from the database?

The solution

The solution I came up with after some research is that we can use the outbox pattern!

We created a new table called order_outbox in the database, so every time we update the order state we also insert a new record into the order_outbox table in the same transaction making them atomic, and having a new message relay to read from the table. The message relay will then publish all unpublished messages it can get from the order_outbox.

A caution

Using the Outbox Pattern will guarantee that our publisher/producer will be an at-least-once publisher. But it may cause duplicated message delivery. So make sure your consumer/subscriber is tolerant of duplicated messages by business requirements or by implementing the Idempotent Consumer pattern.

References

This post is a note to myself, so if you find it interesting but hard to understand. You can follow these two links for references.

• Microservices Pattern: Transactional outbox - microservices.io
• Outbox Pattern: Reliably Save State & Publish Events - codeopinion.com