Event-Driven Microservice – Software Architecture Pattern

In this post I will go over Event-Driven Microservice Architecture, its benefits and a Case Study implementation of an E-Commerce application. Most modern applications are run as micro services as it provides scalability and decoupled structure. One of the core principles of a micro service architecture is Database per Service Pattern where each database will be accessed by only one service, which provides a way of atomicity.

Difficulties in Micro-services Pattern:

One major difficulty in micro-services is, in real world applications a single request cannot be served by a single micro-service, as it might need different data from multiple DB’s. The immediate option is to make inter-process communication between micro-services to make this happen. There are some consequences of doing inter-process communication.

Pitfalls In Inter-process Communication:

• Module A waits for the response
• Dependent Module can be down
• Network delays can decrease performance
• If huge data then pagination is required. This means more REST calls between process
• Any Module can be under heavy load and can introduce latency.

Event-Driven Approach:

To overcome the above short coming, We can use Event-Driven approach. An event-driven microservice architecture uses events to trigger and communicate between services. An event is a state change or an action to which other services might react as per the business process. This will take us to the reactive programming model.

I will run through a real world example of an event-driven architecture implementation for a fairly simple eCommerce application workflow. The workflow is as below:

1. Customer Orders product through Website.
2. Create Temp Order and attach a transaction to order.
3. Payment Confirmation
4. Create Order
5. Order Dispatch from warehouse
6. Update Site with Product quantity(Real Time)
7. Order Shipped
8. Order Delivered

Each micro-service performs a local transaction and publishes an event which can be consumed by other dependent micro-services to act upon accordingly. The Email Service consumes all the events from Order channel. So that every status change in the order has a corresponding email notification. There is no direct inter-process communication but all communications happens through events.

With this approach we can even show real time data changes to the Users. The warehouse service sends an event after the order is dispatched. This event is consumed by the UI and the product stock is adjusted as per the data from the event.

FAT Events:

Use of fat events makes this architecture pure async. Fat events are events where messages posted to the queue contains details of the items processed or a snapshot of the latest data. This ensures the consumer doesn’t need additional API call to get the latest data from the DB.

Idempotency:

No technology is invincible, this goes for the message queue as well. There might be occurrences where the message queue does face some issues and resends the same message multiple times. So it is very important for all the consuming services to check for idempotency of the events before processing it to avoid duplicates.

One quick solution is that we can add a unique key to each event message and check for the key in all the consumer services before processing.

Outbox Pattern:

Another scenario in this architecture is messages getting lost due to Queue downtime. These become irreversible. To mitigate this, we can follow the outbox pattern where all messages are stored in DB and then processed by an event service and sent to consumers. This way even if the Queue is down the messages are still recorded and can be replayed to Queue when it is back on.

Conclusion:

In micro service architecture, we have to keep coupling low. To be able to keep the coupling low, we have to focus on the connections between modules. Also we want scalability and flexibility at will, and this can be achieved by Event-Driven approach. We can combine this architecture with power of serverless computing.