Why 2-Phase Commit Fails in Microservices — And How the Saga Pattern Saves the Day

When talking about microservices, it consists of multiple services communicating with each other giving the application an edge over the monolytic architecture.

But the biggest problem encountered in a micro-service architecture is how to handle the transaction that spans multiple services.
Hence making data consistency across the service database a big challenge. In a micro-service architecture, every service can have its own database, tech stack and database.

So when a transaction has to depend on more than one service, data consistency is needed.

Distributed transactions —

In the above diagram, we have 4 services with their transactions. To ensure a successful order processing service, all four microservices must complete their indivicual transactions. If any transaction fails, all the preceding transaction updates must be rolled back.

But the distrubuted transactions have following challenges —

1. It must maintain ACID : To ensure correctness of a transaction, it must be atomic, consistent, isolated and durable. As in this scenario, the transaction spans multiple services, ensuring ACID is a big challenge.
2. Isolation of transactions : If a common object is changed by one service and is read by another service, there is data inconsistency.

To overcome these limitations, 2 Phase-Commit can be used.

Two Phase Commit —

The Two-Phase Commit protocol (2PC) is a widely used pattern to implement distributed transactions. We can use this pattern in a microservice architecture to implement distributed transactions.

In two phase commit, there is a central body called coordinator which is connected to the services which is responsible to control the transactions and contains the logic to manage the transactions.

The two phase commit runs a distributed transaction in two phases —
Phase 1 **: In this phase, coordinator asks all the nodes whether they are ready to commit the transaction or not.
**Phase 2 : In this phase, depending on the ack received by the nodes, if all ack are yes, then the coordinator asks all of them to commit. Otherwise, if even one ack is a no, then the coordinator tells all the nodes to roll back their transactions.

2 PC sounds great. But what is the problem with this design?

1. Single Point of Failure(SPoF) — If the coordinator is down, the whole system goes down.
2. All the services need to wait for the slowest service for confirmation. So, the overall performance of the transaction is bound by the slowest service. 3. Two-phase commit protocol is not supported in NoSQL databases.

SAGA Architecture Pattern The SAGA pattern provides transaction management using a sequence of local transactions. Each service is considered as a SAGA participant and each participant has its individual transaction. SAGA makes sure that either all the transactions complete successfully or the corresponding compensation transactions are run to undo the work previously completed ie. bascially rolling back the other transactions.
In this pattern, the compensation transactions must me idempotent and retryable.

The Saga Execution Coordinator (SEC) guarantees these principles:

The Saga Execution Coordinator is a central component used to implement a SAGA flow. It contains a saga log that captures the sequence of events of a distributed transaction.
In the diagram below, the success scenario is shown for all the transactions completed and the logs captured in the saga log.

There are two approaches to implement SAGA design pattern —

1. SAGA Choreography Pattern
2. SAGA Orchestration Pattern

SAGA Chorography Pattern

In this pattern, each microservice pushes an event to be fetched by the next microservice. Here the microservices should have their individual transactions and be a part of the Saga.
In this pattern, the Saga Execution Coordinator is either embedded within the microservice or can be a standalone component. In the Saga, choreography flow is successful if all the microservices complete their local transaction, and none of the microservices reported any failure.
So, if two services ran successfully and the third failed, the other services listen to the event of failure and roll back their own transactions. Hence eliminating the SPOF.

The following diagram demonstrates the successful Saga flow for the online order processing application:

In the case of failure, the microservice logs the failure in the log and reports the failure to SEC and its SEC’s responsibility to run the compensating transactions using the logs and keep retrying of it fails.

In this example, the Payment microservice reports a failure, and the SEC invokes the compensating transaction to unblock the seat. If the call to the compensating transaction fails, it is the SEC’s responsibility to retry it until it is successfully completed.
Recall that in Saga, a compensating transaction must be idempotent and retryable. The Choreography pattern works for greenfield (from scratch) microservice application development. Also, this pattern is suitable when there are fewer participants in the transaction.
Here are a few frameworks available to implement the choreography pattern:

1. Axon Saga — a lightweight framework and widely used with Spring Boot-based microservices
2. Eclipse MicroProfile LRA — implementation of distributed transactions in Saga for HTTP transport based on REST principles
3. Eventuate Tram Saga — Saga orchestration framework for Spring Boot and Micronaut-based microservices.

SAGA Orchestration Pattern

In this pattern, a single orchestrator is responsible for managing the overall transaction status. If any of the microservices encounter a failure, the orchestrator is responsible for invoking the necessary compensating transactions:

The Saga orchestration pattern is useful for brownfield microservice application development architecture. In other words, this pattern works when we already have a set of microservices and would like to implement the Saga pattern in the application.
We need to define the appropriate compensating transactions to proceed with this pattern. It is different from choreography, as the orchesrator decides every thing and has the control with it. Whereas in choreography, one service has to send event to next, which makes the services in control.

Here are a few frameworks available to implement the orchestrator pattern:

1. Camunda is a Java-based framework that supports Business Process Model and Notation (BPMN) standard for workflow and process automation.
2. Apache Camel provides the implementation for Saga Enterprise Integration Pattern (EIP).

Conclusion

In this article, we discussed the Saga architecture pattern to implement distributed transactions in a microservice-based application.
We first introduced the challenges of these implementations.
We then explored the two-phase commit protocol, a popular alternative of Saga, and examined its limitation to implement distributed transactions in microservice-based applications.
Lastly, we discussed the Saga architecture pattern, how it works and the two main approaches to implementing the Saga pattern in microservice-based applications.