Handling Distributed Transactions with Orchestrator Pattern (Withdrawal & Deposit Example)

When building microservices, one of the common challenges is dealing with distributed transactions — ensuring data consistency when multiple services need to work together.

Let's consider a simple but very real-world example:

• Service A: Withdrawal
• Service B: Deposit

We want to withdraw money from one account and deposit into another.
But what if one of them fails?

The Problem

Imagine the following flow:

1. Withdrawal Service reduces the balance.
2. Deposit Service adds the amount to the destination account.

If step 1 succeeds but step 2 fails, you have just lost money in the system.
This is where distributed transaction management becomes critical

Common Approaches

1. Two-Phase Commit (2PC)

A transaction coordinator asks each service to prepare and commit.
If all services agree, the transaction is committed.
If any fail, everything is rolled back.

✅ Strong consistency

❌ High complexity, risk of blocking, not always a good fit for microservices

1. Saga Pattern with Orchestrator

The Saga pattern breaks the big transaction into a sequence of smaller, local transactions.
Each step has a compensating transaction (rollback action) if something goes wrong.

Example:

Step Action Compensation
1 Withdraw from Account A Deposit back to Account A
2 Deposit to Account B Withdraw from Account B

The orchestrator is a service that manages this workflow:

• Starts with withdrawal
• If successful, triggers deposit
• If deposit fails, runs compensation (refund) for withdrawal This is much more scalable and microservice-friendly.

1. Event-Driven & Eventually Consistent

Another approach is using message queues (Kafka, RabbitMQ):

• Send a WithdrawalCompleted event
• Deposit service consumes and processes it
• Retry on failure until success
• Make services idempotent (safe to retry without double processing)

This ensures eventual consistency even if failures occur.

Orchestrator Workflow Example

Example: Orchestrator Implementation (Pseudo-Code)

public class TransferOrchestrator {

    public void transfer(String fromAccount, String toAccount, BigDecimal amount) {
        boolean withdrawSuccess = withdrawalService.withdraw(fromAccount, amount);

        if (!withdrawSuccess) {
            log.error("Withdrawal failed");
            return;
        }

        boolean depositSuccess = depositService.deposit(toAccount, amount);

        if (!depositSuccess) {
            log.error("Deposit failed, triggering compensation...");
            withdrawalService.compensate(fromAccount, amount); // rollback
        } else {
            log.info("Transfer completed successfully");
        }
    }
}

Key Best Practices

✅ Idempotent APIs – handle retries safely
✅ Proper Logging – so you can trace what happened
✅ Dead Letter Queues – for failed events that need manual review
✅ Monitoring & Alerts – you don’t want silent failures

Final Thoughts
**
Distributed transactions are challenging, but with the **Saga pattern and orchestration, you can build resilient and scalable systems.

The orchestrator gives you full control over the transaction flow and lets you recover gracefully from failures — which is critical for financial and mission-critical systems