DEV Community

Hayden Cordeiro
Hayden Cordeiro

Posted on

Database Consistency in Microservices!

If you're diving into the world of microservices, you've likely heard the of scalability, independent deployments, and tech-stack freedom. But beneath these shimmering waters is a beast, one that no man can tame (PS: this article will unlock a superpower which you can use to tame it), known as Data Consistency.

Remember the good old days of monoliths? You had one big, cozy database. You wrapped everything in a beautiful, all-or-nothing ACID transaction (Atomicity, Consistency, Isolation, Durability), and life was... well, simpler in that respect.

Image description

But in Microservice-Land, each service proudly has its own database. Our Order Service has its database, and our Inventory Service has its own. They might even be different types of databases (SQL! NoSQL! Ooh, fancy!). This is great for loose coupling, but how do you make sure that when an order is placed, inventory actually gets reduced, and what if one of them fails? You can't just stretch an ACID transaction across the network and different databases – that way lies madness (and often, something called Two-Phase Commit (2PC), which comes with its own box of horrors like blocking locks and lower availability ).

So, how do we keep our data from becoming a chaotic mess without tying our services back together? Enter our heroes: The Saga Pattern and the Transactional Outbox!

The Saga Pattern: Your Business Workflow's Storyteller

Think of a Saga as a story – a sequence of events. In our world, it's a sequence of local transactions. Each microservice performs its own piece of the work in its own local transaction and then signals the next service to pick up the baton.

Let's say a customer orders 'Product 1'. The Order Service starts the Saga:

  1. Order Service: Creates an order, marks it 'PENDING' (Local Transaction 1). Publishes an 'OrderPlaced' event.
  2. Inventory Service: Hears 'OrderPlaced', reserves inventory (Local Transaction 2). Publishes 'InventoryReserved'.
  3. Payment Service: Hears 'InventoryReserved', charges the card (Local Transaction 3). Publishes 'PaymentProcessed'.
  4. Order Service: Hears 'PaymentProcessed', updates order to 'CONFIRMED' (Local Transaction 4).

"But Hayden," you ask, "what if the Payment Service fails? We've already reserved inventory!"

Aha! That's where the magic of Sagas lies: Compensating Transactions. If any step fails, the Saga triggers 'undo' operations for all the preceding steps that succeeded. So, if payment fails, the Saga would tell the Inventory Service to 'ReleaseInventory' and the Order Service to 'CancelOrder'. Neat, huh? We aim for eventual consistency – the system will become consistent, just maybe not instantaneously.

Choreography vs. Orchestration: Who's Calling the Shots?

Sagas come in two main flavors:

  1. Choreography (The Dance Floor): Services talk to each other by publishing and listening to events. The Order Service shouts "Order Placed!", Inventory hears it and shouts "Inventory Reserved!", Payment hears that and shouts "Paid!".
    • Pros: Simple for small workflows, no central bottleneck.
    • Cons: Can get messy fast, hard to track, risk of circular matches.

Image description

  1. Orchestration (The Conductor): A central Orchestrator acts like a conductor, telling each service what to do and when. "Order Service, create order! Inventory, reserve! Payment, charge!".
    • Pros: Easier for complex flows, avoids cycles, better visibility.
    • Cons: Introduces a central component (potential bottleneck/failure point).

Image description

Know Your Steps: Compensable, Pivot, Retryable

To build a good Saga, you need to know your transaction types:

  • Compensable: These can be undone. Think 'Reserve Inventory' – you can always 'Release Inventory'.
  • Pivot: The point of no return. Once this step succeeds, the Saga must finish; no going back. Often, this is something irreversible, like charging a credit card.
  • Retryable: These come after the pivot. Since we can't go back, these steps must succeed eventually, even if we have to retry them a bunch of times (making them idempotent is key!). Think 'Send Confirmation Email'.

Understanding these helps you design robust Sagas that know when to roll back and when to push forward.

Transactional Outbox: The Unshakeable Mailman

Image description

Okay, Sagas are cool. They coordinate the workflow. But how does the Order Service reliably tell the Inventory Service that an order was placed? What if it updates its own database but then crashes before sending the message to Kafka (or RabbitMQ, or whatever you're using)?

Boom! Inconsistency. The Order Service thinks an order exists, but nobody else knows. This is the dreaded dual-write problem.

"But Kafka is fault-tolerant!" you cry. Yes, it is... once the message gets there. The gap between your local DB commit and the message hitting the broker is the danger zone.

This is where the Transactional Outbox pattern saves the day. It's surprisingly simple:

  1. When the Order Service creates an order, it does two things in the same, single, local ACID transaction:
    • Inserts the 'Order' row into its Orders table.
    • Inserts an 'OrderPlaced' event message into a special Outbox table.
  2. Because it's one transaction, either both writes succeed, or neither does. No more dual-write gap!
  3. A separate, reliable Message Relay process monitors the Outbox table.
  4. This relay picks up new events and reliably publishes them to your message broker.
  5. Once successfully published, the relay marks the event as 'sent' or deletes it from the Outbox.

The relay can use Polling (periodically checking the table ) or Change Data Capture (CDC) (tailing the database logs, often preferred for lower latency and load ).

The Outbox acts like a durable, guaranteed-to-send mailbox inside your service's database.

Better Together: Saga + Outbox

So, is it Saga or Outbox? Nope! It's usually Saga and Outbox.

  • Saga defines the business logic and workflow across services.
  • Outbox provides the reliable messaging infrastructure that allows each step in the Saga to communicate without losing messages.

Think of Outbox as the durable pipe and Saga as the traffic controller. Every time a Saga step (whether choreographed or orchestrated) needs to send a message, it uses its local Outbox to ensure that message gets out if, and only if, its local work was successfully committed.

When Do I Need This Stuff?

Should every microservice use Sagas and Outboxes? Probably not! Always aim for simplicity.

Here’s a quick-and-dirty decision tree:

Image description

  1. Does your action touch > 1 service's DB?
    • No: Hallelujah! Use a simple local ACID transaction. You're done!
    • Yes: Keep going...
  2. Do you need instantaneous, strict global consistency?
    • Yes: Oof. Maybe microservices aren't the best fit here? Consider consolidating or facing the 2PC hydra.
    • No (Eventual is OK): Keep going...
  3. Do you need guaranteed messaging (no lost events)?
    • Yes: You need the Transactional Outbox.
    • No: Maybe simple fire-and-forget messaging is enough (but be careful!).
  4. Is there a multi-step flow that might fail midway and needs coordination/rollback?
    • Yes: You need a Saga.
    • No: Outbox alone might be sufficient.

The Takeaway 🎉

Distributed systems are hard, and maintaining data consistency across microservices is one of the trickier puzzles. But by understanding the Saga pattern (for workflow orchestration) and the Transactional Outbox pattern (for reliable messaging), you have powerful tools to build resilient, scalable, and eventually consistent systems without falling back into the monolithic abyss or getting burned by 2PC.

Now go forth and build amazing things!

Top comments (0)