You save an order to the database and then publish an order.created event. Two lines of code. What could go wrong?
DB::transaction(function () use ($order) {
$order->save();
});
Kafka::publish('order.created', $order); // what if this fails?
This is a dual write: one operation writing to two different systems (database + broker, database + cache, database + search index) with no transaction covering both.
Why it breaks:
- Partial failure: the order is committed, but the publish fails (network blip, broker down, process killed mid-request). The order exists, and no downstream service ever hears about it.
- Swapping the order doesn't help: publish first and you can end up with an event for an order that was never saved.
- Race conditions: two concurrent requests can write in a different order to each system, leaving them out of sync.
It even hides in familiar patterns. Dispatching a queued job with afterCommit() in Laravel is still a dual write: if the process dies between the commit and the push to Redis, the job is gone.
The fix: Transactional Outbox
Write the event to an outbox table inside the same database transaction:
DB::transaction(function () use ($order) {
$order->save();
OutboxMessage::create([
'topic' => 'order.created',
'payload' => $order->toJson(),
]);
});
A separate worker reads pending outbox rows, publishes them, and marks them as sent. Now it's atomic: either both the order and the event are saved, or neither is.
One catch: the relay can publish a message and crash before marking it as sent, so delivery is at-least-once. Your consumers need to be idempotent.
Other options:
- Change Data Capture (CDC): tools like Debezium read the database's transaction log and turn changes into events. The database stays the single source of truth.
- Event sourcing: the event log is the source of truth, and everything else is a projection.
- Two-phase commit: works in theory, but it's heavy, couples your services, and most brokers don't support it.
Ever been bitten by a dual write in production? How did you handle it?
Top comments (0)