The transactional outbox exists to close one gap. A service has to change its own database and tell other services about it, and a database transaction cannot commit to a broker. Write state first and the event can be lost. Publish first and you can announce a transaction that later rolls back. So you insert an event row in the same local transaction as the business mutation, and change data capture relays the committed row afterwards.
That part is genuinely solved, and it is why we reach for the pattern. What we have learned to say out loud in design review is what it does not solve.
The relay is still at-least-once. A connector can read a row, publish it, and lose the broker acknowledgement; on restart it publishes again. So the event ID has to be generated inside the business transaction and survive every hop unchanged, and the consumer has to be built to see it twice. An inbox row keyed by source plus event ID, written in the same local transaction as the effect, is the version of this we trust for local effects. Remote calls do not get that for free. They need their own idempotency identity, or their own outbox.
Two decisions then fail quietly.
The first is the broker key. Ordering is guaranteed within a partition, so the key has to be the boundary the invariant actually lives on. Key on event type and events for one order scatter across partitions. Key on tenant and a large tenant serializes behind itself. Neither is visible in a test that runs one aggregate at a time.
The second is cleanup. Outbox tables grow indefinitely, so something deletes from them, and that job can outrun CDC. A row sitting behind the connector's offset is not proof that every downstream system accepted it. We treat the deletion watermark as a separate control from connector progress, leave margin, and test a connector rebuild against it. The related failure is log retention: a long enough connector outage and the database log segments the connector still needs are gone, which turns a delayed relay into a resnapshot. That deserves an alert well before the horizon rather than at it.
The last thing we push on is lag. Outbox publication is asynchronous, which makes connector lag a product behavior and not an infrastructure metric. Someone has to answer how long after commit search, notifications, fulfillment or analytics may lag. If the honest answer is that they cannot, the outbox is the wrong shape for that workflow and polling harder will not fix it.
The full write-up on our own site covers the rest: what belongs in the event payload versus a versioned reference, why the relay must never reread current tables, the ownership split between application and platform teams, and the failure-to-recovery table we work from.
Transactional Outbox with CDC: Close the Database-to-Broker Gap
Written by the engineering team at Edilec.
Top comments (0)