DEV Community

Ed Legaspi
Ed Legaspi

Posted on Originally published at czetsuyatech.com

Transactional Outbox Pattern with Spring Boot: Reliable Event

#java #springboot #architecture #kafka


A service saves data to the database.

Then it publishes an event to Kafka.

Simple enough.

Until the database transaction succeeds and the Kafka publish fails.

Now your application says something happened, but the rest of the system never hears about it.

This is the dual-write problem, and it's one of the fundamental reliability problems in event-driven systems.

The Transactional Outbox Pattern gives us a practical way to solve it without trying to coordinate a distributed transaction between the database and the message broker.

In this article, we'll look at how it works, what guarantees it actually provides, and some of the production concerns that appear once you start running it across multiple application instances.

The Dual-Write Problem

Consider a typical Spring Boot service:

@Transactional
public void completePayment(Payment payment) {
    paymentRepository.save(payment);

    kafkaTemplate.send(
        "payments",
        new PaymentCompletedEvent(payment.getId())
    );
}
Enter fullscreen mode Exit fullscreen mode

At first glance, this looks reasonable.

But two independent systems participate in this operation:

  1. The application database
  2. Kafka

Your database transaction does not automatically include Kafka.

That creates failure scenarios.

Database succeeds, Kafka fails 

Database
Payment = COMPLETED  ✓

Kafka
PaymentCompleted     ✗
Enter fullscreen mode Exit fullscreen mode

The payment exists, but downstream services never receive the event.

Kafka succeeds, database fails

The opposite can also happen depending on when publishing occurs:

Kafka
PaymentCompleted     ✓

Database
Payment = COMPLETED  ✗
Enter fullscreen mode Exit fullscreen mode

Consumers may now process an event representing state that was never committed.

What we really want is this:

If the business transaction commits, the intention to publish its event must commit with it.

That's the problem the Transactional Outbox Pattern addresses.

Introducing the Transactional Outbox

Instead of publishing directly to Kafka inside the business transaction, we persist the event in the same database transaction as the business data.

          DATABASE TRANSACTION
     ┌─────────────────────────────┐
     │                             │
     │  Update Payment             │
     │         +                   │
     │  Insert Outbox Event        │
     │                             │
     └──────────────┬──────────────┘
                    │
                  COMMIT
                    │
                    ▼
             Outbox Dispatcher
                    │
                    ▼
                  Kafka
Enter fullscreen mode Exit fullscreen mode

Now the database provides the atomic boundary.

Either both records commit:

Payment       ✓
Outbox Event  ✓
Enter fullscreen mode Exit fullscreen mode

or neither does:

Payment       ✗
Outbox Event  ✗
Enter fullscreen mode Exit fullscreen mode

Kafka no longer has to participate in the business transaction.

That's the key idea.

We're not making the database and Kafka transactional together.

We're making the business state and the intention to publish transactional together.

What Goes Into the Outbox?

An outbox record contains enough information to publish the event later.

Conceptually:

event_id       = 01J...
event_type     = PaymentCompleted
source         = payment-service
payload        = {...}
content_type   = application/json
status         = PENDING
created_at     = ...
Enter fullscreen mode Exit fullscreen mode

The important part is that both the event payload and its delivery state are persisted.

I prefer keeping the payload readable rather than hiding it behind opaque serialization.

Why?

Because eventually something will fail in production.

When that happens, being able to inspect exactly what your service intended to publish makes debugging considerably easier.

The outbox isn't only a delivery mechanism.

It also becomes useful operational history.

Dispatching the Event

Once the business transaction has committed, a separate dispatcher processes the outbox.

A simplified lifecycle might look like this:

PENDING
   │
   ▼
PROCESSING
   │
   ├───────────────► PUBLISHED
   │
   └── failure
          │
          ▼
       retry later
Enter fullscreen mode Exit fullscreen mode

If an event repeatedly fails and exhausts its retry policy, it can eventually transition to:

FAILED
Enter fullscreen mode Exit fullscreen mode

This separation is important.

The business transaction is responsible for recording what happened.

The dispatcher is responsible for delivering that information.

Those responsibilities should be able to fail independently.

Why Not Just Retry Kafka Inside the Transaction?

A common alternative is to retry the Kafka operation:

@Transactional
public void completePayment(Payment payment) {
    paymentRepository.save(payment);

    retryTemplate.execute(context ->
        kafkaTemplate.send("payments", event)
    );
}
Enter fullscreen mode Exit fullscreen mode

Retries help with temporary broker failures.

But they don't solve the underlying coupling.

Imagine Kafka is unavailable for several minutes.

Should your payment transaction stay open while your application waits for Kafka to recover?

Usually, no.

Long-running transactions can:

  • hold database resources longer
  • increase lock duration
  • increase contention
  • couple business availability to broker availability

With an outbox, the responsibilities are separated:

Business Transaction
        │
        ├── commits quickly
        │
        ▼
      Outbox
        │
        │ Kafka unavailable?
        │
        └── retry asynchronously
Enter fullscreen mode Exit fullscreen mode

Your business operation doesn't need to wait for the broker.

The Important Catch: At-Least-Once Delivery

The outbox solves message loss, but it introduces another architectural consideration.

Imagine this:

1. Dispatcher publishes event to Kafka
2. Kafka accepts the event
3. Application crashes
4. Outbox wasn't marked PUBLISHED
5. Application restarts
6. Dispatcher publishes the event again
Enter fullscreen mode Exit fullscreen mode

The same event may be delivered twice.

This isn't necessarily a bug.

It's a consequence of favoring reliable delivery over silently losing events.

So a reliable event architecture usually becomes:

Transactional Outbox
        +
At-Least-Once Delivery
        +
Idempotent Consumer
Enter fullscreen mode Exit fullscreen mode

The producer cannot simply assume that every event will be processed exactly once.

Consumers must be designed to tolerate duplicates.

That naturally leads us to the Inbox Pattern, which I'll cover in the next article in this series.

What Happens With Multiple Application Instances?

Production applications rarely run as a single instance.

Suppose we have three pods:

                 OUTBOX
                   │
          ┌────────┼────────┐
          ▼        ▼        ▼
        Pod A    Pod B    Pod C
Enter fullscreen mode Exit fullscreen mode

If every instance polls the same outbox table without coordination, multiple workers could select the same events.

Database-level locking can provide one solution.

For databases that support it, a common approach is conceptually similar to:

SELECT ...
FROM outbox
WHERE status = 'PENDING'
FOR UPDATE SKIP LOCKED;
Enter fullscreen mode Exit fullscreen mode

Suppose Pod A locks:

1  2  3
Enter fullscreen mode Exit fullscreen mode

Pod B doesn't wait for those rows.

It skips them and can claim:

4  5  6
Enter fullscreen mode Exit fullscreen mode

This makes it possible for multiple workers to process the outbox concurrently without all competing for the same records.

But locking strategy matters.

A naive SELECT ... FOR UPDATE implementation can create unnecessary contention as throughput and worker count increase.

The important goal is:

Multiple workers should be able to safely claim work without serializing the entire outbox.

Polling vs Change Data Capture

Polling isn't the only way to implement the outbox pattern.

Another common architecture uses Change Data Capture (CDC):

Application
     │
     ▼
Database Outbox
     │
     ▼
    CDC
     │
     ▼
   Kafka
Enter fullscreen mode Exit fullscreen mode

Tools such as Debezium can stream database changes into Kafka instead of having application workers poll the outbox.

CDC can be a great choice, particularly when the required infrastructure already exists.

But it comes with different operational trade-offs.

Application-Level Polling

Advantages include:

  • simpler infrastructure
  • straightforward local development
  • application-controlled retries
  • easier application-level debugging
  • potential broker independence

Change Data Capture

Advantages can include:

  • reduced application polling
  • high-throughput change streaming
  • natural Kafka integration
  • separation of event extraction from application workers

Neither approach is universally better.

More importantly, polling vs CDC isn't the core of the pattern.

The architectural guarantee comes from this:

Business State + Outbox Event
              │
       SAME TRANSACTION
Enter fullscreen mode Exit fullscreen mode

How you transport that committed event afterward is a separate design decision.

Why I Built NERV Event

Once you've implemented this architecture several times, you start seeing the same infrastructure repeatedly:

Outbox persistence
       │
       ├── Serialization
       ├── Dispatching
       ├── Retries
       ├── Concurrency
       ├── Failure states
       ├── Metrics
       ├── Kafka integration
       ├── SQS integration
       └── Operational tooling
Enter fullscreen mode Exit fullscreen mode

That's infrastructure every business application shouldn't have to rebuild.

It's one of the reasons I created NERV Event.

NERV Event is an open-source Spring Boot library focused on reliable event processing using transactional Outbox and Inbox patterns.

The architecture is intentionally straightforward:

Business Service
       │
       ▼
   NERV Event
       │
       ▼
Transactional Outbox
       │
       ▼
   Dispatcher
       │
     ┌─┴─┐
     ▼   ▼
  Kafka  SQS
Enter fullscreen mode Exit fullscreen mode

The business application owns the event.

The infrastructure owns its reliable delivery.

And importantly, the state remains inspectable when something goes wrong.

Reliability Doesn't End at the Producer

The Transactional Outbox Pattern answers an important question:

How do I prevent an event from being lost after my business transaction commits?

But successfully publishing the event isn't the end of the reliability story.

Now we have another service receiving it.

What happens if:

  • the same event arrives twice?
  • processing succeeds but acknowledgement fails?
  • the consumer crashes halfway through processing?
  • processing needs to be retried?
  • we need to determine whether an event has already been handled?

These are consumer-side reliability problems.

And that's where the Inbox Pattern comes in.

In Article #2, we'll look at:

Inbox Pattern and Idempotent Consumers

and build the other half of reliable event processing.

NERV Event

NERV Event is open source and available on GitHub:

👉 github.com/czetsuyatech/nerv-event

If you're building event-driven Spring Boot systems and have dealt with outbox/inbox implementations in production, I'm interested in hearing how you're handling these problems — particularly polling vs CDC and multi-instance event claiming.

This article is part of my series on building reliable event-driven systems with Spring Boot.

Top comments (3)

Collapse
 
raknaos profile image
Baptiste Le Bouquin •

The detail I keep re-learning about outboxes is that the relay itself has to be idempotent or the pattern just relocates the bug. We ran a DB-poller version for a while and two replicas claiming the same row produced a duplicate event that looked exactly like the original dual-write failure, just later in the chain. Row-level claim with a lease + expiry fixed it, and a consumer-side dedupe key on top so a re-delivery after a crash is a no-op. For the polling-vs-CDC question you raise: we stayed on polling because CDC added an infra dependency we didn't want to operate for the volume we had, and at our scale the polling latency was noise. At what event rate did CDC start paying for you?

Collapse
 
czetsuya profile image
Ed Legaspi •

That matches our experience pretty closely.

We also found that the relay itself has to be treated as part of the reliability model, not just as a background job. Claims, leases, expiry, and consumer-side idempotency are what make the polling approach safe under crashes and multiple replicas.

On CDC: we tested it, but for our use case it brought more infrastructure and operational cost than we wanted to own. Once you add the CDC layer, you’re also operating things like connectors, replication/WAL concerns, offsets, monitoring, failure recovery, and another component in the delivery path.

At the event volumes we were targeting, polling with bounded batches, SKIP LOCKED, short claim transactions, and broker I/O outside the DB transaction was already cheap enough that the extra polling latency was effectively noise.

So we didn’t reach a specific “X events/sec” threshold where CDC suddenly won. The decision was more: the database poller was comfortably within our throughput/latency envelope, while CDC increased infrastructure complexity and cost without solving a problem we actually had yet.

If the workload eventually reaches a point where polling pressure on the primary DB becomes measurable, or if the organization already operates CDC infrastructure for other reasons, I’d absolutely reevaluate it.

Some comments may only be visible to logged-in visitors. Sign in to view all comments.