DEV Community: Yevhenii Kukhol

Outbox Pattern: RabbitMQ Publishing Strategies for High-Performance Systems

Yevhenii Kukhol — Sun, 22 Jun 2025 12:12:45 +0000

Why Outbox Publishing?
Outbox: Table + Scheduled Publishing
Database Side: Already Solved
Potential Publishing Incidents: What Goes Wrong Under Load
Understanding Publisher Confirms: Foundation for Reliability
Strategy 1: Fire and Forget (Simple but Risky)
Strategy 2: Synchronous Batch ACK
Strategy 3: Async ACK with Correlation (Complex but High Throughput)
Channel Churn Pitfall: Critical for Both Strategies
Monitoring and Observability
Conclusion

Why Outbox Publishing?

The database side of the outbox pattern has been excellently covered by @msdousti. But what about the publishing side?

My investigation started when a seemingly robust outbox implementation began causing incidents under high load. The system worked flawlessly during development and low-traffic periods, but when traffic spiked, we experienced:

Memory exhaustion from improper channel management
Publishing delays that created growing backlogs

What you might also experience:

Lost messages due to various reasons
Complete system freezes when RabbitMQ publishing became a bottleneck

These incidents taught me that the publishing is just as critical as the database design in the outbox pattern. A poor publishing implementation can negate all the reliability benefits of the outbox approach.

This post shares the lessons learned and presents a few strategies for reliable, high-performance outbox message publishing.

Outbox: Table + Scheduled Publishing

Before diving into publishing strategies, let's clarify what I mean by the outbox pattern and why we focus on the scheduled publishing approach.

What is the Outbox Pattern?

The outbox pattern ensures reliable message delivery in distributed systems through three key steps:

Persist messages in a database table alongside business data (same transaction)
Scheduled publisher reads unpublished messages from the outbox table
Mark as published once successfully delivered to the message broker

CREATE TABLE outbox (
    id BIGINT PRIMARY KEY,
    payload JSON NOT NULL,
    created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
    published_at TIMESTAMPTZ  -- NULL = unpublished
);

Why Focus on Scheduled Publishing?

There are several outbox implementation approaches:

Approach	Mechanism	Pros	Cons
Debezium CDC	Change Data Capture	Automatic, Kafka-optimized	Complex setup, Kafka-specific
Event-driven	Database triggers/events	Immediate publishing	Tight coupling, hard to debug
Scheduled Publishing	Periodic job reads table	Full control, flexible	Manual implementation

This article focuses on scheduled publishing because it provides:

Full control over publishing logic and error handling
Flexibility to publish to any message broker
Simpler debugging and observability
Custom retry strategies and performance tuning

The scheduled approach fits perfectly when you need reliable publishing with complete control over the process.

Database Side: Already Solved

The database optimization aspects of the outbox pattern—including table partitioning, indexing strategies, and query performance—are thoroughly covered in @msdousti's comprehensive article.

His work demonstrates how to:

Implement partitioned outbox tables for performance
Optimize queries for fetching unpublished messages
Handle index cleanup and maintenance
Avoid common PostgreSQL pitfalls

This article picks up where his leaves off: once you have optimized outbox table queries, how do you publish those messages to RabbitMQ efficiently and reliably?

Potential Publishing Incidents: What Goes Wrong Under Load

1. Channel Exhaustion (OutOfMemoryError)

Symptoms:

Application crashes with OutOfMemoryError
High memory usage in heap dumps
RabbitMQ connection metrics show excessive channel creation

Root Cause:

// ❌ DANGEROUS: Each call may create new channel
unpublishedMessages.forEach { message ->
    rabbitTemplate.convertAndSend(exchange, routingKey, message)
}

Each convertAndSend() can check out a new channel from the cache. Under high load, channels with pending operations can't be returned to cache, forcing creation of new channels until memory is exhausted.

Note: Channel churn can also lead to poor performance.

2. Publishing Bottlenecks

Symptoms:

Growing outbox table backlog
Publishing throughput can't keep up with message creation
System becomes unresponsive during high traffic

Root Cause:
Poor acknowledgment strategies that block publishing threads:

// ❌ BLOCKS: Each message waits for individual confirmation
unpublishedMessages.forEach { message ->
    rabbitTemplate.invoke { channel ->
        channel.convertAndSend(exchange, routingKey, message)
        channel.waitForConfirmsOrDie(10_000) // Blocks here!
    }
}

3. Silent Message Loss

Symptoms:

Messages disappear without error logs
Inconsistencies between outbox table and actual delivered messages
"Missing" business events in downstream systems

Root Cause:
Fire-and-forget publishing without any reliability checks:

// ❌ NO GUARANTEES: Message might never reach broker
rabbitTemplate.convertAndSend(exchange, routingKey, message)
// Immediately marked as published, but was it really?
markAsPublished(message)

Understanding Publisher Confirms: Foundation for Reliability

Before exploring specific strategies, it's crucial to understand RabbitMQ's publisher confirm mechanism, which I covered in detail in my previous articles:

From Fire-and-Forget to Reliable: RabbitMQ ACK - covers simple and batch acknowledgments
From Fire-and-Forget to Reliable: RabbitMQ ACK [pt. 2] - covers async confirmations with correlation

Publisher confirms ensure message delivery by having the broker acknowledge receipt of each message. This is essential for reliable outbox publishing, as it tells you definitively whether a message reached the broker.

Strategy 1: Fire and Forget (Simple but Risky)

Let's start with the simplest approach, though it's likely not suitable for most outbox implementations.

When Fire-and-Forget Makes Sense

Fire-and-forget is appropriate when:

Performance is critical over reliability
Occasional message loss is acceptable
Network environment is very stable
Non-critical business events

Implementation

@Service
class FireAndForgetPublisher(
    private val rabbitTemplate: RabbitTemplate
) {
    fun publishOutboxMessages(messages: List<OutboxMessage>) {
        // ✅ CRITICAL: Channel churn does not occur here because no confirmation type is set.
        // Therefore, no related background processes (e.g., waiting for acknowledgment) are attached to the channel,
        // allowing the channel to be returned to the cache immediately after publishing.
        rabbitTemplate.invoke { channel ->
            messages.forEach { outboxMessage ->
                channel.convertAndSend(
                    exchangeName,
                    routingKey,
                    outboxMessage.payload
                )
            }
        }

        // Mark all as published immediately
        // ⚠️ RISK: No guarantee they actually reached the broker
        outboxRepository.markAsPublished(messages.map { it.id })
    }
}

Key Characteristics

Throughput: A LOT of messages/second
Latency: Minimal (no network round-trips for confirmations)
Reliability: None (messages can be lost silently)

Strategy 2: Synchronous Batch ACK

For most outbox implementations, synchronous batch acknowledgments provide the optimal balance of reliability, simplicity, and performance while avoiding the transaction boundary issues inherent in async approaches.

Why Synchronous Batch ACK is Ideal for Outbox

The synchronous batch approach solves critical outbox publishing challenges that async approaches cannot address:

Transaction atomicity - publishing and database updates happen in the same transaction
No limbo state - messages are never stuck between published and marked-as-published
Simple error handling - either all messages succeed or all retry together
Guaranteed consistency - SELECT FOR UPDATE prevents duplicate processing across threads
Predictable behavior - no async callbacks or timeout handling complexity

Core Implementation

@Service
class SyncBatchOutboxPublisher(
    private val rabbitTemplate: RabbitTemplate,
    private val outboxRepository: OutboxRepository
) {
    @Transactional
    fun publishOutboxMessages() {
        // ✅ SELECT FOR UPDATE prevents other threads from selecting same messages
        val messages = outboxRepository.findUnpublishedAndLock(limit = 1000)

        if (messages.isEmpty()) return

        try {
            // Publish all messages in single channel with batch confirmation
            rabbitTemplate.invoke { channel ->
                messages.forEach { outboxMessage ->
                    channel.convertAndSend(
                        exchangeName,
                        routingKey,
                        outboxMessage.payload
                    )
                }

                // ✅ Wait for ALL confirmations before proceeding
                // This ensures all messages reached the broker
                channel.waitForConfirmsOrDie(30_000)
            }

            // ✅ ALL messages confirmed - mark in SAME transaction
            // Either all succeed or transaction rolls back
            outboxRepository.markAsPublished(messages.map { it.id })

        } catch (Exception e) {
            // ❌ Publishing failed - transaction rolls back
            // Messages remain unpublished and will be retried
            log.error("Publishing failed for batch of ${messages.size} messages", e)
            throw e  // Let transaction rollback
        }
    }
}

Configuration for Sync Batch ACK

spring:
  rabbitmq:
    publisher-confirm-type: simple  # Required for confirmations
    cache:
      channel:
        size: 10
        checkout-timeout: 5000

Key Characteristics

Throughput: ~1,000-5,000 messages/second (depending on batch size)
Latency: Moderate (waits for batch confirmations)
Reliability: Highest (atomic transactions, no lost messages, duplicates possible on retry)
Memory: Low (no pending confirmation tracking needed)
Complexity: Low (simple transaction flow)

Advantages for Outbox Pattern

✅ Transaction Safety

@Transactional
fun publishOutboxMessages() {
    val messages = findAndLock()     // Same transaction
    publishAllMessages(messages)     // Same transaction  
    markAsPublished(messages)        // Same transaction
    // Either ALL succeed or ALL rollback - no partial state
}

✅ Thread Safety (Note: publish out-of-order possible)

// Thread 1: Locks messages 1-1000
val batch1 = outboxRepository.findUnpublishedAndLock(limit = 1000)

// Thread 2: Gets messages 1001-2000 (different records)
val batch2 = outboxRepository.findUnpublishedAndLock(limit = 1000)

// No overlap possible due to SELECT FOR UPDATE

✅ Simple Error Recovery

try {
    publishAndConfirm(messages)
    markAsPublished(messages)
} catch (Exception e) {
    // Transaction rolls back automatically
    // Messages remain unpublished for next retry
    // No complex cleanup needed
}

⚠️ Publish Duplicates on NACK

try {
    publishAndConfirm(messages)
    markAsPublished(messages)
} catch (Exception e) {
    // If publishAndConfirm failed due to received NACK - we're
    // forced to retry the whole batch which creates duplicates
}

Theoretically, to address duplicate publications on NACK, you can use the callbacks provided by the invoke() method. You can define two callbacks: one for ACK and one for NACK. These callbacks only provide the deliveryTag, but since you're publishing from the same channel, you can try caching a deliveryTag-to-outboxMessageId mapping in order to individually track and handle NACKed messages and avoid this issue.

Strategy 3: Async ACK with Correlation (Complex but High Throughput)

While async acknowledgments offer higher throughput, they introduce significant complexity and potential consistency issues for outbox implementations. This approach is not recommended until other options are not enough for your use case.

Critical Issues with Async ACK for Outbox

Before exploring the async approach, it's important to understand the fundamental challenges it creates:

⚠️ Transaction Boundary Violation

@Transactional
fun publishOutboxMessages() {
    val messages = findUnpublishedAndLock()
    publishAsync(messages)  // Messages sent to broker
    // Transaction COMMITS here - but ACKs arrive later!
}

// Different thread, different transaction
confirmCallback { correlationData, ack, cause ->
    if (ack) {
        // ❌ If this fails, message was published but not marked!
        markAsPublished(messageId)
    }
}

⚠️ Lost ACK Problem

// What if ACK/NACK never arrives due to:
// - Network issues
// - Broker restart  
// - Connection loss
// 
// Message is published but stuck in "pending" state forever
pendingConfirmations[messageId] = message  // Limbo state

Why Async ACK is Problematic for Outbox

The async approach conflicts with key outbox pattern guarantees:

Breaks atomicity - publish and mark-as-published happen in different transactions
Creates limbo state - messages can be published but never marked as such
Timeout complexity - need cleanup jobs for lost ACKs
Complex recovery - sophisticated state machine required

When Async ACK Makes Sense

Despite the issues, async ACK can work when:

Batch sync ACK is not enough
NACKs happen often, which leads to tons of duplicates with sync ACK
Complex state management is acceptable

Async Implementation (Use with Caution)

@Service
class AsyncAckOutboxPublisher(
    private val connectionFactory: ConnectionFactory,
    private val outboxRepository: OutboxRepository
) {
    private val pendingConfirmations = ConcurrentHashMap<String, OutboxMessage>()

    @Transactional
    fun publishOutboxMessages() {
        val messages = outboxRepository.findUnpublishedAndLock(limit = 1000)

        // ⚠️ Mark as IN_FLIGHT to prevent re-selection
        outboxRepository.markAsInFlight(messages.map { it.id })

        publishAsync(messages)
        // Transaction commits with messages in IN_FLIGHT state
    }

    private fun publishAsync(messages: List<OutboxMessage>) {
        template.invoke { channel ->
            messages.forEach { outboxMessage ->
                // Track pending confirmation
                pendingConfirmations[outboxMessage.id] = outboxMessage

                // Publish with correlation data
                channel.convertAndSend(
                    exchangeName,
                    routingKey,
                    outboxMessage.payload,
                    CorrelationData(outboxMessage.id)
                )
            }
        }
    }

    private fun createAsyncTemplate(): RabbitTemplate {
        return RabbitTemplate(connectionFactory).apply {
            messageConverter = Jackson2JsonMessageConverter()
            setMandatory(true)

            setConfirmCallback { correlationData, ack, cause ->
                val messageId = correlationData?.id
                val outboxMessage = messageId?.let { 
                    pendingConfirmations.remove(it) 
                }

                if (ack && outboxMessage != null) {
                    handleSuccessfulPublish(outboxMessage)
                } else if (outboxMessage != null) {
                    handleFailedPublish(outboxMessage, cause ?: "Unknown error")
                }
            }
        }
    }

    private fun handleSuccessfulPublish(outboxMessage: OutboxMessage) {
        try {
            outboxRepository.markAsPublished(outboxMessage.id)
        } catch (e: Exception) {
            // ❌ Critical: Message was delivered but not marked as published!
            log.error("Failed to mark message as published: ${outboxMessage.id}", e)
            // Need sophisticated recovery mechanism here
        }
    }

    @Scheduled(fixedDelay = 60000)  // Cleanup job required
    fun cleanupTimeouts() {
        val timeoutMessages = outboxRepository.findInFlightOlderThan(Duration.ofMinutes(5))
        timeoutMessages.forEach { message ->
            if (pendingConfirmations.containsKey(message.id)) {
                // Still pending - reset to unpublished for retry
                outboxRepository.markAsUnpublished(message.id)
                pendingConfirmations.remove(message.id)
            }
        }
    }
}

Required Infrastructure for Async ACK

If you choose async ACK despite the complexity, you need:

State machine with PENDING/IN_FLIGHT/PUBLISHED/FAILED states
Cleanup jobs for handling timeouts and lost ACKs
Duplicate detection at consumer level (always should be there IMO)
Sophisticated monitoring for limbo states
Manual intervention procedures for poison pill messages (needed for sync ACK as well)

Configuration for Async ACK

spring:
  rabbitmq:
    publisher-confirm-type: correlated  # Required for correlation
    cache:
      channel:
        size: 10
        checkout-timeout: 5000

Key Characteristics

Throughput: Limited only by network and number of workers
Latency: Low (non-blocking confirmations)
Reliability: Moderate (complex error scenarios)
Memory: High overhead (tracking pending confirmations + cleanup jobs)
Complexity: High (async flows, state machines, limbo state handling)

Channel Churn Pitfall: Critical for Both Strategies

Channel churn is one of the most dangerous issues in high-throughput RabbitMQ applications. Both publishing strategies must address this properly.

Understanding the Problem

As documented in the RabbitMQ Java Client Guide, the RabbitMQ client uses connection pooling and channel caching. However, under high load:

Channels with pending operations cannot be returned to cache immediately
New channels get created when cache is exhausted
Memory usage grows as channels accumulate
OutOfMemoryError occurs when too many channels exist

Solution: Publish Batch with Same Channel

// ❌ PROBLEMATIC: Each call may check out new channel
outboxMessages.forEach { message ->
    rabbitTemplate.convertAndSend(exchange, routingKey, message.payload)
}

// ✅ CORRECT: Single channel for all operations
rabbitTemplate.invoke { channel ->
    outboxMessages.forEach { message ->
        channel.convertAndSend(exchange, routingKey, message.payload)
    }
}

Why invoke() Works

The invoke() method ensures:

Single channel checkout for the entire operation
No channel creation overhead during bulk operations
Predictable memory usage regardless of message count with a limited number of publisher threads

Additional Channel Management

For production systems, consider these channel cache settings:

spring:
  rabbitmq:
    cache:
      channel:
        size: 25              # Reasonable cache size
        checkout-timeout: 5000 # 5 second timeout

When checkout-timeout is set, the channel cache becomes a hard limit. If all channels are busy, threads will wait up to 5 seconds for an available channel, preventing unlimited channel creation. In other words, you can avoid OOM at the price of channel checkout timeout exceptions. A rule of thumb here: make sure you're not wasting resources on creating/closing channels while the cache size is sufficient to handle your load, so your application is not waiting for channels to be ready.

Monitoring and Observability

For production outbox implementations, implement these monitoring strategies:

Key Metrics

Outbox table size - number of unpublished messages
Publishing rate - messages published per second
Confirmation rate - ACK/NACK ratio
Pending confirmations - messages waiting for confirmation
Channel utilization - active channels vs cache size

Conclusion

The outbox pattern is only as reliable as its publishing implementation.

Key Takeaways

Channel management is critical: Try using invoke() to batch publish messages with a single channel.
Choose the right strategy for your needs:
- Fire-and-Forget: Only for non-critical data where performance trumps reliability
- Synchronous Batch ACK: The recommended approach for most outbox implementations
- Async ACK with Correlation: Only when sync batch throughput is insufficient and you can handle the complexity
Sync Batch ACK is usually the best choice because it:
- Maintains transaction atomicity
- Avoids limbo states
- Simplifies error handling
- Requires minimal infrastructure
Monitor everything: Track outbox table size, publishing rates, and channel utilization to catch issues before they become incidents.

Remember: the goal of the outbox pattern is reliability. Don't compromise that reliability for marginal performance gains. A slightly slower but rock-solid implementation will serve you better than a fast but fragile one.

The synchronous batch ACK strategy provides the optimal balance for most use cases—embrace its simplicity and let it handle your critical business events with confidence. But remember about the duplicate issue.

From Fire-and-Forget to Reliable: RabbitMQ Ack [pt. 2]

Yevhenii Kukhol — Sun, 15 Jun 2025 11:49:28 +0000

When building high-throughput messaging systems with RabbitMQ, the choice of publishing strategy can dramatically impact both performance and reliability, especially when you rely on abstractions like Spring RabbitTemplate. This article is a follow-up to my previous article where async confirmations were not covered in detail.

Simplest Confirmations with Callbacks

The foundation of reliable RabbitMQ publishing starts with publisher confirmations. Here's a practical example that demonstrates how to track individual message acknowledgments:

rabbitTemplate.invoke ({ channel ->
    chunk.forEach { message ->
        channel.convertAndSend(
            RabbitMQConfig.EXCHANGE_NAME,
            RabbitMQConfig.ROUTING_KEY,
            message
        )
    }
    channel.waitForConfirmsOrDie(10_000)
},
    { deliveryTag, multiple ->
        println("ACK tag: $deliveryTag multiple: $multiple")
    },
    { deliveryTag, multiple ->
        println("NACK tag $deliveryTag multiple: $multiple")
    }
)

Key Details:

Delivery Tags: Each message gets a unique delivery tag that's scoped to the channel
Callbacks: Separate handlers for ACK (success) and NACK (failure) responses
waitForConfirms: Essential for ensuring all callbacks are triggered before proceeding and returning channel to the cache

To be honest, it's almost the same as just using waitForConfirms, but here you get some observability on failures and can theoretically even retry if you keep a mapping of message to channel + delivery tag.

Correlated Async ACK: Non-Blocking Individual Message Tracking

For granular control, correlated publisher confirmations offer asynchronous, non-blocking message tracking:

fun publishWithAsyncAck(messages: List<Message>) {
    measureTime {
        template.invoke { channel ->
            messages.forEach { message ->
                val correlationData = CorrelationData(message.id)
                pendingConfirmations[message.id] = message

                channel.convertAndSend(
                    RabbitMQConfig.EXCHANGE_NAME,
                    RabbitMQConfig.ROUTING_KEY,
                    message,
                    correlationData
                )
            }
        }
    }.let { millis ->
        println("Published ${messages.size} messages in $millis")
        println("Confirmations will arrive via callbacks")
    }
}

@Bean
fun createAsyncTemplate(): RabbitTemplate {
    val template = RabbitTemplate(connectionFactory)
    template.messageConverter = Jackson2JsonMessageConverter()
    template.setMandatory(true)

    template.setConfirmCallback { correlationData, ack, cause ->
        val messageId = correlationData?.id
        val message = messageId?.let { pendingConfirmations.remove(it) }

        if (!ack && message != null) {
            println("❌ Message ${message.id} NACKed: $cause")
            retry(message)
        }
    }

    return template
}

Advantages of Correlated Async ACK:

Non-blocking: No waitForConfirms() calls blocking the publishing thread
Individual tracking: Each message tracked by its unique correlation ID
Granular retry: Only failed messages need to be retried, not entire batches

Channel Churn: Hidden Performance Killer

One of the most critical discoveries is how channel churn can destroy publishing performance.

Performance Issue with confirm-type: correlated

// ❌ Problematic - creates channel churn
messages.forEach { message ->
    template.convertAndSend(exchange, routingKey, message) // Each call = potential new channel
}

According to the RabbitMQ Java Client API Guide, publisher confirmations are per-channel due to delivery tags being channel-scoped in the AMQP protocol:

Each convertAndSend() call may checkout a new channel from the cache
Channels with pending confirmations cannot be immediately returned to cache
High-throughput publishing can exhaust the channel cache
New channels get created when cache is full, leading to channel churn and potential OOM exceptions

// ✅ Correct - uses single channel for all operations
template.invoke { channel ->
    messages.forEach { message ->
        channel.convertAndSend(exchange, routingKey, message) // All use same channel
    }
}

The invoke() function ensures:

All operations use the same channel
Single channel handles all messages and their confirmations
Eliminates channel create/close overhead by using the same channel per batch
Prevents memory issues from excessive channel creation

Note: Using .invoke() gives no benefit if you're not using batches but just publishing one-by-one.

Publisher Confirm Types: Simple vs Correlated

The choice between simple and correlated publisher confirmation types affects both functionality and channel behavior:

Simple Confirmations

spring:
  rabbitmq:
    publisher-confirm-type: simple

You can have confirm-type: simple and still use correlation data, but you might experience an issue due to a bug in Spring RabbitTemplate: with confirm-type: simple, channels are returned to cache immediately if you don't tell them to wait for confirmations (with waitForConfirm() for example). This might be good since there's less possibility to exhaust the channel cache, but it might lead to unexpected results, so it's not recommended. Please use correlated if you want to use confirmCallback.

Correlated Confirmations

spring:
  rabbitmq:
    publisher-confirm-type: correlated

There’s no such issue when using confirm-type: correlated. However, with the default configuration - where channels are held until all confirmations are received - you may encounter performance issues due to channel churn.

Flow Summary:

Each publish operation attempts to retrieve a channel from the cache.
If all cached channels are in use, a new channel is created.
Once the operation is complete, the newly created channel cannot be returned to the cache (due to its size limit) and is therefore closed.
This repeated creation and closing of channels results in increased channel churn, negatively affecting performance. In extreme cases, it can even lead to OutOfMemory (OOM) exceptions.

Why Channel Limits Matter

A known performance consideration is limiting the number of channels in your pool. Here's why this matters:

From the Spring AMQP Documentation:

"The cache size is (by default) not a limit but is merely the number of channels that can be cached. With a cache size of, say, 10, any number of channels can actually be in use."

Unlimited vs Limited Channel Configuration

Unlimited Channels (Default):

spring:
  rabbitmq:
    cache:
      channel:
        size: 3
        checkout-timeout: 0  # Unlimited channel creation

✅ No blocking - immediate channel access
❌ Higher memory usage - unlimited channel creation
❌ Potential resource exhaustion under high load

Limited Channels:

spring:
  rabbitmq:
    cache:
      channel:
        size: 3
        checkout-timeout: 5000  # 5 seconds timeout

✅ Controlled memory usage - strict channel limits
✅ Predictable resource consumption
❌ Potential blocking - threads may wait or timeout

When to limit channels:

High-load production systems requiring resource control
When you need predictable memory consumption
Applications with many concurrent publishing threads

When to use unlimited:

Low to medium load scenarios
Development and testing environments
When thread blocking is unacceptable

Key Takeaways

Don't blindly publish and forget if you want messages to be delivered
Monitor your RabbitMQ stats
Async ACK offers the best balance
Channel limits matter

The RabbitMQ Java client's threading model, with separate I/O threads and consumer thread pools, enables these optimizations to work effectively. Understanding these patterns can transform your messaging system from a performance bottleneck into a high-throughput, reliable component.

What's Next

In the next article, based on these findings, I plan to cover the outbox publishing pattern, which was initially planned but couldn't fit into this post.

References

From Fire-and-Forget to Reliable: RabbitMQ Ack

Yevhenii Kukhol — Fri, 30 May 2025 17:38:37 +0000

How to ensure your messages actually reach RabbitMQ broker without killing performance

TL;DR

Publishing messages to RabbitMQ without confirmations is fast but unreliable - you never know if messages actually reached the broker. Simple acknowledgments solve reliability but destroy performance. Batch acknowledgments provide good performance with reliability guarantees.

Key takeaways:

Simple one-by-one ACK: Reliable but extremely slow
Batch ACK: Fast and reliable
Individual message tracking comes with trade-offs (covered in next article)

🔗 Full code examples in this repository

The Problem: Fire-and-Forget Publishing

Most developers start with RabbitMQ using the simplest approach - fire-and-forget publishing:

// Fast but unreliable - did the message actually reach the broker?
// Spoiler: no one knows
rabbitTemplate.convertAndSend(
    "my.exchange", 
    "routing.key", 
    message
)

This approach is blazing fast, but there's a critical problem: you have no idea if your messages actually reached the RabbitMQ broker.

What Can Go Wrong?

Network failures: Message lost in transit
Broker downtime: RabbitMQ unavailable during publish
Resource limits: Broker rejects due to memory/disk constraints
Connection drops: Network hiccups during publishing

In production systems, losing messages is often unacceptable. You need reliability.

The Solution: Publisher Confirms (Simple ACK)

RabbitMQ provides publisher confirms - a mechanism where the broker acknowledges receipt of each message. Here's how to implement it:

Step 1: Configure RabbitMQ

Enable publisher confirms in your RabbitMQ configuration:

spring:
  rabbitmq:
    publisher-confirm-type: simple

Step 2: Use RabbitTemplate.invoke

Instead of direct convertAndSend, use invoke to access the underlying channel:

@Service
class ReliablePublisher(
    private val rabbitTemplate: RabbitTemplate
) {
    fun publishWithConfirmation(message: Message) {
        rabbitTemplate.invoke { channel ->
            // Send the message
            channel.convertAndSend(
                "my.exchange",
                "routing.key", 
                message
            )

            // Wait for broker confirmation
            channel.waitForConfirmsOrDie(10_000)
        }
    }
}

What Happens Behind the Scenes

Message sent: Publisher sends message to broker
Broker processes: RabbitMQ receives and routes the message
Confirmation sent: Broker sends ACK back to publisher
Publisher continues: Only after ACK is received

This guarantees your message reached the broker safely!

The Performance Problem with Simple ACK

While simple ACK solves reliability, it creates a massive performance bottleneck:

// This is SLOW - each message waits for individual confirmation
repeat(1_000_000) { index ->
    rabbitTemplate.invoke { channel ->
        channel.convertAndSend(exchange, routingKey, createMessage(index))
        channel.waitForConfirmsOrDie(10_000) // Wait for EACH message
    }
}

Result: 1 million messages take 14.5 minutes 😱

Why Is It So Slow?

Each message requires a full round-trip to the broker:

Send message → Wait for ACK → Send next message → Wait for ACK → ...

The network latency kills performance. Even with 1ms round-trip time:

1,000,000 messages × 1ms = 1,000 seconds = 16+ minutes

The Batch Solution: Best of Both Worlds

The solution is batch acknowledgments - send multiple messages, then wait for confirmation once:

@Service
class BatchAckPublisher(
    private val rabbitTemplate: RabbitTemplate
) {
    fun publishBatch(messages: List<Message>) {
        rabbitTemplate.invoke { channel ->
            // Send ALL messages first
            messages.forEach { message ->
                channel.convertAndSend(
                    "my.exchange",
                    "routing.key",
                    message
                )
            }

            // Wait for confirmation of ALL messages at once
            channel.waitForConfirmsOrDie(10_000)
        }
    }
}

Key Concepts of Batch Confirmation

Channel State: RabbitMQ tracks unconfirmed messages per channel
Bulk Confirmation: waitForConfirms confirms ALL messages since last call
Atomic Operation: Either all messages in batch succeed or fail together
Reduced Round-trips: Dramatically fewer network calls

Implementation Example

Here's the actual implementation from RmqAckPublisher.kt:

fun simpleAckPublish(times: Int = 1_000_000) {
    val messages = generateMessages(times)

    measureTime {
        messages.chunked(10_000).forEach { chunk ->
            rabbitTemplate.invoke { channel ->
                chunk.forEach { message ->
                    channel.convertAndSend(
                        RabbitMQConfig.EXCHANGE_NAME,
                        RabbitMQConfig.ROUTING_KEY,
                        message
                    )
                }
                // Confirm entire chunk at once
                channel.waitForConfirmsOrDie(10_000)
            }
        }
    }.let { duration ->
        println("Published 1M messages in $duration")
    }
}

Batch size matters: We use 10k messages per batch for testing purposes. Please donʼt use same values in production.

Performance Comparison

Here are the actual benchmark results from our testing:

Method	Messages	Strategy	Time	Performance Gain
Fire-and-Forget	1M	No confirmation	~12.5s	Baseline
Simple ACK	1M	Per message	~14.5m	70x slower
Batch ACK	1M	Per 10k batch	~17s	50x faster than simple ACK

Why Batch ACK Is So Much Faster

Reduced network calls: 100 confirmations instead of 1,000,000
Better channel utilization: Channel stays busy sending messages
Bulk operations: RabbitMQ optimizes bulk confirmations internally

The batch approach gives you almost fire-and-forget performance but with reliability!

The Limitations of Batch Synchronous ACK

While batch ACK solves the performance problem, it introduces new challenges:

1. No Individual Message Tracking

try {
    channel.waitForConfirmsOrDie(10_000)
    // Success! But which specific messages were confirmed? 🤷‍♂️
} catch (Exception e) {
    // Failure! But which specific messages failed? 🤷‍♂️
    // Must republish entire batch of 10,000 messages
}

2. All-or-Nothing Error Handling

If one message in a 10k message batch fails:

You don't know which message failed
Must retry the entire batch
Wastes resources on successful messages

3. Still Blocking

The channel blocks during waitForConfirms():

Cannot send other messages
Reduced overall throughput
Poor resource utilization

4. Limited Error Recovery

// Batch fails - now what?
messages.chunked(10_000).forEach { chunk ->
    try {
        publishBatch(chunk)
    } catch (Exception e) {
        // Can only retry ALL 10,000 messages
        // No granular retry possible
        retryEntireBatch(chunk) // Inefficient!
    }
}

What's Next?

Batch acknowledgments provide excellent performance with reliability, but the lack of individual message tracking limits error recovery strategies.

In the next article, we'll explore asynchronous acknowledgments with correlation data - a technique that provides:

✅ Individual message tracking
✅ Non-blocking performance
✅ Granular error recovery
✅ Efficient retry mechanisms

We'll dive into:

Implementing correlated publisher confirms
Handling ACK/NACK callbacks asynchronously

Only way to learn: Try It Yourself

🔗 Clone the repository and run the examples:

git clone https://github.com/Eragoo/rmq-lab
cd rmq-lab/rmq.spring.publisher

# Start RabbitMQ
docker run -d --name rabbitmq -p 5672:5672 -p 15672:15672 rabbitmq:3-management

# Run the examples
./gradlew bootRun

Compare the performance and see the difference between simple and batch acknowledgments!

Key Takeaways

Fire-and-forget is fast but unreliable - great for development, risky for production
Simple ACK provides reliability but kills performance - 70x slower than fire-and-forget
Batch ACK is the sweet spot - almost fire-and-forget speed with full reliability
Batch size matters - find the best value for your setup
Individual tracking requires different approaches - stay tuned for the async solution!

📚 Further Reading:

RabbitMQ Publisher Confirms Documentation
Coming next: "Asynchronous RabbitMQ Confirmations: Individual Message Tracking Without Performance Loss"

How is your data actually flushed? Hibernate ActionQueue event priorities.

Yevhenii Kukhol — Sat, 23 Sep 2023 17:23:06 +0000

We have a problem🤔

Imagine you're working with a default project using Spring Data JPA. Here's the entity in question:

@Entity
@NoArgsConstructor
@AllArgsConstructor
@Getter
@Setter
public class Person {
    private static final String GENERATOR = "person_generator";

    @Id
    @GeneratedValue(strategy = GenerationType.SEQUENCE, generator = GENERATOR)
    @SequenceGenerator(name = GENERATOR, sequenceName = "person_sequence")
    private Long id;
    private String name;
    @Column(unique = true)
    private String email;
}

What's the output of the following code?

    @Transactional
    public void someImportantOperation(Long personId) {
        var oldPerson = personRepository.findById(personId)
                .orElseThrow();
        var oldPersonEmail = oldPerson.getEmail();

        personRepository.delete(oldPerson);

        var newPerson = new Person(
                null, "New Person With The Same Email", oldPersonEmail
        );

        personRepository.save(newPerson);
    }

1... 2... 3...
Exception!

ERROR: duplicate key value violates unique constraint "person_email_uniq"
  Detail: Key (email)=(mail@gmail.com) already exists.

Q:
But why? I've just deleted the oldPerson!
A:
It's because Hibernate executes SQL in a specific order!😎

Explanation

Try to delve into Hibernate's code: in methods like Session.persist, Session.remove, and others. For example in org.hibernate.internal.SessionImpl#delete you will find:

    @Override
    public void delete(Object object) throws HibernateException {
        checkOpen();
        fireDelete( new DeleteEvent( object, this ) );
    }

No actual SQL is constructed or executed! Hibernate simply triggers a DeleteEvent!
The same happens with ALL other operations (except READ operations)

These events need processing, right?

The logic for handling those events is related to the class org.hibernate.engine.spi.ActionQueue.

There are two facts about it:

These events are handled ONLY on Transaction Commit or Flush operations.
These events are not handled in the order they were added to the ActionQueue. That is why we had a DataIntegrityViolationException in the example above.

Hibernate documentation states:

The ActionQueue executes all operations in the following order:

OrphanRemovalAction

EntityInsertAction or EntityIdentityInsertAction

EntityUpdateAction

QueuedOperationCollectionAction

CollectionRemoveAction

CollectionUpdateAction

CollectionRecreateAction

EntityDeleteAction

So, it means that in the example above (from the first section), Hibernate should:

Do select in order to get oldPerson
Insert newPerson. Because EntityInsertAction has a higher priority.
Delete oldPerson. Because EntityDeleteAction has the lowest priority.

And... Here is the SQL log that proves it!

Hibernate: select p1_0.id,p1_0.email,p1_0.name from person p1_0 where p1_0.id=?
Hibernate: select nextval('person_sequence')
Hibernate: insert into person (email,name,id) values (?,?,?)
WARN 60977 --- [           main] o.h.engine.jdbc.spi.SqlExceptionHelper   : SQL Error: 0, SQLState: 23505
ERROR 60977 --- [           main] o.h.engine.jdbc.spi.SqlExceptionHelper   : ERROR: duplicate key value violates unique constraint "person_email_uniq"
  Detail: Key (email)=(mail@gmail.com) already exists.

If you're a truly patient person, you can always delve into the ActionQueue code, set breakpoints, and DYOR (do your own research)!

What's in it for me?

Now you can flex in technical interviews that you're a TOTAL PRO EXPERT in Hibernate 💪
You understand that Hibernate is asynchronous 🤓
You recognize that Hibernate executes SQL commands in a specific order, and you can use this knowledge in your apps to avoid tricky situations like the ones shown at the beginning of the article 😏

Bonus: Why ActionQueue have such a specific order of operations?

Let's break these operations down:

OrphanRemovalAction
Used when you delete something from the @OneToMany(orphanRemoval = true) association. More about it here: link.
EntityInsertAction or EntityIdentityInsertAction
Insertions are prioritized due to the Hibernate developers' suggestion that it's more consistent to create entities at the beginning. This ensures that any subsequent operations that refer to this entity will find it in the database.
For example: if you prioritize delete/update, you might attempt to delete/update something that hasn't been created yet.
EntityUpdateAction
There are no comments necessary – it seems logical to run update queries after creating them. This guarantees that all entities you're attempting to update exist in the database.
QueuedOperationCollectionAction
This action refers to all collection-related queued operations (not entity-related!). For these actions to occur, your entity should own that collection relation on the JPA level.
CollectionRemoveAction
Try to verify this one yourself!🤪
CollectionUpdateAction
Try to verify this one yourself!🤪
CollectionRecreateAction
This concerns dropping and recreating a collection in its entirety. This might be necessary when the collection's structure has changed significantly. As this could be a resource-intensive operation, it's prioritized after other more granular collection operations.
EntityDeleteAction
Delete the entity. It's likely positioned with the lowest priority to avoid situations where you reference a non-existent entity. So it seems logical.

That's it! Thank you for the reading!

PS: If you find something wrong, please correct me in the comments!

My links:
Linkedin
GitHub (mostly dead)

DEV Community: Yevhenii Kukhol

Outbox Pattern: RabbitMQ Publishing Strategies for High-Performance Systems

Table of Contents

Why Outbox Publishing?

Outbox: Table + Scheduled Publishing

What is the Outbox Pattern?

Why Focus on Scheduled Publishing?

Database Side: Already Solved

Potential Publishing Incidents: What Goes Wrong Under Load

1. Channel Exhaustion (OutOfMemoryError)

2. Publishing Bottlenecks

3. Silent Message Loss

Understanding Publisher Confirms: Foundation for Reliability

Strategy 1: Fire and Forget (Simple but Risky)

When Fire-and-Forget Makes Sense

Implementation

Key Characteristics

Strategy 2: Synchronous Batch ACK

Why Synchronous Batch ACK is Ideal for Outbox

Core Implementation

Configuration for Sync Batch ACK

Key Characteristics

Advantages for Outbox Pattern

Strategy 3: Async ACK with Correlation (Complex but High Throughput)

Critical Issues with Async ACK for Outbox

Why Async ACK is Problematic for Outbox

When Async ACK Makes Sense

Async Implementation (Use with Caution)

Required Infrastructure for Async ACK

Configuration for Async ACK

Key Characteristics

Channel Churn Pitfall: Critical for Both Strategies

Understanding the Problem

Solution: Publish Batch with Same Channel

Why invoke() Works

Additional Channel Management

Monitoring and Observability

Key Metrics

Conclusion

Key Takeaways

From Fire-and-Forget to Reliable: RabbitMQ Ack [pt. 2]

Simplest Confirmations with Callbacks

Correlated Async ACK: Non-Blocking Individual Message Tracking

Channel Churn: Hidden Performance Killer

Performance Issue with confirm-type: correlated

Publisher Confirm Types: Simple vs Correlated

Simple Confirmations

Correlated Confirmations

Why Channel Limits Matter

Unlimited vs Limited Channel Configuration

Key Takeaways

What's Next

References

From Fire-and-Forget to Reliable: RabbitMQ Ack

TL;DR

The Problem: Fire-and-Forget Publishing

What Can Go Wrong?

The Solution: Publisher Confirms (Simple ACK)

Step 1: Configure RabbitMQ

Step 2: Use RabbitTemplate.invoke

What Happens Behind the Scenes

The Performance Problem with Simple ACK

Why Is It So Slow?

The Batch Solution: Best of Both Worlds

Key Concepts of Batch Confirmation

Implementation Example

Performance Comparison

Why Batch ACK Is So Much Faster

The Limitations of Batch Synchronous ACK

1. No Individual Message Tracking

2. All-or-Nothing Error Handling

3. Still Blocking

4. Limited Error Recovery

What's Next?

Only way to learn: Try It Yourself

Key Takeaways

How is your data actually flushed? Hibernate ActionQueue event priorities.

We have a problem🤔

Explanation

These events need processing, right?