From Fire-and-Forget to Reliable: RabbitMQ Ack

How to ensure your messages actually reach RabbitMQ broker without killing performance

TL;DR

Publishing messages to RabbitMQ without confirmations is fast but unreliable - you never know if messages actually reached the broker. Simple acknowledgments solve reliability but destroy performance. Batch acknowledgments provide good performance with reliability guarantees.

Key takeaways:

• Simple one-by-one ACK: Reliable but extremely slow
• Batch ACK: Fast and reliable
• Individual message tracking comes with trade-offs (covered in next article)

🔗 Full code examples in this repository

---

The Problem: Fire-and-Forget Publishing

Most developers start with RabbitMQ using the simplest approach - fire-and-forget publishing:

// Fast but unreliable - did the message actually reach the broker?
// Spoiler: no one knows
rabbitTemplate.convertAndSend(
    "my.exchange", 
    "routing.key", 
    message
)

This approach is blazing fast, but there's a critical problem: you have no idea if your messages actually reached the RabbitMQ broker.

What Can Go Wrong?

• Network failures: Message lost in transit
• Broker downtime: RabbitMQ unavailable during publish
• Resource limits: Broker rejects due to memory/disk constraints
• Connection drops: Network hiccups during publishing

In production systems, losing messages is often unacceptable. You need reliability.

---

The Solution: Publisher Confirms (Simple ACK)

RabbitMQ provides publisher confirms - a mechanism where the broker acknowledges receipt of each message. Here's how to implement it:

Step 1: Configure RabbitMQ

Enable publisher confirms in your RabbitMQ configuration:

spring:
  rabbitmq:
    publisher-confirm-type: simple

Step 2: Use RabbitTemplate.invoke

Instead of direct convertAndSend, use invoke to access the underlying channel:

@Service
class ReliablePublisher(
    private val rabbitTemplate: RabbitTemplate
) {
    fun publishWithConfirmation(message: Message) {
        rabbitTemplate.invoke { channel ->
            // Send the message
            channel.convertAndSend(
                "my.exchange",
                "routing.key", 
                message
            )

            // Wait for broker confirmation
            channel.waitForConfirmsOrDie(10_000)
        }
    }
}

What Happens Behind the Scenes

1. Message sent: Publisher sends message to broker
2. Broker processes: RabbitMQ receives and routes the message
3. Confirmation sent: Broker sends ACK back to publisher
4. Publisher continues: Only after ACK is received

This guarantees your message reached the broker safely!

---

The Performance Problem with Simple ACK

While simple ACK solves reliability, it creates a massive performance bottleneck:

// This is SLOW - each message waits for individual confirmation
repeat(1_000_000) { index ->
    rabbitTemplate.invoke { channel ->
        channel.convertAndSend(exchange, routingKey, createMessage(index))
        channel.waitForConfirmsOrDie(10_000) // Wait for EACH message
    }
}

Result: 1 million messages take 14.5 minutes 😱

Why Is It So Slow?

Each message requires a full round-trip to the broker:

1. Send message → Wait for ACK → Send next message → Wait for ACK → ...

The network latency kills performance. Even with 1ms round-trip time:

• 1,000,000 messages × 1ms = 1,000 seconds = 16+ minutes

---

The Batch Solution: Best of Both Worlds

The solution is batch acknowledgments - send multiple messages, then wait for confirmation once:

@Service
class BatchAckPublisher(
    private val rabbitTemplate: RabbitTemplate
) {
    fun publishBatch(messages: List<Message>) {
        rabbitTemplate.invoke { channel ->
            // Send ALL messages first
            messages.forEach { message ->
                channel.convertAndSend(
                    "my.exchange",
                    "routing.key",
                    message
                )
            }

            // Wait for confirmation of ALL messages at once
            channel.waitForConfirmsOrDie(10_000)
        }
    }
}

Key Concepts of Batch Confirmation

1. Channel State: RabbitMQ tracks unconfirmed messages per channel
2. Bulk Confirmation: waitForConfirms confirms ALL messages since last call
3. Atomic Operation: Either all messages in batch succeed or fail together
4. Reduced Round-trips: Dramatically fewer network calls

Implementation Example

Here's the actual implementation from RmqAckPublisher.kt:

fun simpleAckPublish(times: Int = 1_000_000) {
    val messages = generateMessages(times)

    measureTime {
        messages.chunked(10_000).forEach { chunk ->
            rabbitTemplate.invoke { channel ->
                chunk.forEach { message ->
                    channel.convertAndSend(
                        RabbitMQConfig.EXCHANGE_NAME,
                        RabbitMQConfig.ROUTING_KEY,
                        message
                    )
                }
                // Confirm entire chunk at once
                channel.waitForConfirmsOrDie(10_000)
            }
        }
    }.let { duration ->
        println("Published 1M messages in $duration")
    }
}

Batch size matters: We use 10k messages per batch for testing purposes. Please donʼt use same values in production.

---

Performance Comparison

Here are the actual benchmark results from our testing:

Method	Messages	Strategy	Time	Performance Gain
Fire-and-Forget	1M	No confirmation	~12.5s	Baseline
Simple ACK	1M	Per message	~14.5m	70x slower
Batch ACK	1M	Per 10k batch	~17s	50x faster than simple ACK

Why Batch ACK Is So Much Faster

• Reduced network calls: 100 confirmations instead of 1,000,000
• Better channel utilization: Channel stays busy sending messages
• Bulk operations: RabbitMQ optimizes bulk confirmations internally

The batch approach gives you almost fire-and-forget performance but with reliability!

---

The Limitations of Batch Synchronous ACK

While batch ACK solves the performance problem, it introduces new challenges:

1. No Individual Message Tracking

try {
    channel.waitForConfirmsOrDie(10_000)
    // Success! But which specific messages were confirmed? 🤷‍♂️
} catch (Exception e) {
    // Failure! But which specific messages failed? 🤷‍♂️
    // Must republish entire batch of 10,000 messages
}

2. All-or-Nothing Error Handling

If one message in a 10k message batch fails:

• You don't know which message failed
• Must retry the entire batch
• Wastes resources on successful messages

3. Still Blocking

The channel blocks during waitForConfirms():

• Cannot send other messages
• Reduced overall throughput
• Poor resource utilization

4. Limited Error Recovery

// Batch fails - now what?
messages.chunked(10_000).forEach { chunk ->
    try {
        publishBatch(chunk)
    } catch (Exception e) {
        // Can only retry ALL 10,000 messages
        // No granular retry possible
        retryEntireBatch(chunk) // Inefficient!
    }
}

---

What's Next?

Batch acknowledgments provide excellent performance with reliability, but the lack of individual message tracking limits error recovery strategies.

In the next article, we'll explore asynchronous acknowledgments with correlation data - a technique that provides:

• ✅ Individual message tracking
• ✅ Non-blocking performance
• ✅ Granular error recovery
• ✅ Efficient retry mechanisms

We'll dive into:

• Implementing correlated publisher confirms
• Handling ACK/NACK callbacks asynchronously

---

Only way to learn: Try It Yourself

🔗 Clone the repository and run the examples:

git clone https://github.com/Eragoo/rmq-lab
cd rmq-lab/rmq.spring.publisher

# Start RabbitMQ
docker run -d --name rabbitmq -p 5672:5672 -p 15672:15672 rabbitmq:3-management

# Run the examples
./gradlew bootRun

Compare the performance and see the difference between simple and batch acknowledgments!

---

Key Takeaways

1. Fire-and-forget is fast but unreliable - great for development, risky for production
2. Simple ACK provides reliability but kills performance - 70x slower than fire-and-forget
3. Batch ACK is the sweet spot - almost fire-and-forget speed with full reliability
4. Batch size matters - find the best value for your setup
5. Individual tracking requires different approaches - stay tuned for the async solution!

---

📚 Further Reading:

• RabbitMQ Publisher Confirms Documentation
• Coming next: "Asynchronous RabbitMQ Confirmations: Individual Message Tracking Without Performance Loss"