Most comparisons between Kafka and RabbitMQ read like a real estate listing. Kafka: high throughput, distributed log, great for streaming. RabbitMQ: flexible routing, low latency, great for task queues. Pick the one that sounds like your problem. The comparison table ends, you make a choice, and six months later you are in an incident call trying to explain to your team why the system is behaving in a way nobody anticipated.
The comparison table approach fails because it describes what each system does well and leaves out what each system does when things go wrong. That is the only part that matters in production. The previous piece in this series introduced a four-question framework for trade-off thinking: what are we optimising for, what are we sacrificing, what breaks first, and how do we recover. This piece applies that framework to Kafka and RabbitMQ specifically, because the failure modes of these two systems are not just different in degree, they are different in kind, and choosing the wrong one means you inherit a class of problems your architecture was not designed to handle.
The thing each system is actually optimising for
Kafka is optimised for retaining events and serving them to multiple independent consumers at high throughput. The core data structure is an append-only log. Producers write to the end of the log. Consumers read from whatever offset they last committed. The log persists based on a retention policy you configure. Nothing gets deleted on consumption. Multiple consumer groups can read the same log independently, each maintaining its own position, completely unaware of each other.
RabbitMQ is optimised for routing work to the right consumer and confirming that the work was done. The core data structure is a queue. Producers publish to an exchange. The exchange applies routing rules and pushes messages to one or more queues. Consumers pull from queues and send acknowledgements. When a consumer acknowledges a message, the broker deletes it. The queue exists to be emptied.
These are not different implementations of the same idea. They are different ideas. Kafka is a log that happens to support consumption. RabbitMQ is a work dispatcher that happens to buffer messages while workers are busy. Conflating them because both involve producers and consumers sending data is roughly like conflating a database with a cache because both store data and respond to queries.
Top comments (0)