Kafka Ordering: The Guarantee Is Narrower Than You Think

Every Kafka tutorial says the same thing: messages in a partition are ordered. Every production incident report says the same thing too, usually right before explaining how messages arrived out of order anyway. Both statements are true. The gap between them is where most teams get hurt.

Kafka's ordering guarantee is real, but it is narrow. It applies to one partition, for one producer writing in sequence, read by one consumer processing in sequence. Change any part of that sentence and the guarantee changes with it. This article walks through exactly where ordering holds, where it breaks, and how to design around both.

What Kafka actually guarantees

A Kafka topic is split into partitions. Each partition is an append-only log, and every message in it gets a monotonically increasing offset. A consumer reading one partition sees messages in offset order, always. That is the whole guarantee, and it is a strong one.

What it does not say: nothing about order across partitions. If your topic has twelve partitions and you publish two related events, they can land in different partitions and arrive in any relative order. There is no global order in Kafka, and there never will be, because global order would require global coordination, which would destroy the throughput that makes Kafka worth using.

The practical rule follows directly: if two events must be processed in order, they must go to the same partition. The mechanism for that is the message key. Kafka hashes the key and assigns the message to a partition deterministically, so every message with the same key lands in the same partition, in publish order.

// The key is the ordering decision. Same key, same partition, same order.
msg := &kafka.Message{
    TopicPartition: kafka.TopicPartition{Topic: &topic, Partition: kafka.PartitionAny},
    Key:            []byte(orderID),
    Value:          eventJSON,
}
err := producer.Produce(msg, nil)

Choose the key at the granularity of the entity whose order matters: an order ID, a user ID, an account number. Not a timestamp, not a random UUID, not a constant. A random key scatters related events across partitions and quietly deletes your ordering. A constant key puts everything in one partition and deletes your parallelism. The key is a design decision, not a formality.

Where ordering breaks in practice

The first break is retries. A producer that retries a failed send can reorder messages unless idempotence is enabled. With enable.idempotence=true, the producer attaches a sequence number per partition and the broker rejects duplicates and out-of-sequence writes. Without it, a retried message can land after a newer one. Turn idempotence on and leave it on; it costs almost nothing in modern Kafka.

The second break is the consumer. Ordering on the wire means nothing if the consumer processes messages concurrently. A consumer that hands each message to a goroutine pool, or a consumer group where two members split a partition's work, will process out of order no matter what the log says. If order matters, process one partition's messages sequentially in one goroutine. Parallelism comes from partitions, not from threads inside one partition.

// One goroutine per assigned partition: order preserved, parallelism kept.
for msg := range consumerChannel {
    handle(msg)          // sequential, in offset order
    consumer.Commit(msg) // commit only after successful handling
}

The third break is rebalancing. When a consumer joins or leaves the group, partitions move between members. During the move, the new owner starts from the last committed offset, and any message processed but not yet committed gets processed again. Reprocessing is not reordering in the strict sense, but your downstream system sees an old event after a newer one, which looks exactly like reordering. The fix is the same one that fixes most Kafka pain: make consumers idempotent. Track processed message IDs, or design handlers so that applying the same event twice is harmless.

The fourth break is the one people forget: partition count changes. Adding partitions to a topic rehashes keys, so a key that used to live in partition three may move to partition nine. Events published before and after the change can be consumed out of order across the boundary. Never increase the partition count of a topic where ordering matters without a migration plan, and prefer to over-provision partitions at creation time instead.

Designing for the order you actually need

Most systems do not need global order. They need order per entity, and that is exactly what Kafka sells. Design around it explicitly.

First, name the ordering unit in your design document: "events for one order are ordered; events across orders are not." Write that sentence down, because every future bug report will test it.

Second, size partitions for both throughput and parallelism. A topic with three partitions gives you at most three consumers working in parallel per group. If you need twelve parallel consumers later, you need twelve partitions now, because adding them later breaks key placement. A common starting point is one partition per expected consumer, rounded up, with headroom for growth.

Third, treat the consumer as part of the ordering contract. Code review should flag any consumer that processes a partition's messages concurrently when order matters, any consumer that commits before processing, and any handler that is not idempotent. These three review checks catch nearly every ordering incident before it ships.

Fourth, test ordering the way you test correctness. Publish a sequence of events for one key from a producer with retries enabled, kill and restart consumers mid-stream, force a rebalance, and assert that the downstream state matches the event sequence exactly. If your test only publishes ten messages to one partition with no failures, it is testing the happy path that never pages anyone.

The honest summary

Kafka gives you ordered logs, one per partition, and leaves the rest to you. The guarantee is narrow by design: narrow guarantees are the ones a system can actually keep at scale. Your job is to map your ordering needs onto partitions with keys, keep consumers sequential per partition, enable idempotence on both sides, and never change partition counts casually. Do those four things and Kafka ordering stops being a footgun. Skip any one of them and the next incident report will explain why, in detail, at 3 AM.

A worked example: order events for one checkout

Make it concrete. A checkout produces three events: payment authorized, inventory reserved, confirmation sent. These must be processed in that order for one order, but orders are independent of each other. The key is the order ID. All three events for order 8471 land in the same partition, in order, and a single consumer goroutine processes them sequentially. Meanwhile, order 8472's events live in a different partition, processed in parallel by another consumer, with no coordination between the two. Throughput scales with partitions; correctness scales with keys.

Now break one rule and watch what happens. Suppose the consumer processes each event in its own goroutine to "go faster." The confirmation handler runs before the payment handler finishes, and a customer gets a confirmation for an order whose payment then fails. The log was ordered. The processing was not. The incident report will blame Kafka, but Kafka kept its promise. The consumer broke it.

The same example shows why idempotence matters. If the consumer crashes after reserving inventory but before committing the offset, the new consumer replays the payment event. A handler that charges again on replay double-charges the customer. A handler that checks "has this payment event ID been applied already" replays harmlessly. Ordering plus idempotence is the complete contract. Either one alone is half a system.