apache-kafka

Who guarantees per-partition ordering in Kafka Producer with enable.idempotence=true? Producer or Broker?

I found out that in Kafka Producer, when enable.idempotence=true and max.in.flight.requests.per.connection <= 5, the order of messages is guaranteed within a single partition. However, I'm confused whether this order guarantee is enforced by the Broker or by the Producer.

Many sources, including Stack Overflow, mention OutOfOrderSequenceException and suggest that message batch ordering is guaranteed by the broker. But based on my research, this seems incorrect. It appears that the Producer primarily ensures the ordering, and the broker's role is more of a secondary safeguard.

For example, if you look at the sendProducerData method in the Sender class, you can find the following logic:

// Create produce requests
Map<Integer, List<ProducerBatch>> batches = this.accumulator.drain(metadataSnapshot, result.readyNodes, this.maxRequestSize, now);
addToInflightBatches(batches);

if (guaranteeMessageOrder) {
    // Mute all the partitions drained
    for (List<ProducerBatch> batchList : batches.values()) {
        for (ProducerBatch batch : batchList)
            this.accumulator.mutePartition(batch.topicPartition);
    }
}

Here, after draining batches, the producer mutes all the drained partitions. (“Muting” means temporarily not draining new batches for that partition.)

This suggests that when enable.idempotence=true and max.in.flight.requests.per.connection <= 5, the Producer ensures that only one batch per partition is in-flight at a time, thereby preserving the order at the producer side.

Additionally, you might wonder if multiple batches for the same partition could be drained at once. However, this doesn’t seem to happen either. If you check the drainBatchesForOneNode method in RecordAccumulator, you can confirm that only one batch per partition is drained at a time.

In conclusion, when enable.idempotence=true and max.in.flight.requests.per.connection <= 5, it is correct to say that the Producer ensures ordering per partition. The broker’s role is more about defending against anomalies, like detecting out-of-order sequences.

I’m curious to hear other people’s opinions on this as well.

Solution

  • You are indeed correct in saying that the Producer guarantees ordering per partition.

    The Broker must be simple, "dumb", if you want to call it. Kafka pushes complexity (like idempotence, ordering) to the clients.

    When it comes to message ordering in Kafka, it's really the producer doing all the heavy lifting. If you turn on enable.idempotence and keep your in-flight requests low (≤ 5), the producer makes sure that messages for each partition stay in order, just like in a queue.

    The broker? It's just sitting there, taking messages as they come and writing them down in the order it gets them. Simple, fast, and no fancy reordering magic.

    We may think at the beginning that the broker is smart enough to manage ordering, but nope; Kafka's design is all about keeping the broker "dumb" but very, very fast, and letting the clients (like the producer) handle this stuff.

    With transactions, we are giving to our producer a promise: “Hey, I’ll either deliver this whole batch perfectly, or I won’t deliver anything at all.”

    Not to the broker, but to the producer.

    TL;DR