[SOLVED] In Kafka, if a message is manually sent to a specific partition, will subsequent messages with the same key go the same partition?

In Kafka, if a message is manually sent to a specific partition, will subsequent messages with the same key go the same partition?

Consider the following sequence:

Message1 with key "A" is sent to a specific partition of a topic:
kafkaProducer.send("my-topic", 0, "A", "Message content");
Message2 with also key "A" is sent to the same topic without specifying the partition
kafkaProducer.send("my-topic", "A", "Message content 2");

Question: Will Message2 be written to the same partition as Message1 (0)?

Actually, I have the following use case. There exist no stable ID for a message type. Under certain circumstances, the ID may change. In the act of changing the ID, I have both the old an the new ID. I want the the old and the new ID go to the same partition. My approach is as follows: when I have a Message with the old an the new ID, I determine the partition the old ID would go. Then I manually set the partition for the new ID. Subsequent Messages the the new ID then go to the same partition (when the question above will be answered with yes.)

Thank you

Solution

Will Message2 be written to the same partition as Message1

Only if the hashing algorithm used by the default partitioner decides to.

If you have an unstable ID, then it would make sense to write your own partitioner, not hardcode the value in the send or ProducerRecord constructor