[SOLVED] If idempotency is true and the number of retries is set to 3, will there be no conflicts?

If idempotency is true and the number of retries is set to 3, will there be no conflicts?

If idempotency is true in your Kafka producer configuration and you set the number of retries to 3, won't there be a conflict?

As far as I know, idempotency sets a unique message ID for a new message ID and prevents resending with the same message ID, SEQ, ACK.

If message delivery fails (ACK or SEQ fail), the producer sends the message back to kafka broker using the same ACK, SEQ, and message ID. However, when idempotency is set to true, producers will ignore the messages which have the same ACK, SEQ, and message ID.

Or maybe setting idempotency to true in a Kafka producer just makes the producer using same ID, ACK, SEQ when message failed.

Am I understanding this concept correctly? I'm a little confused :(

Solution

I found the exact concept of idempotency

Idempotency is primarily about ensuring that retries don't accidentally lead to duplicate messages with different message IDs. It helps maintain data integrity and prevent message duplication.

So if you set idempotency to true, message retries will send exactly same message ID.

if you set idempotency to false, message retries will generate new message ID and resend with new message ID.