In earlier versions of Kafka exactly-once semantics static mapping should have between transaction id and topic partitions during consumer group mismatch there are chances that transaction id gets different topic partition.
To avoid such a scenario KIP-447: Producer scalability for exactly once semantics was implemented, what I was understood from the KIP-477 that the old producer fenced using the fetch offset call with the help of a new API(sendOffsetToProdcuer) so transactio.id not used for fencing.
But my doubt here is,
still transactional producer expect transaction.id how should I choose this value for the latest Kafka version?
transaction.id should have a static mapping with partitions, fetch offset fencing take effect only during consumer group rebalancing?
Is this value invalid for the latest version?
Please help me with this I am trying to understand Kafka EoS and implement it in the production system.
Since you tagged this with spring-kafka, I assume you are using it; the transactional.id can now be different for each instance (as it was previously required for producer-only transactions). There is no longer a need to tie the id to the group/topic/partition, and a much smaller number of producers is needed.
See https://docs.spring.io/spring-kafka/docs/current/reference/html/#exactly-once
The broker needs to be 2.5 or later.