apache-kafkaspring-kafkakafka-transactions-api

Does it make sense to have a single send inside a kafka transaction using spring kafka?


From kafka 3.0 we get strongest delivery guarantees by default (acks=all, enable.idempotence=true). ( https://www.confluent.io/blog/apache-kafka-3-0-major-improvements-and-new-features/ ).

Mostly, transactions mean atomicity ( and in sql, isolation, etc ).

Having strongly delivery guarantee, I can send a message in kafka either as:

kafkaTemplate.send("my_topic", "my value");

or

kafkaTemplate.executeInTransaction(kt -> kt.send("my_topic", "my value"));

but is there any difference if I only have a single send inside?

I would see the sense of a transaction if I would want to send atomically more messages like:

List<String> values = Arrays.asList("dog", "cat", "horse", "zebra");
kafkaTemplate.executeInTransaction(kt -> {
    values.forEach(v -> kt.send("my_topic", v));
    return null;
});

Solution

  • If you only have a single send scenario, then using transactions to publish might not be suitable. You can accomplish it relying on other gurantees (acks=all, sync producer etc.). Here is a blog that mentions this fact: https://spring.io/blog/2023/09/28/producer-initiated-transactions-in-spring-cloud-stream-kafka-applications.

    Transactions make the most sense, when you have multiple records to publish as part of a single send or transactionally publish to multiple topics.