springspring-kafka

Why does the last record's success determine Kafka transaction commit in Spring Kafka?

I'm using Spring Kafka 3.0.5 with the KafkaTransactionManager and DefaultErrorHandler.

In my consumer configuration:

Ack mode is set to BATCH

Listener type is @KafkaListener (record-based)

Transactions are enabled (KafkaTransactionManager)

Retry is configured with DefaultErrorHandler.setRetryListeners(...) and 10 retries

While testing with a batch of messages, I observed the following:

If the last record in a polled batch fails, the whole transaction is rolled back and the offset is not committed — as expected.

If the last record succeeds, even if previous records in the batch had failed but retried successfully, the entire transaction commits, and the consumer lag becomes 0.

This means the final outcome of the last record in the batch seems to determine whether the transaction is committed or not.

Questions:

Is this behavior expected — that the last record's result decides the transaction commit?

Is this documented anywhere in Spring Kafka or Kafka's own transactional behavior?

Is there a way to make transaction commit more explicit per record or decouple it from the final record status?

Any clarification or documentation links would be greatly appreciated.

Solution

  • I found the answer.

    When using Kafka transactions, the acknowledge mode (AckMode) does not actually affect the behavior.

    See this discussion for reference:

    Spring Kafka Consumer ACKMODE & Producer buffering for Kafka transactions

    Even in the TransactionTemplate, it simply executes the code and then commits — there's no coordination with AckMode.

    So regardless of whether you set AckMode.BATCH or AckMode.RECORD, the transaction boundary determines the commit, not the acknowledge mode.
    https://github.com/spring-projects/spring-framework/blob/39e263fe5d8ba767d22e594cffd02420bab43f2a/spring-tx/src/main/java/org/springframework/transaction/support/TransactionTemplate.java#L127