apache-kafkaspring-kafkaspring-cloud-stream-binder-kafkakafka-transactions-api

Can I consume and produce multiple times in a single Kafka transaction?

I'm studying kafka transaction and have a question.

I understood a Kafka transaction to be a bundle of one consume and one produce.

Then, can I consume and produce multiple times in a single Kafka transaction?

For example,

  1. application A produces message B
  2. application C consumes message B and produces message D
  3. application E consumes message D and produces message F

Can the above process be contained in a single transaction?

I'm building an app and wondering if what I'm asking is possible.

Thank you.

Solution

  • No; that is not possible; transactions cannot cross application boundaries.

    However, a single application can consume and/or produce multiple records within the same transaction, you are not restricted to only one at a time.