[SOLVED] Kafka Failed to rebalance when PROCESSING_GUARANTEE_CONFIG set to EXACTLY

Kafka Failed to rebalance when PROCESSING_GUARANTEE_CONFIG set to EXACTLY_ONCE

I have a Kafka stream application that works fine. However when I add the property:

properties.put(StreamsConfig.PROCESSING_GUARANTEE_CONFIG, StreamsConfig.EXACTLY_ONCE);

Then I get the following error:

Exception in thread "trade-state-decider-a271c9f6-d9d8-488c-b91c-57cdfaf54894-StreamThread-1" org.apache.kafka.streams.errors.StreamsException: stream-thread [trade-state-decider-a271c9f6-d9d8-488c-b91c-57cdfaf54894-StreamThread-1] Failed to rebalance.


Caused by: org.apache.kafka.common.errors.TransactionalIdAuthorizationException: Transactional Id authorization failed.

Anyone has any idea about what could course this?

Solution

As the error indicates, your application does not have permissions to use transactions. Using transactions requires special permissions that you can enable via ACLs.

See the docs for more details on what permission you need to grant: https://docs.confluent.io/current/kafka/authorization.html#enabling-authorization-for-idempotent-and-transactional-apis