We are using Aiven as a Kafka as a Service provider, and they support credential rotation. E.g. every 24 hours.
So far, we have yet to find a way to create new consumers / producers in Spring Kafka when we need to rotate. Basically, a TopicAuthorizationException is thrown on consumer.poll.
What we need to do, is basically create a new consumer with a update truststore and keystore path. Keystore and truststore is loaded from kubernetes secrets.
Anyone know how we can make this work?
The listener container will stop after any AuthorizationException.
You can add an ApplicationListener or @EventListener method to receive a ListenerContainerStoppedEvent.
You can either wait for all child containers to stop, or immediately stop the concurrent container; update the container's containerProperties.kafkaConsumerProperties with the new store locations and restart the container.
I don't currently see a good solution on the producer side; you would need to replace the producer factory bean or the KafkaTemplate(s) with new overrides for the store locations.
I'll have opened an issue to make it easier there.