Configure retention policy of all topics during creation
Trying to configure rentention.ms using spring, as I get an error of:
Caused by: java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.PolicyViolationException: Invalid retention.ms specified. The allowed range is [3600000..2592000000] From what I've read the new value is -1 (infinity) so is out of that range
Following what was in How to configure kafka topic retention policy during creation in spring-mvc? ,I added the below code but it seems to have no effect. Any ideas/hints on how might solve this?
ApplicationConfigurationTest.java
@test
public void kafkaAdmin () {
KafkaAdmin admin = configuration.admin();
assertThat(admin, instanceOf(KafkaAdmin.class));
}
ApplicationConfiguration.java
@Bean
public KafkaAdmin admin() {
Map<String, Object> configs = new HashMap<>();
configs.put(TopicConfig.RETENTION_MS_CONFIG, "1680000");
return new KafkaAdmin(configs);
}
Found the solution by setting the value
spring.kafka.streams.topic.retention.ms: 86400000
in application.yml.
Our application uses spring mvc, hence the spring notation.
topic.retention.ms is the value that needs to be set in the streams config
86400000 is a random value just used as it is in range of [3600000..2592000000]