Basically the title. I am using a JDBC item reader and JDBC item writer and I'm changing a particular status through the processor by using an API, if the API fails to change the status, I want to use exponential backoff to retry this at a later instant. I'm not able to figure out how to implement this
You have two options:
There are basically two ways to do that, either programmatically or in a declarative way.
You first define your retry template with a backoff policy as needed:
@Bean
public RetryTemplate retryTemplate() {
// configure backoff policy
ExponentialBackOffPolicy exponentialBackOffPolicy = new ExponentialBackOffPolicy();
exponentialBackOffPolicy.setInitialInterval(1000);
exponentialBackOffPolicy.setMultiplier(2.0);
exponentialBackOffPolicy.setMaxInterval(10000);
// configure retry policy
SimpleRetryPolicy simpleRetryPolicy = new SimpleRetryPolicy();
simpleRetryPolicy.setMaxAttempts(5);
// configure retry template
RetryTemplate retryTemplate = new RetryTemplate();
retryTemplate.setBackOffPolicy(exponentialBackOffPolicy);
retryTemplate.setRetryPolicy(simpleRetryPolicy);
return retryTemplate;
}
Then use that retry template in your item processor:
import org.springframework.batch.item.ItemProcessor;
import org.springframework.retry.RetryCallback;
import org.springframework.retry.RetryContext;
import org.springframework.retry.support.RetryTemplate;
public class MyRetryableItemProcessor implements ItemProcessor {
RetryTemplate retryTemplate;
public MyRetryableItemProcessor(RetryTemplate retryTemplate) {
this.retryTemplate = retryTemplate;
}
@Override
public Object process(Object item) throws Exception {
return retryTemplate.execute(new RetryCallback<Object, Exception>() {
@Override
public Object doWithRetry(RetryContext retryContext) throws Exception {
// API call
return item;
}
});
}
}
Here is an example:
import org.springframework.batch.item.ItemProcessor;
import org.springframework.retry.RetryCallback;
import org.springframework.retry.RetryContext;
import org.springframework.retry.annotation.Backoff;
import org.springframework.retry.annotation.Retryable;
import org.springframework.retry.support.RetryTemplate;
import org.springframework.stereotype.Component;
@Component
public class MyAnnotationBasedRetryableItemProcessor implements ItemProcessor {
@Override
@Retryable(backoff = @Backoff(delay = 1000L, maxDelay = 10000, multiplier = 2.0))
public Object process(Object item) throws Exception {
// Do API call
return item;
}
}
In this case, you can set a custom RetryPolicy
in your fault-tolerant step:
@Bean
public Step step(StepBuilderFactory stepBuilderFactory) {
// configure backoff policy
ExponentialBackOffPolicy exponentialBackOffPolicy = new ExponentialBackOffPolicy();
exponentialBackOffPolicy.setInitialInterval(1000);
exponentialBackOffPolicy.setMultiplier(2.0);
exponentialBackOffPolicy.setMaxInterval(10000);
// configure retry policy
SimpleRetryPolicy simpleRetryPolicy = new SimpleRetryPolicy();
simpleRetryPolicy.setMaxAttempts(5);
return stepBuilderFactory.get("myStep")
.<Integer, Integer>chunk(5)
.reader(itemReader())
.processor(itemProcessor())
.writer(itemWriter())
.faultTolerant()
.retryPolicy(simpleRetryPolicy)
.backOffPolicy(exponentialBackOffPolicy)
.build();
}
Note that in this case, whenever your processor throws an exception for an item, the entire chunk is retried item by item (and each item will be re-processed in its own transaction).
The examples above use spring-retry since you mentioned you have a preference for that. But the same ideas can be applied with any other fault-tolerance library.