I need to implement an exponential backoff on a request that might fail. However, it's implemented as an async request. Had this been done synchronously, I'd have a better idea on where to put the delay. Roughly, I'm thinking it'd work something like this:
// These would be configurable in application.yml
currentAttempt = 0;
maxAttempts = 3;
timeoutGrowth = 2;
currentDelayTime = 5ms;
repeatNeeded = false;
while(repeatNeeded && currentAttempt < maxAttempts) {
httpStatusCode = makeRequest(someService)
if(httpStatusCode == 503) {
repeatNeeded=true;
currentAttempt++;
currentDelayTime*=timeoutGrowthRate;
sleep(currentDelayTime)
}
}
However, with an async call, the caller to the function is given the time back to do something else until the Future is has something. Do I code the backoff within the getObservations() method below, or do I code this in the caller of that getObservations() method? Below is the call as it currently is:
public CompletableFuture<ToolResponse> getObservations(String text, Map<String, Object> bodyParams) throws URISyntaxException {
URI uri = getUri(text);
HttpRequest request = getRequest(uri, text, bodyParams);
Map<String, String> contextMap = Optional.ofNullable(MDC.getCopyOfContextMap()).orElse(Collections.emptyMap());
Instant startTime = Instant.now();
return httpClient.sendAsync(request, BodyHandlers.ofString())
.exceptionally(ex -> {
throw new ExternalToolException(externalServiceConfig.getName(), ex);
})
.thenApply(response -> {
long toolRequestDurationMillis = ChronoUnit.MILLIS.between(startTime, Instant.now());
if (HttpStatus.valueOf(response.statusCode()).is2xxSuccessful()) {
ToolResponse toolResponse = processResponse(response, toolRequestDurationMillis);
logToolResponse(toolResponse);
return toolResponse;
}
log.error("{} returned non-200 response code: {}", externalServiceConfig.getName(), response.statusCode());
throw new ExternalToolException(externalServiceConfig.getName(), response.statusCode());
});
}
If you could consider using reactive java that has very powerful API including retries. For example,
request()
.retryWhen(Retry.backoff(3, Duration.ofSeconds(1)));
there are more options like retries for the specific exceptions only or defining max backoff
request()
.retryWhen(Retry.backoff(3, Duration.ofSeconds(2)));
.maxBackoff(5)
.filter(throwable -> isRetryableError(throwable))
You could use WebClient
that is a non-blocking client exposing a fluent, reactive API over underlying HTTP client libraries such as Reactor Netty
webClient.get()
.uri("/api")
.retrieve()
.bodyToMono(String.class)
.retryWhen(Retry.backoff(3, Duration.ofSeconds(1)));
if for some reason, you still want to use HttpClient you can wrap CompletableFuture
Mono.fromFuture(httpClient.sendAsync(request, BodyHandlers.ofString()))
.retryWhen(Retry.backoff(3, Duration.ofSeconds(1)));