javaspring-bootasync-awaitexponential-backoff

HttpClient asyncRequest and exponential backoff


I need to implement an exponential backoff on a request that might fail. However, it's implemented as an async request. Had this been done synchronously, I'd have a better idea on where to put the delay. Roughly, I'm thinking it'd work something like this:

// These would be configurable in application.yml
currentAttempt = 0;
maxAttempts = 3;
timeoutGrowth = 2;
currentDelayTime = 5ms;
repeatNeeded = false;
while(repeatNeeded && currentAttempt < maxAttempts) {
    httpStatusCode = makeRequest(someService)
    if(httpStatusCode == 503) {
        repeatNeeded=true;
        currentAttempt++;
        currentDelayTime*=timeoutGrowthRate;
        sleep(currentDelayTime)
    }
}

However, with an async call, the caller to the function is given the time back to do something else until the Future is has something. Do I code the backoff within the getObservations() method below, or do I code this in the caller of that getObservations() method? Below is the call as it currently is:

public CompletableFuture<ToolResponse> getObservations(String text, Map<String, Object> bodyParams) throws URISyntaxException {
        URI uri = getUri(text);
        HttpRequest request = getRequest(uri, text, bodyParams);
        Map<String, String> contextMap = Optional.ofNullable(MDC.getCopyOfContextMap()).orElse(Collections.emptyMap());

        Instant startTime = Instant.now();

        return httpClient.sendAsync(request, BodyHandlers.ofString())
                .exceptionally(ex -> {
                    throw new ExternalToolException(externalServiceConfig.getName(), ex);
                })
                .thenApply(response -> {
                    long toolRequestDurationMillis = ChronoUnit.MILLIS.between(startTime, Instant.now());
                    if (HttpStatus.valueOf(response.statusCode()).is2xxSuccessful()) {
                        ToolResponse toolResponse = processResponse(response, toolRequestDurationMillis);
                        logToolResponse(toolResponse);
                        return toolResponse;
                    }

                    log.error("{} returned non-200 response code: {}", externalServiceConfig.getName(), response.statusCode());
                    throw new ExternalToolException(externalServiceConfig.getName(), response.statusCode());
                });
}

Solution

  • If you could consider using reactive java that has very powerful API including retries. For example,

    request()
      .retryWhen(Retry.backoff(3, Duration.ofSeconds(1)));
    

    there are more options like retries for the specific exceptions only or defining max backoff

    request()
      .retryWhen(Retry.backoff(3, Duration.ofSeconds(2)));
      .maxBackoff(5)
      .filter(throwable -> isRetryableError(throwable))
    

    You could use WebClient that is a non-blocking client exposing a fluent, reactive API over underlying HTTP client libraries such as Reactor Netty

    webClient.get()
            .uri("/api")
            .retrieve()
            .bodyToMono(String.class)
            .retryWhen(Retry.backoff(3, Duration.ofSeconds(1)));
    

    if for some reason, you still want to use HttpClient you can wrap CompletableFuture

    Mono.fromFuture(httpClient.sendAsync(request, BodyHandlers.ofString()))
      .retryWhen(Retry.backoff(3, Duration.ofSeconds(1)));