How to implement and test retry option with S3CrtRetryConfiguration

I've a Spring Boot application integrated with AWS SDK written in Java. Also, the project uses Gradle as the build tool.

build.gradle:

dependencies {
    ......
    .........
    implementation 'software.amazon.awssdk.crt:aws-crt:0.24.0'
    implementation 'software.amazon.awssdk:s3-transfer-manager:2.20.119'
}

Now, I've the below code to generate S3AsyncClient with S3CrtRetryConfiguration.

S3CrtAsyncClientBuilder s3CrtAsyncClientBuilder = S3AsyncClient.crtBuilder()
            .credentialsProvider(creds).region(region);
S3CrtRetryConfiguration s3CrtRetryConfiguration = S3CrtRetryConfiguration.builder()
            .numRetries(s3UploadMaxRetries).build();
s3CrtAsyncClientBuilder.retryConfiguration(s3CrtRetryConfiguration);
S3AsyncClient s3CrtAsyncClient = s3CrtAsyncClientBuilder.build();

Problem: When I upload a file using this client, the code should retry for any failure. But I don't see any retry being triggered when a failure is happening.

Is it something that I missed in the configuration which needs to be added to make this work? Please suggest.

AWS provides different strategies for retrying errors. The AWS documentation describes them in detail, although there are subtle differences for the different SDK, tools and languages. Please, consider review the relevant documentation for the Java SDK.

With that in mind, I think you are providing the right configuration in your code:

S3CrtRetryConfiguration s3CrtRetryConfiguration = S3CrtRetryConfiguration.builder()
            .numRetries(s3UploadMaxRetries).build();
s3CrtAsyncClientBuilder.retryConfiguration(s3CrtRetryConfiguration);

As you can see in the source code of DefaultS3CrtAsyncClient, this configuration will be used to configure the underlying retry mechanism (as the stndard one, with exponential backoff based on the number of retries configured):

if (builder.retryConfiguration != null) {
  nativeClientBuilder.standardRetryOptions(
    new StandardRetryOptions()
      .withBackoffRetryOptions(
        new ExponentialBackoffRetryOptions()
          .withMaxRetries(
            builder.retryConfiguration.numRetries())));
}

Please, be aware that, by default, the AWS SDK will retry only under certain HTTP error codes and conditions:

static {
    Set<Integer> retryableStatusCodes = new HashSet<>();
    retryableStatusCodes.add(HttpStatusCode.INTERNAL_SERVER_ERROR);
    retryableStatusCodes.add(HttpStatusCode.BAD_GATEWAY);
    retryableStatusCodes.add(HttpStatusCode.SERVICE_UNAVAILABLE);
    retryableStatusCodes.add(HttpStatusCode.GATEWAY_TIMEOUT);
    RETRYABLE_STATUS_CODES = unmodifiableSet(retryableStatusCodes);


    Set<Class<? extends Exception>> retryableExceptions = new HashSet<>();
    retryableExceptions.add(RetryableException.class);
    retryableExceptions.add(IOException.class);
    retryableExceptions.add(UncheckedIOException.class);
    retryableExceptions.add(ApiCallAttemptTimeoutException.class);
    RETRYABLE_EXCEPTIONS = unmodifiableSet(retryableExceptions);
}

I am not certain in the case of CRT but, in general, you can in addition tweak this retry behavior and add certain error codes or exceptions you want to retry on.