javaazure-blob-storageazure-storageuuid

Azure Storage blob upload filename collision with UUID

During performance test we got some error from Azure Storage saying that

com.azure.storage.blob.models.BlobStorageException: Status code 409, "BlobAlreadyExistsThe specified blob already exists. RequestId:53c64237-701e-0086-6766-52cc7f000000

The obvious reason would be that we want to use the same id to upload the blob to Azure.
However what we have is

@Override
public String storeQuote(final JsonNode quote) {
    final String uuid = UUID.randomUUID().toString();
    try {
        final String blobName = BLOB_PREFIX + uuid;

        LOGGER.debug("Uploading content to blob[{}].", blobName);
        BlobClient blobClient = blobContainerClient.getBlobClient(blobName);
        blobClient.upload(BinaryData.fromObject(quote));
        LOGGER.debug("Upload was successful to blob [{}].", blobName);

        return uuid;
    }
    catch (Exception e) {
        throw new AzureBlogStorageErrorException("Error occurred when stored the quote in to Azure Blob Storage!", e);
    }
}

I can hardly imagine that we had 10+ times UUID collision in 1-2 hours performance test (I am just joking, obviously it can't happen or I am just very-very-... unlucky).

Any idea please what the issue can be please?

//////////////////////// question 1 ////////////////////////
I haven't set any retry mechanism, can this problem be caused by that? Is there a default retry if nothing is set? As far as I know there isn't but I am not 100% sure.

Solution

  • Azure Storage SDK does, indeed, have a default retry mechanism built in. Most likely, the issue you're running into stems from partially completed uploads.

    Since you're performance testing, the most likely scenario is that the upload is creating the file in your storage container, but due to high loads, the success response isn't being received by the client. Thus, the client will retry the upload using the same UUID.

    In order to mitigate the issue you're going to want to use the BlobServiceClientBuilder.retryOptions() method to setup your retries. You can find documentation regarding the different options you can work with here.