pythonazurehttpazure-blob-storage

Handle DNS timeout with call to blob_client.upload_blob

While using the Azure storage SDK in Python, I have been unable to override what appears to be a default 90-second timeout to catch a DNS exception occurring within a call to blob_client.upload_blob(). I am looking for a way to override this with a shorter time interval (i.e. 5 seconds).

The following code illustrates this issue using a fictitious account name which DNS cannot resolve. I am using a timeout argument in the call to upload_blob, and I understand from reviewing documentation this enforces a server-side threshold, not a client-side threshold. I have not been successful in getting a client-side threshold to be enforced.

This issue appears similar to this unanswered question: How to handle timeout for Uploading a blob in Azure Storage using Python SDK?. The one (not accepted) solution suggests using a timeout threshold within the call to upload_blob. As noted above (and shown within the code below), this is not producing the desired effect.

from azure.core.exceptions import AzureError
from azure.storage.blob import BlobServiceClient

# Define Azure Storage Blob connection details
connection_string = "DefaultEndpointsProtocol=https;AccountName=test;AccountKey=removed==;EndpointSuffix=core.windows.net"
container_name = "containername"
blob_name = "blobname"
local_file_path = "c:/temp/test.txt"
 
 
# Create the BlobServiceClient
blob_service_client = BlobServiceClient.from_connection_string(connection_string)
 
# Function to perform the blob upload
def upload_blob_process():
    try:
        with open(local_file_path, "rb") as data:
            blob_client = blob_service_client.get_blob_client(container_name, blob_name)
            blob_client.upload_blob(data, timeout=5)
        print("Blob uploaded successfully!")
    except AzureError as e:
        print(f"Azure error occurred: {e}")
    except Exception as e:
        print(f"An error occurred: {e}")
 
 
upload_blob_process()
Solution

  • While using the Azure storage SDK in Python, I have been unable to override what appears to be a default 90-second timeout to catch a DNS exception occurring within a call to blob_client.upload_blob(). I am looking for a way to override this with a shorter time interval (i.e. 5 seconds).

    As, I mentioned in comment, The timeout parameter in upload_blob() applies to each individual HTTP request to the Azure Blob service and it does not affect DNS resolution or connection establishment timeouts but instead limits the server-side operation duration for each request.

    thanks, I'm aware. Do you know of any way to apply a DNS timeout on top of this?

    Code:

    import dns.resolver
from azure.storage.blob import BlobServiceClient
import time


connection_string = "DefaultEndpointsProtocol=https;AccountName=venkat326;AccountKey=xxxx=;EndpointSuffix=core.windows.net"

container_name = "result"
blob_name = "test.csv"
local_file_path = "path to file"

# DNS timeout in seconds
DNS_TIMEOUT = 5


# Function to resolve DNS with timeout
def resolve_dns_with_timeout(connection_string, timeout=DNS_TIMEOUT):
 
    blob_service_client = BlobServiceClient.from_connection_string(connection_string)
    account_url = blob_service_client.url
    hostname = account_url.split("//")[1].split("/")[0]

    resolver = dns.resolver.Resolver()
    resolver.lifetime = timeout
    resolver.timeout = timeout
    try:
        resolver.resolve(hostname, "A")
        print(f"DNS resolution for {hostname} succeeded.")
    except dns.resolver.Timeout:
        raise Exception(f"DNS resolution for {hostname} timed out after {timeout} seconds.")
    except dns.resolver.NXDOMAIN:
        raise Exception(f"The domain {hostname} does not exist.")
    except Exception as e:
        raise Exception(f"DNS resolution failed for {hostname}: {e}")


# Function to upload blob with DNS resolution
def upload_blob_with_dns_check(connection_string, container_name, blob_name, local_file_path):
    start_time = time.time()
    try:
        resolve_dns_with_timeout(connection_string)
        blob_service_client = BlobServiceClient.from_connection_string(connection_string)
        blob_client = blob_service_client.get_blob_client(container_name, blob_name)

        with open(local_file_path, "rb") as data:
            blob_client.upload_blob(data)
        print("Blob uploaded successfully!")
    except Exception as e:
        elapsed_time = time.time() - start_time
        print(f"An error occurred: {e}")
        print(f"Elapsed time before timeout or failure: {elapsed_time:.2f} seconds")


# Run the upload process
upload_blob_with_dns_check(connection_string, container_name, blob_name, local_file_path)

    If the DNS is wrong, you'll get output like:.

    An error occurred: The domain venkat326.blob.core.windows.net does not exist.
Elapsed time before timeout or failure: 0.13 seconds

    enter image description here

    If the DNS is correct, you'll get output like:

    DNS resolution for venkat326123.blob.core.windows.net succeeded.
Blob uploaded successfully!

    enter image description here