pythonazureazure-blob-storage

Hitting a stubborn ClientAuthenticationError while attempting to use the ContainerClient class

I'm trying to run some integration tests in Azure via a pipeline designed by a previous team, and I'm not having much luck. Specifically, the pipeline keeps hitting a ClientAuthenticationError.

This is a minimal working example of the Python code which runs right before it crashes:

SAS_TOKEN = os.environ["SAS_TOKEN"]

credential = AzureSasCredential(SAS_TOKEN)
account_name = "ACCOUNT_NAME"
account_url = f"https://{account_name}.blob.core.windows.net"
container_name = "CONTAINER_NAME"

blob_service_client = BlobServiceClient(account_url, credential=credential)
container = blob_service_client.get_container_client(container_name)
parquet_names = container.list_blob_names(name_starts_with="PATTERN")
list_of_parquet = list(parquet_names)

And this is the traceback that the above code produces:

File "/home/vsts/work/1/s/./tests/run_tests.py", line 24, in access_datalake_locally
  list_of_parquet = list(parquet_names)
File "/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/azure/core/paging.py", line 123, in __next__
  return next(self._page_iterator)
File "/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/azure/core/paging.py", line 75, in __next__
  self._response = self._get_next(self.continuation_token)
File "/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/azure/storage/blob/_list_blobs_helper.py", line 175, in _get_next_cb
  process_storage_error(error)
File "/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/azure/storage/blob/_shared/response_handlers.py", line 186, in process_storage_error
  exec("raise error from None")   # pylint: disable=exec-used # nosec
File "<string>", line 1, in <module>
azure.core.exceptions.ClientAuthenticationError: Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.

I can think of a few things which might be going wrong here:

  1. That parquet_names object really doesn't like being type-cast into a list.
  2. I'm doing something bone-headed in the way in which I construct that credential object. (I have tried blob_service_client = BlobServiceClient(account_url, credential=SAS_TOKEN. Hits exactly the same error.)
  3. The SAS_TOKEN environmental variable is wrong.

I would be very grateful for any suggestions for how to fix this situation.

Solution

  • azure.core.exceptions.ClientAuthenticationError: Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.

    The above error occurred due to the way the SAS token is being generated or passed to the Azure Blob Storage service.

    You can generate the sas token from the portal.

    Portal:

    Storageaccount-> shared access signature -> check the required fields -> Generate SAS-token.

    enter image description here

    I'm using windows, So I used the below command to store the sas token from the portal.

    I agree with Gaurav Mantri's comment, the format of sas token should be like this.

     $env:SAS_TOKEN="?sv=2022-11-02&ss=bfqt&srt=sco&sp=rwdlacupiytfx&se=2024-10-30T12:50:48Z&st=2024-10-30T04:50:48Z&spr=https&sig=redacted"

    Code and output:

    import os
from azure.core.credentials import AzureSasCredential
from azure.storage.blob import BlobServiceClient

SAS_TOKEN = os.environ.get("SAS_TOKEN")
print(f"SAS_TOKEN: {SAS_TOKEN}")

account_name = "venkat326123" 
account_url = f"https://{account_name}.blob.core.windows.net"
credential = AzureSasCredential(SAS_TOKEN)

try:
    blob_service_client = BlobServiceClient(account_url, credential=credential)
    container_name = "venkat" 
    container = blob_service_client.get_container_client(container_name)
    parquet_names = container.list_blob_names(name_starts_with="Pattern")  
    list_of_parquet = list(parquet_names) 
    print(list_of_parquet)

except Exception as e:
    print(f"An error occurred: {e}")

    Output:

    SAS_TOKEN: ?sv=2022-11-02&ss=bfqt&srt=sco&sp=rwdlacupiytfx&se=2024-10-30T12:50:48Z&st=2024-10-30T04:50:48Z&spr=https&sig=redacted
['Pattern/data.parquet', 'Pattern/flights-1m.parquet', 'Pattern/mtcars.parquet']

    enter image description here