azureexceptionpysparkazure-databricksmount-point

shaded.databricks.org.apache.hadoop.fs.azure.AzureException: An exception while trying to list a directory after mounting


I am getting below exception,

shaded.databricks.org.apache.hadoop.fs.azure.AzureException: java.util.NoSuchElementException: An error occurred while enumerating the result, check the original exception for details

First I mounted the directory in dbfs like below,

dbutils.fs.mount(
  source = f"wasbs://{containerName}@{storageAccount}.blob.core.windows.net/",
  mount_point = "/mnt/a",
  extra_configs = {f"fs.azure.sas.{containerName}.{storageAccount}.blob.core.windows.net": sasKey}
)

then I did,

dbutils.fs.ls("/mnt/a")

I see below reason,

Caused by: java.util.NoSuchElementException: An error occurred while enumerating the result, check the original exception for details. at hadoop_azure_shaded.com.microsoft.azure.storage.core.LazySegmentedIterator.hasNext(LazySegmentedIterator.java:113) at shaded.databricks.org.apache.hadoop.fs.azure.StorageInterfaceImpl$WrappingIterator.hasNext(StorageInterfaceImpl.java:158) at shaded.databricks.org.apache.hadoop.fs.azure.AzureNativeFileSystemStore.listInternal(AzureNativeFileSystemStore.java:2444) ... 41 more Caused by: hadoop_azure_shaded.com.microsoft.azure.storage.StorageException: This request is not authorized to perform this operation using this permission. at hadoop_azure_shaded.com.microsoft.azure.storage.StorageException.translateException(StorageException.java:87) at hadoop_azure_shaded.com.microsoft.azure.storage.core.StorageRequest.materializeException(StorageRequest.java:305) at hadoop_azure_shaded.com.microsoft.azure.storage.core.ExecutionEngine.executeWithRetry(ExecutionEngine.java:196) at hadoop_azure_shaded.com.microsoft.azure.storage.core.LazySegmentedIterator.hasNext(LazySegmentedIterator.java:109)

Could someone please help me on this?


Solution

  • This has happened due to wrong SAS key configuration which did not have all permissions for the container. The issue has been resolved after giving right SAS key with all permissions.