databricksazure-databricks

Unable to mount ADLS Gen 2 abfss Storage account with Databricks: IllegalArgumentException: Unsupported Azure Scheme: abfss


When I attempt to mount an ADLS Gen Storage account with the below code, I get the error:

IllegalArgumentException: Unsupported Azure Scheme: abfss

container_name = "mycontainer"
storage_account = "MyStorageAccount"
key = "xxxxxxxxxx=="

url = "abfss://" + container_name + "@" + storage_account + ".dfs.core.windows.net/"
config = "fs.azure.account.key." + storage_account + ".dfs.core.windows.net"

mount_folder = "/mnt/lake"
mounted_list = dbutils.fs.mounts()

mounted_exist = False
for item in mounted_list:
  if mount_folder in item[0]:
    mounted_exist = True
    break

  if not mounted_exist:
    dbutils.fs.mount(source = url, mount_point = mount_folder, extra_configs = {config : key})

I have successfully mounted an ADLS Gen 2 account from Databricks in the past using this method, so I'm uncertain why I'm getting this error?

I wanted to update this question to mention that our current environment prevents us from creating an App Registration and thus prevents us from creating a Service Principal. Which is why I'm trying to mount the storage account using the 'Account Key'


Solution

  • I encountered the same error when attempting to mount Azure Data Lake Storage using an account key.

    enter image description here

    Since abfss typically requires OAuth authentication, trying to mount it using an account key resulted in an error. However, after modifying the code as shown below, I was able to successfully mount it and list its contents.

    container_name = "synpdemocontainer"
    storage_account = "allnewsynp"
    key = "<YOUR ACCOUNT KEY>"
    url = "wasbs://" + container_name + "@" + storage_account + ".blob.core.windows.net/"
    config = "fs.azure.account.key." + storage_account + ".blob.core.windows.net"
    mount_folder = "/mnt/lake"
    dbutils.fs.mount(source = url, mount_point = mount_folder, extra_configs = {config : key})
    

    enter image description here

    I successfully listed the contents using dbutils.fs.ls. Here's an output image for reference:

    enter image description here