databricksazure-databricksdatabricks-connect

How to connect and read/write to Azure ADLS Gen2 or Blob Store using Shared Access Signature with Databricks

I created SAS keys for my Azure Storage account, see image below.

Can someone show me how to connect to the Storage account with Databricks using the one of the Keys listed below:

enter image description here

I tried the suggested answer, however I'm getting the following error

[enter image description here

Any thoughts

enter image description here

The following shows the SAS token I used, but still getting the error enter image description here

Solution

  • Copy the SAS token and use the below code.

    spark.conf.set("fs.azure.account.auth.type.<storage-account>.dfs.core.windows.net", "SAS")
spark.conf.set("fs.azure.sas.token.provider.type.<storage-account>.dfs.core.windows.net", "org.apache.hadoop.fs.azurebfs.sas.FixedSASTokenProvider")
spark.conf.set("fs.azure.sas.fixed.token.<storage-account>.dfs.core.windows.net", "?sv=...SAS_token")

spark.read.csv('abfss://data@jadls2.dfs.core.windows.net/outfromVS.csv').display()

    Output:

    _c0 _c1
    0 2016-07-28,CA,1,A,A1,7.92
    1 2016-07-29,CA,1,A,A1,7.88
    2 2016-07-30,CA,1,A,A1,8.0
    3 2016-07-31,CA,1,A,A1,7.94
    4 2016-08-01,CA,1,A,A1,8.0
    5 2016-08-02,CA,1,A,A1,7.98
    6 2016-08-03,CA,1,A,A1,7.99

    Refer this for more information.