databricksazure-databricks

How to read a json file in databricks from any workspace folder


I have a simple json file(test.json) located in workspace folder(TestWS) I created as shown below: enter image description here

I want to read the json file from a notebook present in Repo Workspace.

I am using simple command in python

import json

with open(f"/TestWS/test.json","r") as f:
    js = json.load(f)

but all try in vain, I read and checked in workspace only Repo folder is accessible not other, but if its true is there any other workaround we can have to read it that way.

If I am using dbutils then only notebook I can execute using (dbutils.notebook.run('location',timeout)) but I need to read json for my use case. Any help will be appreciated.


Solution

  • For Databricks Runtime (DBR) 9.1, the Workspace File System (WSFS) is available only for Databricks Repos and only if it's explicitly configured in the Admin settings (another option is only to start with DBR 11.x). As documentation says:

    Workspace files are enabled everywhere by default for Databricks Runtime 11.2 and above.

    so you need to upgrade if you want to use it outside of Repos.