databricksazure-databricksdbutils

How to delete all files from folder with Databricks dbutils

Can someone let me know how to use the databricks dbutils to delete all files from a folder. I have tried the following but unfortunately, Databricks doesn't support wildcards.

dbutils.fs.rm('adl://azurelake.azuredatalakestore.net/landing/stageone/*')

Thanks

Solution

  • According to the documentation, the rm function receives 2 parameters :

    rm(dir: String, recurse: boolean = false): boolean -> Removes a file or directory

    Where the second parameter is a boolean flag to set the recursitivity, so you just need to set it to true:

    dbutils.fs.rm('adl://azurelake.azuredatalakestore.net/landing/stageone/',True)