databricks

Azure Databricks SQL download query results

I'm fairly new to Databricks. I have an SQL query in a notebook and I want to download the full results (about 3000 rows) to a CSV file. However, when I run the query, it takes half an hour to display the first 1000 rows (which is useless to me) and then I have to click on "Download full results" which re-runs the query, hence the half hour it had just spent was completely wasted.

Is there a way to download the full results without first displaying the first 1000 rows in the browser?

Solution

  • might be this will help - Create 1 variable and Load your table into the variable

    Mytable = spark.table("tableName")

    then storing the data into the csv file with option like:

    (
    Mytable.write.format("com.databricks.spark.csv")
    .option("delimiter", "as per your requirement")
    .option("header", "true")
    .save("dbfs:/df/mytabledata.csv")
)

    Then you can download/access the file under the data bricks instance file system.