azure-databricksdatabricks-notebook

Get Command number in databricks


How can i get the current command number dynamically from a cell in Azure databricks notebook? Is there any dbutils command to retrieve it?

For below cell, it should return 1.

Databricks command example


Solution

  • There is no direct built-in function in dbutils or Spark to retrieve the current command number dynamically from a cell in an Azure Databricks notebook.

    I have tried the following approach:

    spark._jsparkSession.sparkContext().getLocalProperty("spark.databricks.notebook.id")
    result = spark.sql("SELECT * FROM default.my_table")
    print(f"Current command number: {command_number}")
    

    enter image description here

    Result:

    Current command number: 2585995787002146
    

    spark._jsparkSession.sparkContext().getLocalProperty("spark.databricks.notebook.id"): The command above retrieves the number of the current cell in the Databricks notebook.