azure-databricksdbxdatabricks-dbx

Setup E-mail notification on failure run with DBX deployment


I am deploying workflows to Databricks using DBX. Here I want to add a step which will send an e-mail to email123@email.com whenever the workflow fails. The outline of my deloyment.yml file is as below:

deployments:
  - name: my_workflow
    
    schedule:
      quartz_cron_expression: "0 0 5 * * ?"
      timezone_id: "Europe/Berline"

    format: MULTI_TASK

    job_clusters:
      - job_cluster_key: "basic-job-cluster"
        <<: *base-job-cluster

    tasks:
      task_key: "my-task
      job_cluster_key: "basic-job-cluster"
      spark_python_task:
        python_files: "file://my_file.py"
 
    << Insert notification code here >>

I have not been able to find documentation about it, so if you can point me to that I will also be happy.


Solution

  • I found the solution in DBX documentation. For other looking add the following:

    email_notifications:
      on_failure: [ "user@email.com" ]
    

    Link to documentation: https://dbx.readthedocs.io/en/latest/reference/deployment/