crondatabricksjobsschedule

I would like to Create a (cron) schedule in Databricks that runs a job on Wednesday after the first Monday of a month

I need a Databricks job to run on Wednesday after the first Monday of a month. Or the same: two days after the first Monday.
I found that Cron for the first Wednesday of a month

0 30 6 ? * 4#1

but because the job relies on data we produce on the first Monday of the month, that won't work if the Wednesday is the first or second of a month.

Any help is greatly appreciated!

Solution

  • Quartz cron expressions cannot handle a complex pattern like this; the closest it could do alone is the Nth Monday or Wednesday of a month as you pointed out.

    As a workaround, you could use a cron schedule of every Wednesday: 0 0 0 ? * WED and then implement the rest of the scheduling logic in your code. In Python this might look like so:

    from datetime import datetime, timedelta

def should_run() -> bool:
    today = datetime.today()
    first = today.replace(day=1)
    days_to_first_monday = (7 - first.weekday() + 0) % 7
    first_monday = first + timedelta(days=days_to_first_monday)
    two_days_after_first_monday = first_monday + timedelta(days=2)
    return today.date() == two_days_after_first_monday.date()

if not should_run():
    dbutils.notebook.exit(f"{datetime.today().date()} is not the Wednesday after the first Monday of the month. Job run will be skipped this time.")