[SOLVED] jinja2.exceptions.TemplateNotFound error with airflow bash operator

jinja2.exceptions.TemplateNotFound error with airflow bash operator

Running the below DAG task.

dag = DAG(
    'TEST',
    default_args=default_args,
    schedule_interval=None,
    max_active_runs=1)

create_command0 = "./auto_script/jobs/start_mail.sh"
t0 = BashOperator(
    task_id='start_mail',
    bash_command=create_command0,
    dag=dag,
    env={
        'subject_area': feed_name,
        'customer_name': customer,
        'job_home': home,
        'job_start_time': start_time})

DAG is keep on failing with the error, "jinja2.exceptions.TemplateNotFound: ./auto_script/jobs/start_mail.sh"

Solution

This is a common error with airflow. Try the following. This should be done in general so will hopefully help with your specific issue.

You need to add a space at the end of the command like so. (Notice the space at end of string)
You should also use the full path if possible instead of the relative path where '/home/user/bob/airflow/' is the location.

create_command0 = "/home/user/bob/airflow/auto_script/jobs/start_mail.sh "

instead of

create_command0 = "/home/user/bob/airflow/auto_script/jobs/start_mail.sh"

Hope that helps. More information on the exact cause can be found in the link below. https://cwiki.apache.org/confluence/display/AIRFLOW/Common+Pitfalls