pythonairflow

How to add new DAGs to Airflow?


I have defined a DAG in a file called tutorial_2.py (actually a copy of the tutorial.py provided in the airflow tutorial, except with the dag_id changed to tutorial_2).

When I look inside my default, unmodified airflow.cfg (located in ~/airflow), I see that dags_folder is set to /home/alex/airflow/dags.

I do cd /home/alex/airflow; mkdir dags; cd dags; cp [...]/tutorial_2.py tutorial_2.py. Now I have a dags folder matching the path set in airflow.cfg , containing the tutorial_2.py file I created earlier.

However, when I run airflow list_dags, I only get the names corresponding with the default, tutorial DAGs.

I would like to have tutorial_2 show up in my DAG list, so that I can begin interacting with. Neither python tutorial_2.py nor airflow resetdb have caused it to appear in the list.

How do I remedy this?


Solution

  • I think the reason for this is because you haven't exported AIRFLOW_HOME. Try doing:

    AIRFLOW_HOME="/home/alex/airflow/dags" airflow list_dags
    

    If that's not working then do two steps:

    export AIRFLOW_HOME="/home/alex/airflow/dags"
    airflow list_dags
    

    I believe this should work. Give it a go?