I really need to know how to use Airflow to have multiple DAGs call each other (e.g. DAG_1 >> DAG_2) similar to how it uses the upstream/downstream operators to run tasks within a DAG. I need to know how to set this up so Airflow will run DAG_1, then will run DAG_2 once DAG_1 has completed successfully.
Preferably I would like to have DAG_1 and DAG_2 in separate files within the Airflow dag folder, then either have DAG_2 call DAG_1 as a pre-step at the start of its run, or have a separate "main" DAG which creates and runs a DAG where each task is another DAG rather than a task operator.
If anyone knows how to do this, please let me know, I really need some help here. I will also accept making and connecting two separate DAGs in the same file if anyone knows how to do that, but preferably the linked DAGs will be in different files.
Thank you in advance for your assistance.
you can use TriggerDagRunOperator which takes the DAG id as a parameter and triggers it
see official example here