I try to use Apache Airflow's @dag decorator with parameters (params argument) to be able to run same instructions for different configurations, but can't find information on how to access these params' values within the code.
Here is the sample:
@dag(
dag_id='DAG_ID',
start_date=datetime(2023, 6, 1),
params={
'id': 'ID_A'
}
)
def main(**kwargs):
print(...) # want to print 'ID_A' here
I want to print 'ID_A' value in the function and later pass it as an argument to one of the DAG tasks, how can I do it?
You can use Jinja Template,
also, you can get the dag_run and find it inside "conf" property.
from datetime import datetime
from airflow.decorators import task, dag
id_param = "{{ params.id }}"
@dag(
dag_id='DAG_ID',
start_date=datetime(2023, 6, 1),
params={
'id': 'ID_A'
}
)
def my_dag():
@task
def print_value(value):
print(value)
@task
def get_conf():
context = get_current_context()
dag_run: DagRun = context["dag_run"]
print(dag_run.conf.get("id"))
print_value(id_param) >> get_conf()
my_dag()