pythonjsongoogle-cloud-platformairflowairflow-api

how to pass airflow ts_nodash in a json template


I need to have an API call from the Ariflow dag.

Below is the sample json_string I use in code:

    def api_call_func(**context):
        source_file_name = context["dag_run"].conf["source_file_names"]
    json_data = { "filenames":[
                 {"FileName":[f'Source_credit_{{{{ ts_nodash }}}}']}
                  ]
                }

        json_string = json.dumps(json_data, 
                        skipkeys = True, 
                        allow_nan = True)
        requests.post(url = API_URL, data = json_string)

    api_task = PythonOperator(
    task_id='api_task',
    provide_context=True,
    python_callable=api_call_func,
    dag=dag,
    )

but it is resulting in the below response:

<filenames>
         <FileName>f'Source_credit_{{ ts_nodash }}</FileName>
</filenames>

below is the response desired with the currentDateTime:

<filenames>
         <FileName>f'Source_credit_20210408010223</FileName> 
</filenames>

How to pass the ts_nodash macros in json? or how to pass the dag's execution datetime in json?


Solution

  • Templates are processed only if the string containing them is passed as an argument to a templated parameter of an operator. E.g., op_args parameter of PythonOperator.

    In your case, the value of ts_nodash is passed to api_call_func() as an argument by the PythonOperator. So, you can use the context parameter to access it:

    json_data = {
        "ID": "A001-001",
        "SourceName": sourcefile,
        "filenames": [{"FileName": [f"Source_credit_{context['ts_nodash']}"]}],
    }
    

    Note, the values for the rest of the default variables can be accessed in the same way.