I am a newbie to Airflow. I have some .jar jobs generated with Talend Open Studio for Big Data, and I want to schedule and manage those with Airflow. My question is, does Airflow support .jar file or generated by TOS as DAG? And if it does, how? Or is there any alternative to run .jar on Airlow?
I'm using Airflow v1.10.3. The jobs are mainly to extract and process data from a MongoDB database, then update the database with the new processed data.
Thanks !
Airflow will happily run .jar files. There is a few examples kicking about for you to have a look at.
Running a standard .jar file: run_jar.py
Running a "built" Talend jobl loan_application_data.py
Obviously with both these examples the .jar or Talend file(s) will need to be on the server Airflow is executing on (as well as Java).