I have a shell oozie action that takes in a number of arguments that get passed to the shell script. I want to trigger that action multiple times with different arguments each time. An example dag would look something like:
Start
/ | \
f(a) f(d) f(g)
| | |
V V V
f(b) f(e) f(h)
| | |
V V V
f(c) f(f) f(i)
\ | /
End
Is it possible to achieve this without having a bunch of duplicate actions?
yes, it is possible with Oozie sub-workflow
option.
Compose the Shell action as a sub-workflow and call the sub-workflow from many places from the main workflow with different input values.
<action name="call_sub_workflow_1">
<sub-workflow>
<app-path>${workflowAppUri}/sub_workflow_with_shell_action/</app-path>
<propagate-configuration/>
<configuration>
<property>
<name>argument_1</name>
<value>${wf:actionData('previous_action')['val1']}</value>
</property>
<property>
<name>argument_2</name>
<value>${val2}</value>
</property>
</configuration>
</sub-workflow>
<ok to="End"/>
<error to="Fail"/>
</action>
Main workflow with sub-workflow directory structure will looks like,
main_workflow_dir
|
|--job.properties
|
|--workflow.xml # main workflow, can call sub workflow from multiple places
|
|--sub_workflow_with_shell_action
|
|--shell script
|
|--workflow.xml # sub workflow contains the shell script action and takes arguments