ooziedirected-acyclic-graphsoozie-workflow

Can I use an oozie action as a template that I call many times?


I have a shell oozie action that takes in a number of arguments that get passed to the shell script. I want to trigger that action multiple times with different arguments each time. An example dag would look something like:

    Start
   /  |  \
f(a) f(d) f(g)
 |    |    |
 V    V    V
f(b) f(e) f(h)
 |    |    |
 V    V    V
f(c) f(f) f(i)
   \  |  /
     End

Is it possible to achieve this without having a bunch of duplicate actions?


Solution

  • yes, it is possible with Oozie sub-workflow option.

    Compose the Shell action as a sub-workflow and call the sub-workflow from many places from the main workflow with different input values.

        <action name="call_sub_workflow_1">
          <sub-workflow>
              <app-path>${workflowAppUri}/sub_workflow_with_shell_action/</app-path>
              <propagate-configuration/>
              <configuration>
                  <property>
                      <name>argument_1</name>
                      <value>${wf:actionData('previous_action')['val1']}</value>
                  </property>
                  <property>
                      <name>argument_2</name>
                      <value>${val2}</value>
                  </property>
              </configuration>
          </sub-workflow>
          <ok to="End"/>
          <error to="Fail"/>
        </action>
    

    Main workflow with sub-workflow directory structure will looks like,

       main_workflow_dir
       |
       |--job.properties
       |
       |--workflow.xml  # main workflow, can call sub workflow from multiple places
       |
       |--sub_workflow_with_shell_action
           |
           |--shell script
           |
           |--workflow.xml  # sub workflow contains the shell script action and takes arguments