hadoopoozieoozie-coordinator

Propagating an Oozie coordinator's run date into the workflow


Problem Summary

When running an Oozie coordinator, I need the workflow to be able to have access to the current date (i.e. the date on which the coordinator is running) in the format yyyyMMdd.

Information

I have a large Oozie workflow that is initialised by a coordinator.

coord.properties

nameNode=<namenode>
jobTracker=<jobtracker>:8050

queueName=default
oozie.use.system.libpath=true
oozie.libpath=${nameNode}/project/workflow/lib
oozie.coord.application.path=${nameNode}/project/workflow

coordinator.xml:

<?xml version="1.0" encoding="UTF-8"?>
<coordinator-app xmlns="uri:oozie:coordinator:0.1" name="wf_scheduler" frequency="${coord:days(1)}" start="2015-01-30T08:50Z" end="2017-01-30T05:00Z" timezone="UTC">
   <action>
      <workflow>
         <app-path>${nameNode}/project/workflow</app-path>
         <configuration>
            <property>
               <name>currentDate</name>
               <value>${coord:actualTime()}</value>
            </property>
            <property>
            <name>runDate</name>
            <value>${convertDate(currentDate,"YYYY-MM-DDTHH:mmZ","yyyyMMdd")}</value>
            </property>
         </configuration>
      </workflow>
   </action>
</coordinator-app>

When running this workflow from job.properties (i.e. not a coordinator), I have a runDate=20150125 property in job.properties. Obviously when running from a coordinator I need to have runDate set each day by the coordinator. This would seem trivial, but I can't find a solution.

I already have an EL function mentioned in oozie.service.ELService.ext.functions.workflow which allows me to give a string representing a date, a date format, and the desired output format, and then returns a string date in that format. i.e. ${convertDate("20150125", "yyyyMMdd", "yyyy-MM-dd") = 2015-01-25. This function works perfectly when running from job.properties and my ideal solution would involve being able to use it in coordinator.xml. However, I get the following error when submitting the coordinator:

Error: E1004 : E1004: Expression language evaluation error, Unable to evaluate :${convertDate(currentDate,"YYYY-MM-DDTHH:mmZ","yyyyMMdd")}:

Solution

  • You will probably find it easier to use the built-in EL function coord:formatTime(String timeStamp, String format) to convert timestamp formats:

    https://oozie.apache.org/docs/4.1.0/CoordinatorFunctionalSpec.html#a6.9.2._coord:formatTimeString_ts_String_format_EL_Function_since_Oozie_2.3.2

    For example:

    ${coord:formatTime(coord:actualTime(), "yyyyMMdd")}