oozieoozie-coordinator

Oozie Custom EL function oozie for coordinator


I want to create a custom el function that i will use inside oozie coordinator. My custom el function is working fine but when i pass already defined oozie el function as a parameter to my el function it throws exception.

coordinator.xml

${coord:dateToEpoch(coord:nominalTime(), "yyyy-MM-dd'T'hh:mmZ") see below example

<datasets>
    <dataset name="input1" frequency="${inputDataSetFrequence}" initial-instance="${initialInstance}"
             timezone="${timezone}">
        <uri-template>${inputBasePath}/${useCaseName}/bintime=${coord:dateToEpoch(coord:nominalTime(),  "yyyy-MM-dd'T'hh:mmZ")}
        </uri-template>
        <done-flag></done-flag>
    </dataset>
</datasets>
<input-events>
    <data-in name="coordInput1" dataset="input1">
        <instance>${coord:current(0)}</instance>
    </data-in>
</input-events>

Configuarion that i used to test this

<property>
<name>oozie.service.ELService.ext.functions.coord-job-submit-nofuncs</name>
<value>coord:dateToEpoch=com.mobileum.oozie.MobileumELFunctions#dateToEpoch</value>
</property>

constant param is working but dynamic param is not working

Working call

${coord:dateToEpoch("2009-01-01T08:00UTC", "yyyy-MM-dd'T'hh:mmZ")

Exception call

${coord:dateToEpoch(coord:nominalTime(), "yyyy-MM-dd'T'hh:mmZ")

Tried to use all these properties

    <property>
<name>oozie.service.ELService.ext.functions.job-submit</name>
<value>coord:dateToEpoch=com.mobileum.oozie.MobileumELFunctions#dateToEpoch</value>
</property>
<property>
<name>oozie.service.ELService.ext.functions.workflow</name>
<value>coord:dateToEpoch=com.mobileum.oozie.MobileumELFunctions#dateToEpoch</value>
</property>
<property>
<name>oozie.service.ELService.ext.functions.wf-sla-submit</name>
<value>coord:dateToEpoch=com.mobileum.oozie.MobileumELFunctions#dateToEpoch</value>
</property>
<property>
<name>oozie.service.ELService.ext.functions.coord-job-submit-freq</name>
<value>coord:dateToEpoch=com.mobileum.oozie.MobileumELFunctions#dateToEpoch</value>
</property>
<property>
<name>oozie.service.ELService.ext.functions.coord-job-submit-data</name>
<value>coord:dateToEpoch=com.mobileum.oozie.MobileumELFunctions#dateToEpoch</value>
</property>
<property>
<name>oozie.service.ELService.ext.functions.coord-job-submit-instances</name>
<value>coord:dateToEpoch=com.mobileum.oozie.MobileumELFunctions#dateToEpoch</value>
</property>
<property>
<name>oozie.service.ELService.ext.functions.coord-sla-create</name>
<value>coord:dateToEpoch=com.mobileum.oozie.MobileumELFunctions#dateToEpoch</value>
</property>
<property>
<name>oozie.service.ELService.ext.functions.coord-sla-submit</name>
<value>coord:dateToEpoch=com.mobileum.oozie.MobileumELFunctions#dateToEpoch</value>
</property>
<property>
<name>oozie.service.ELService.ext.functions.coord-action-create</name>
<value>coord:dateToEpoch=com.mobileum.oozie.MobileumELFunctions#dateToEpoch</value>
</property>
<property>
<name>oozie.service.ELService.ext.functions.coord-action-create-inst</name>
<value>coord:dateToEpoch=com.mobileum.oozie.MobileumELFunctions#dateToEpoch</value>
</property>
<property>
<name>oozie.service.ELService.ext.functions.coord-action-start</name>
<value>coord:dateToEpoch=com.mobileum.oozie.MobileumELFunctions#dateToEpoch</value>
</property>
<property>
<name>oozie.service.ELService.ext.functions.coord-job-wait-timeout</name>
<value>coord:dateToEpoch=com.mobileum.oozie.MobileumELFunctions#dateToEpoch</value>
</property>
<property>
<name>oozie.service.ELService.ext.functions.bundle-submit</name>
<value>coord:dateToEpoch=com.mobileum.oozie.MobileumELFunctions#dateToEpoch</value>
</property>
<property>
<name>oozie.service.ELService.ext.functions.coord-job-submit-initial-instance</name>
<value>coord:dateToEpoch=com.mobileum.oozie.MobileumELFunctions#dateToEpoch</value>
</property>

and

<property>
<name>oozie.service.ELService.ext.functions.coord-job-submit-nofuncs</name>
<value>coord:dateToEpoch=com.mobileum.oozie.MobileumELFunctions#dateToEpoch</value>
</property>

EXCEPTION

Caused by: java.lang.Exception: Unable to evaluate :${inputBasePath}/${useCaseName}/bintime=${coord:dateToEpoch(coord:nominalTime(),  "yyyy-MM-dd'T'hh:mmZ")}:

    at org.apache.oozie.coord.CoordELFunctions.evalAndWrap(CoordELFunctions.java:743)
    at org.apache.oozie.command.coord.CoordSubmitXCommand.resolveTagContents(CoordSubmitXCommand.java:1002)
    ... 37 more
Caused by: javax.servlet.jsp.el.ELException: No function is mapped to the name "coord:nominalTime"
    at org.apache.commons.el.Logger.logError(Logger.java:481)
    at org.apache.commons.el.Logger.logError(Logger.java:498)
    at org.apache.commons.el.Logger.logError(Logger.java:525)
    at org.apache.commons.el.FunctionInvocation.evaluate(FunctionInvocation.java:150)
    at org.apache.commons.el.FunctionInvocation.evaluate(FunctionInvocation.java:163)
    at org.apache.commons.el.ExpressionString.evaluate(ExpressionString.java:114)
    at org.apache.commons.el.ExpressionEvaluatorImpl.evaluate(ExpressionEvaluatorImpl.java:274)
    at org.apache.commons.el.ExpressionEvaluatorImpl.evaluate(ExpressionEvaluatorImpl.java:190)
    at org.apache.oozie.util.ELEvaluator.evaluate(ELEvaluator.java:204)
    at org.apache.oozie.coord.CoordELFunctions.evalAndWrap(CoordELFunctions.java:734)

Solution

  • Not all Oozie EL constructs are evaluated in uri-template. Please refer section Synchronous Datasets for more details. Below is excerpt on uri-template:

    uri-template: The URI template that identifies the dataset and can be resolved into concrete URIs to identify a particular dataset instance. The URI template is constructed using:

    constants: See the allowable EL Time Constants below. Ex: ${YEAR}/${MONTH}.

    variables: Variables must be resolved at the time a coordinator job is submitted to the coordinator engine. They are normally provided a job parameters (configuration properties). Ex: ${market}/${language}

    The following EL constants can be used within synchronous dataset URI templates:

    Problem is not associated with your custom EL function implementation. So even if you have used following, it will not work:

    <uri-template>${inputBasePath}/${useCaseName}/bintime=${coord:nominalTime()}</uri-template>