In my oozie workflow I dynamically create a hive table, say T1. This hive action is then followed by a map-reduce action. I want to set number of reducers property (mapred.reduce.tasks) equal to distinct values of a field say (T1.group). Any ideas how to set value of some oozie parameter dynamically and how to get value of the parameter from hive distinct action to oozie parameter?
I hope this can help:
Shell
action, which will read the file and echo the value in the form of key=value
. Enable the capture-output
for the shell action.${wf:actionData('ShellAction')['key']}
, pass this value to the mapred.reduce.tasks
in the configuration
tag of the MR action.