Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Oozie shell action not running as submitting user

I've written an Oozie workflow that runs a BASH shell script to do some hive queries and perform some actions on the results. The script runs but throws a permission error when accessing some of the HDFS data. The user that submitted the Oozie workflow has permission but the script is running as the yarn user.

Is it possible to make Oozie execute the script as the user who submitted the workflow? Hive and Java actions both execute as the submitted user, just shell is behaving differently.

Here's the rough outline of my Oozie action

<action name="start_action"
        retry-max="12"
        retry-interval="600">
    <shell xmlns="uri:oozie:shell-action:0.1">
        <job-tracker>${jobTracker}</job-tracker>
        <name-node>${nameNode}</name-node>
        <job-xml>${WorkflowRoot}/hive-site.xml</job-xml>
        <exec>script.sh</exec>
        <file>${WorkflowRoot}/script.sh</file>
        <capture-output />
    </shell>
    <ok to="next_action"/>
    <error to="send_email"/>
</action>

I'm running Oozie 4.1.0 and HDP 2.1.

like image 784
Blake Avatar asked Jul 31 '15 18:07

Blake


2 Answers

This issue will occur in all cluster that are configured using Simple Security. You've an option to override the default configuration. Include the below statement at the starting of the shell script will fix this issue.

export HADOOP_USER_NAME=<Name of submitted user>;
like image 66
SachinJ Avatar answered Oct 26 '22 01:10

SachinJ


you can make run with help of env-var

<env-var>HADOOP_USER_NAME=${wf:user()}</env-var>

<workflow-app xmlns="uri:oozie:workflow:0.3" name="shell-wf">
    <start to="shell-node"/>
    <action name="shell-node">
        <shell xmlns="uri:oozie:shell-action:0.1">
            <job-tracker>${jobTracker}</job-tracker>
            <name-node>${nameNode}</name-node>
            <configuration>
                <property>
                    <name>mapred.job.queue.name</name>
                    <value>${queueName}</value>
                </property>
            </configuration>
            <exec>test.sh</exec>
    <env-var>HADOOP_USER_NAME=${wf:user()}</env-var>
    <file>/user/root/test.sh</file>
        </shell>
        <ok to="end"/>
        <error to="fail"/>
    </action>
    <kill name="fail">
        <message>Shell action failed, error message[${wf:errorMessage(wf:lastErrorNode())}]</message>
    </kill>
    <end name="end"/>
</workflow-app>

like image 37
Mentya Avatar answered Oct 26 '22 00:10

Mentya