Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How do I get more specific error info on killed job in Oozie

I have a hadoop map-reduce job running as a step in Oozie workflow. It is started using java action which implements org.apache.hadoop.util.Tool.

When the job is being killed for some reason I want to be able to email a notification which should contain the stacktrace if there was an exception during processing.

Currently I do it this way:

<action name="sendErrorNotifications">
    <email xmlns="uri:oozie:email-action:0.1">
        <to>[email protected]</to>
        <subject>Job execution failed ${wf:id()}</subject>
        <body>Job execution failed, error message: [${wf:errorMessage(wf:lastErrorNode())}]</body>
    </email>
    <ok to="fail" />
    <error to="fail" />
</action>

But all I receive is just:

Job execution failed, error message: [Job failed!]

Which is not very useful :) and I need to go and check all the nodes' logs by myself.

How can I get more specific messages? Should I catch my exceptions and wrap into some oozie-catchable one in the Tool, or just use something instead of ${wf:errorMessage...

Thanks

like image 399
Art Avatar asked Nov 23 '12 11:11

Art


People also ask

How do I check my Oozie job status?

To check the workflow job status via the Oozie web console, with a browser go to http://localhost:11000/oozie .

Is it possible to parameterized Oozie workflow?

Oozie workflows can be parameterized. The parameters come from a configuration file called as property file. We can run multiple jobs using the same workflow by using multiple . property files.


1 Answers

One suggestion is to catch the exception in your main method, and export a property ('exceptionTrace' for example) with the exception serialized into its value (combined with the capture-output flag), which you can then reference using the wf:actionData('myJavaAction')['exceptionTrace'] EL function.

http://oozie.apache.org/docs/3.2.0-incubating/WorkflowFunctionalSpec.html#a3.2.7_Java_Action

like image 159
Chris White Avatar answered Oct 21 '22 15:10

Chris White