Menu
I need to get the list of job names that currently running, but hadoop -job list give me a list of jobIDs.
604
You can monitor all the jobs that are running on a YARN cluster by tracking the job URL that you will get after submitting the job. You can monitor your Hive job status also using resource manager web UI. Visit localhost:8088, here you can see the list of all jobs that are submitted to the YARN cluster in that session.
From the Dashboard's Common Tasks menu, select MapReduce Workload. The console opens in a new window. Click MapReduce > Workload > Jobs. At this level, you can view details of a job such as job status and map/reduce task summary.
waitForCompletion(true); job. submit();
You schedule IBM Workload Scheduler Hadoop Map Reduce jobs by defining them in job streams. Add the job to a job stream with all the necessary scheduling arguments and submit the job stream. You can submit jobs by using the Dynamic Workload Console, Application Lab or the conman command line.
I've had to do this a number of times so I came up with the following command line that you can throw in a script somewhere and reuse. It prints the jobid followed by the job name.
hadoop job -list | egrep '^job' | awk '{print $1}' | xargs -n 1 -I {} sh -c "hadoop job -status {} | egrep '^tracking' | awk '{print \$3}'" | xargs -n 1 -I{} sh -c "echo -n {} | sed 's/.*jobid=//'; echo -n ' ';curl -s -XGET {} | grep 'Job Name' | sed 's/.* //' | sed 's/<br>//'"
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
