Get the application ID while running a MapReduce job

Question

Is there any way to fetch the application ID when running - for example - the wordcount example with the yarn command?

I wish to initiate a job from another process with the yarn command, and monitor the status of the job through the YARN REST API.

(Using Hadoop 2.4.0)

Answer 1

You can use yarn application -list command to get the list of all the applications.

Specifically if you want to get the list of all the applications, which are currently in RUNNING state, you can execute the following command:

yarn application -list -appStates RUNNING

If you already know the application ID, then you can query the status of the application using the following command:

yarn application -status <application ID>

For e.g.

yarn application -status application_1448359154956_0001

I get following application report (in this case application was KILLED by user):

Application Report :
Application-Id : application_1448359154956_0001
Application-Name : distcp
Application-Type : MAPREDUCE
User : mballur
Queue : default
Start-Time : 1448359237581
Finish-Time : 1448359419592
Progress : 100%
State : KILLED
Final-State : KILLED
Tracking-URL : http://mballur.fareast.corp.microsoft.com:8088/cluster/ap
p/application_1448359154956_0001
RPC Port : -1
AM Host : N/A
Aggregate Resource Allocation : 1652876 MB-seconds, 1337 vcore-seconds
Log Aggregation Status : NOT_START
Diagnostics : Application killed by user.

You can parse this output to get State and Progress of the application.