Yarn get logs with rest API

Question

In Hadoop2, is it possible to use the rest API to get the same result as:

yarn logs -applicationId <application ID>

Answer 1

This is a pain and I don't have a happy answer, but I can point you to some resources.

1. The YARN CLI dumps logs by going to the file system. If your application can access HDFS, it can do the same thing (but it's not simple).

2. Alternatively, you can get the application master log URL (but not the log contents) using the rest call http:///ws/v1/cluster/apps/{appid}. From this URL, you can fetch an HTML page with the log contents, which will be returned in a <pre> tag with encoded HTML entities (& etc).

3. If your application is still running, you can get the raw logs using the container id from the above URL: http:///ws/v1/node/containerlogs/{containerId}/stdout. This was implemented as part of YARN-649.

YARN-1264 looks like exactly what you want, but was Closed-Duplicate with the above JIRA. That wasn't quite accurate, since the CLI and web page can get the logs after the container is finished, but the REST service above can't.