Hadoop YARN: Get a list of available queues

Question

Is there a way to get a list of all available YARN queues from the command line, without resorting to parsing the capacity-scheduler.xml file?

I'm using Hadoop version 2.7.2

Answer 1

You can use the hadoop builtin mapred command-line tool

[email protected]$ mapred queue -list
======================
Queue Name : root.tenant1
Queue State : running
Scheduling Info : Capacity: 0.0, MaximumCapacity: UNDEFINED, CurrentCapacity: 0.0
    ======================
    Queue Name : root.tenant1.default
    Queue State : running
    Scheduling Info : Capacity: 0.0, MaximumCapacity: UNDEFINED, CurrentCapacity: 0.0
    ======================
    Queue Name : root.tenant1.users
    Queue State : running
    Scheduling Info : Capacity: 0.0, MaximumCapacity: UNDEFINED, CurrentCapacity: 0.0
======================
Queue Name : root.tenant2
Queue State : running
Scheduling Info : Capacity: 0.0, MaximumCapacity: UNDEFINED, CurrentCapacity: 0.0
    ======================
    Queue Name : root.tenant2.default
    Queue State : running
    Scheduling Info : Capacity: 0.0, MaximumCapacity: UNDEFINED, CurrentCapacity: 0.0
    ======================
    Queue Name : root.tenant2.users
    Queue State : running
    Scheduling Info : Capacity: 0.0, MaximumCapacity: UNDEFINED, CurrentCapacity: 0.0
======================

it provides a simple and nice output with hierarchy

Answer 2

One way is to use ResourceManager REST API, for example:

curl '<resourcemanager_host>:<http_port>/ws/v1/cluster/scheduler' | jq '.scheduler.schedulerInfo.queues.queue[] | .queueName’

will list all top level queues.

curl '<resourcemanager_host>:<http_port>/ws/v1/cluster/scheduler' | jq .

gives you all kind of information about scheduler/queues, thus using jq you can get any information out of it.