How do I list all running EMR clusters using Boto?

Question

How do I list all my running clusters in my aws account using boto? Using the the command line I can get them using :

aws emr list-clusters --profile my-profile --region us-west-2 --active

However I wanna do the same using boto3. However the following code does not return any clusters:

import boto3

session = boto3.Session(profile_name='my-profile')

client = session.client('emr', region_name= 'us-west-2')

response = client.list_clusters(
    ClusterStates=['RUNNING']
)

print response

Result:

{u'Clusters': [], 'ResponseMetadata': {'RetryAttempts': 0, 'HTTPStatusCode': 200, 'RequestId': '577f3961-bdc80772f266', 'HTTPHeaders': {'x-amzn-requestid': '577f3961-34e5-11e7-a12a-bdc80772f266', 'date': 'Tue, 09 May 2017 18:28:47 GMT', 'content-length': '15', 'content-type': 'application/x-amz-json-1.1'}}}

Answer 1

Here is the paginator solution.

import boto3

boto3 = boto3.session.Session(region_name='ap-northeast-2')
emr = boto3.client('emr')

page_iterator = emr.get_paginator('list_clusters').paginate(
    ClusterStates=['RUNNING','WAITING']
)

for page in page_iterator:
    for item in page['Clusters']:
        print(item['Id'])

The result is

j-21*****
j-3S*****

Answer 2

The cluster is initially in Waiting state, when there are jobs running against the cluster, it changes to Running state. In your case, it will only return a Id if there is atleast 1 job running in the cluster.

Change it to below:

ClusterStates=['WAITING', 'RUNNING']