Why do Amazon EMR clusters started with CLI not show up in the web console?

Question

I am working with the AWS CLI to run some map reduce steps. If I use list-clusters I can see my cluster was started:

aws emr list-clusters
{
    "Clusters": [
    {
        "Status": {
            "Timeline": {
                "CreationDateTime": 1418219740.791
            }, 
            "State": "STARTING", 
            "StateChangeReason": {
                "Message": "Configuring cluster software"
            }
        }, 
        "Id": "j-141E0DHGZ1ZA8", 
        "Name": "Development Cluster"
    }]
}

A few minutes later I can see that my step has (unfortunately) failed:

"Status": {
            "Timeline": {
                "ReadyDateTime": 1418219967.64, 
                "CreationDateTime": 1418219740.791
            }, 
            "State": "TERMINATING", 
            "StateChangeReason": {
                "Message": "Shut down as step failed", 
                "Code": "STEP_FAILURE"
            }
        }, 

However, the cluster (both while starting as when failed) don't show up on the amazon webconsole. As far as I know, I am only using one IAM user (with a separate key for the CLI console). What could a reason be for the cluster to not show up on the webconsole?

Answer 1

Two things to check :

1. Are you looking in the correct region? Maybe your CLI is starting the cluster in another region than the one you're looking at in the web console.

2. If you are using different users between web console and CLI, are you using the ---visible-to-all-users option in the CLI ? See http://docs.aws.amazon.com/cli/latest/reference/emr/create-cluster.html for more details