What is the correct ways to write Boto3 filters to use customise tag name?

Question

I am trying to list the instances on tag values of different tag keys For eg> one tag key - Environment, other tag key - Role. My code is given below :

import argparse
import boto3

AWS_ACCESS_KEY_ID = '<Access Key>'
AWS_SECRET_ACCESS_KEY = '<Secret Key>'

def get_ec2_instances(Env,Role):
    ec2 = boto3.client("ec2", region)
    reservations = ec2.describe_instances(Filters={"tag:environment" :   Env, "tag:role" : Role})
    for reservation in reservations["Reservations"] :
        for instance in reservation["Instances"]:
             print  "%s" % (instance.tags['Name'])

if  __name__ == '__main__':

    regions = ['us-east-1','us-west-1','us-west-2','eu-west-1','sa-east-1',
               'ap-southeast-1','ap-southeast-2','ap-northeast-1']
    parser = argparse.ArgumentParser()
    parser.add_argument('Env', default="environment", help='value for   tag:environment');
    parser.add_argument('Role', default="role", help='value for tag:role');
    args = parser.parse_args()

    for region in regions: get_ec2_instances(args.Env, args.Role)

After running this script : python script.py arg1 arg2

I am getting following error

Traceback (most recent call last):
  File "script.py", line 27, in <module>
    for region in regions: get_ec2_instances(args.Env, args.Role)
  File "script.py", line 10, in get_ec2_instances
    reservations = ec2.describe_instances(Filters={"tag:environment" :  Env, "tag:role" : Role})
  File "/usr/local/lib/python2.7/dist-packages/botocore/client.py", line 258, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/usr/local/lib/python2.7/dist-packages/botocore/client.py", line 524, in _make_api_call
    api_params, operation_model, context=request_context)
  File "/usr/local/lib/python2.7/dist-packages/botocore/client.py", line 577, in _convert_to_request_dict
    api_params, operation_model)
  File "/usr/local/lib/python2.7/dist-packages/botocore/validate.py", line 270, in serialize_to_request
    raise ParamValidationError(report=report.generate_report())
botocore.exceptions.ParamValidationError: Parameter validation failed:
Invalid type for parameter Filters, value: {'tag:role': 'arg1', 'tag:environment': 'arg2'}, type: <type 'dict'>, valid types: <type 'list'>, <type 'tuple'>

Answer 1

This looks familiar, did I modify this for somebody somewhere ;-) . Actually the code I wrote is in rush and not tested properly (And I don't bother to amend the % string formating and replace it with str.format() ) . In fact,using Filters parameter is not properly documented in AWS.

Please refer to Russell Ballestrini blog Filtering AWS resources with Boto3 to learn more about correct boto Filters method.

1. Filters accept list value, and info inside the tag should be dict. thus [{}]
2. Boto3 documentation is pretty ambiguous on how to use specify the tag name. It is confusing without examples when they say you may use tag:key. So many people will just do [{"tag:keyname","Values": [""] }] and it doesn't work. (Actually the origin code I assume the developer know how the filters works, so I just amend the structure only).
3. Actually, You MUST explicitly specify "Name" and "Values" pair. So the correct way to specify tag name is [{"Name" :"tag:keyname", "Values":[""] }]. It is tricky.

So the correct way of formatting a filters if you want to use for your example

filters = [{'Name':'tag:environment', 'Values':[Env]},
           {'Name':'tag:role', 'Values':[Role]}
          ]

(Update) And to make sure argparse take up string value, you just enforce the argument to take string values

parser.add_argument('Env', type=str, default="environment",
                    help='value for   tag:environment');
parser.add_argument('Role', type=str,default="role",
                    help='value for tag:role');

Answer 2

Although not actually the answer to your question but DO NOT, NEVER, put your AWS credentials hard coded in your scripts. With your AWS credentials, anyone can use your account. There are bots scouring github and other git repositories looking for hard coded AWS credentials.

Also, when rotating credentials all your code will be broken or you will have a hard time updating all of them.

Some alternatives instead hard coding your AWS credentials:

1. Configure your ~/.aws/credentials file
2. Use IAM Roles
3. Use STS to 'assumeRole'

Follow the best practices described here: Best Practices for Managing AWS Access Keys

Now, for answering your question, here is an example on how to filter by tags:

argEnv = '<any_string_you_want_to_match_as_a_value_for_a_tag>'
ec2Client = boto3.client('ec2')
response = ec2Client.describe_instances(
    Filters=[
            {
                'Name': 'tag:Projeto',
                'Values': [argEnv]
        }
    ]
)

Make sure 'Value' is a list and not a string. For example, if 'argEnv' is a string, make sure you use '[]' to encase your variable.

Then if you want to consult the Tag:Name and get the Value of it (for example, the name you set up for a specific EC2 instance in the console):

for reservation in res['Reservations']:
    for instance in reservation['Instances']:
        for tag in instance['Tags']:
            if tag['Key'] == 'Name':
                consoleName = tag['Value']
print(consoleName)

The output will be the Value of the Name tag for every resource. As you can see, you have to loop through the results to get the result. You can check the Response Syntax here.

Answer 3

In my own python script I use the following:

import boto3
ec2client = boto3.client('ec2','us-east-1')
response = ec2client.describe_instances(Filters=[{'Name' : 'instance-state-name','Values' : ['running']}])