boto3 searching unused security groups

Question

I am using AWS Python SDK Boto3 and I am trying to know which security groups are unused. With boto2 I did it but I do not know how to do the same with boto3.

from boto.ec2.connection import EC2Connection
from boto.ec2.regioninfo import RegionInfo
import boto.sns
import sys
import logging
from security_groups_config import config

# Get settings from config.py
aws_access_key = config['aws_access_key']
aws_secret_key = config['aws_secret_key']    
ec2_region_name = config['ec2_region_name']
ec2_region_endpoint = config['ec2_region_endpoint']

region = RegionInfo(name=ec2_region_name, endpoint=ec2_region_endpoint)

if aws_access_key:
    conn = EC2Connection(aws_access_key, aws_secret_key, region=region)
else:
    conn = EC2Connection(region=region)

sgs = conn.get_all_security_groups()

## Searching unused SG if the instances number is 0
def search_unused_sg(event, context):
    for sg in sgs:
        print sg.name, len(sg.instances())

Answer 1

Use the power of Boto3 and Python's list comprehension and sets to get what you want in 7 lines of code:

import boto3

ec2 = boto3.resource('ec2') #You have to change this line based on how you pass AWS credentials and AWS config

sgs = list(ec2.security_groups.all())
insts = list(ec2.instances.all())

all_sgs = set([sg.group_name for sg in sgs])
all_inst_sgs = set([sg['GroupName'] for inst in insts for sg in inst.security_groups])
unused_sgs = all_sgs - all_inst_sgs

Debug information

print 'Total SGs:', len(all_sgs)
print 'SGS attached to instances:', len(all_inst_sgs)
print 'Orphaned SGs:', len(unused_sgs)
print 'Unattached SG names:', unused_sgs

Output

Total SGs: 289
SGS attached to instances: 129
Orphaned SGs: 160
Unattached SG names: set(['mysg', '...

Answer 2

First , I suggest you relook how boto3 deal with credential. Better use a genereic AWS credential file , so in the future when required, you can switch to IAM roles base credential or AWS STS without changing your code.

import boto3 
# You should use the credential profile file 
ec2 = boto3.client("ec2")

# In boto3, if you have more than 1000 entries, you need to handle the pagination
# using the NextToken parameter, which is not shown here.

all_instances = ec2.describe_instances() 
all_sg = ec2.describe_security_groups()

instance_sg_set = set()
sg_set = set()

for reservation in all_instances["Reservations"] :
  for instance in reservation["Instances"]: 
    for sg in instance["SecurityGroups"]:
      instance_sg_set.add(sg["GroupName"]) 


for security_group in all_sg["SecurityGroups"] :
  sg_set.add(security_group ["GroupName"])

idle_sg = sg_set - instance_sg_set

Note : code are not tested. Please debug it as required.