How to use boto3 waiters to take snapshot from big RDS instances

Question

I started migrating my code to boto 3 and one nice addition I noticed are the waiters.

I want to create a snapshot from a db instance and I want to check for it's availability before I resume with my code.

My approach is the following:

# Notice: Step : Check snapshot availability [1st account - Oregon]
print "--- Check snapshot availability [1st account - Oregon] ---"
new_snap = client1.describe_db_snapshots(DBSnapshotIdentifier=new_snapshot_name)['DBSnapshots'][0]
# print pprint.pprint(new_snap) #debug


waiter = client1.get_waiter('db_snapshot_completed')
print "Manual snapshot is -pending-"
sleep(60)
waiter.wait(
                DBSnapshotIdentifier = new_snapshot_name,
                IncludeShared = True,
                IncludePublic = False
            )

print "OK. Manual snapshot is -available-"

,but the documentation says that it polls the status every 15 seconds for 40 times. That is 10 minutes. Yet, a rather big DB will need more than that .

How could I use the waiter to alleviate for that?

Answer 1

Waiters have configuration parameters'delay' and 'max_attempts' like this :

waiter = rds_client.get_waiter('db_instance_available')
print( "waiter delay: " + str(waiter.config.delay) )

waiter.py on github

Answer 2

You could do it without the waiter if you like.

From the documentation for that waiter: Polls RDS.Client.describe_db_snapshots() every 15 seconds until a successful state is reached. An error is returned after 40 failed checks.

Basically that means it does the following:

RDS = boto3.client('rds')
RDS.describe_db_snapshots()

You can just run that but filter to your snapshot id, here is the syntax.http://boto3.readthedocs.io/en/latest/reference/services/rds.html#RDS.Client.describe_db_snapshots

response = client.describe_db_snapshots(
    DBInstanceIdentifier='string',
    DBSnapshotIdentifier='string',
    SnapshotType='string',
    Filters=[
        {
            'Name': 'string',
            'Values': [
                'string',
            ]
        },
    ],
    MaxRecords=123,
    Marker='string',
    IncludeShared=True|False,
    IncludePublic=True|False
)

This will end up looking something like this:

snapshot_description = RDS.describe_db_snapshots(DBSnapshotIdentifier='YOURIDHERE')

then you can just loop until that returns a snapshot which is available. So here is a very rough idea.

 import boto3
 import time
 RDS = boto3.client('rds')
 RDS.describe_db_snapshots()
 snapshot_description = RDS.describe_db_snapshots(DBSnapshotIdentifier='YOURIDHERE')
 while snapshot_description['DBSnapshots'][0]['Status'] != 'available' :
     print("still waiting")
     time.sleep(15)    
     snapshot_description = RDS.describe_db_snapshots(DBSnapshotIdentifier='YOURIDHERE')