Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to batch delete buckets

How do I delete buckets in a batch?

Here's what i've tried.

def deleteAllBuckets():
    batch = storage_client.batch()
    with batch:
        for bucket in storage_client.list_buckets():
            bucket.delete()

Technically it works because the buckets get deleted, however i'm not convinced i'm sending a single batch request. It looks like i'm sending one request per bucket.

Compare the above code to a batch request in Google Cloud Datastore

def deleteAllEntities():
    query = datastore_client.query(kind="Charge")
    queryIter = query.fetch()

    batch = datastore_client.batch()
    with batch:
        for entity in queryIter:
            batch.delete(entity.key)

You can see that i'm calling a delete method on the batch object. With the storage code, i'm calling delete on the bucket object. Unfortunately the cloud storage python API doesn't have any examples.

like image 817
Eric Guan Avatar asked Apr 20 '18 20:04

Eric Guan


People also ask

How do I delete multiple S3 buckets at once?

To delete multiple S3 objects using a single HTTP request, you can use the AWS CLI, or an AWS SDK. To empty an S3 bucket of its objects, you can use the Amazon S3 console, AWS CLI, lifecycle configuration rule, or AWS SDK.

What is the fastest way to delete S3 buckets?

To delete an S3 bucketSign in to the AWS Management Console and open the Amazon S3 console at https://console.aws.amazon.com/s3/ . In the Buckets list, select the option next to the name of the bucket that you want to delete, and then choose Delete at the top of the page.

What is the best way to delete multiple objects from S3?

Navigate to the Amazon S3 bucket or folder that contains the objects that you want to delete. Select the check box to the left of the names of the objects that you want to delete. Choose Actions and choose Delete from the list of options that appears. Alternatively, choose Delete from the options in the upper right.


1 Answers

When you use the context manager, it automatically adds the request to the batch and it gets executed when exiting the context manager.

However, the problem I see with your code is that you're calling list buckets inside the batch context manager. I think you want to create the bucket iterator outside. Something like this:

def deleteAllBuckets():
    buckets_iterator = storage_client.list_buckets()
    batch = storage_client.batch()
    with batch:
        for bucket in buckets_iterator:
            bucket.delete()
like image 173
jterrace Avatar answered Nov 14 '22 18:11

jterrace