Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to Remove Delete Markers from Multiple Objects on Amazon S3 at once

I have an Amazon S3 bucket with versioning enabled. Due to a misconfigured lifecycle policy, many of the objects in this bucket had Delete Markers added to them.

I can remove these markers from the S3 console to restore the previous versions of these objects, but there are enough objects to make doing this manually on the web console extremely time-inefficient.

Is there a way to find all Delete Markers in an S3 bucket and remove them, restoring all files in that bucket? Ideally I would like to do this from the console itself, although I will happily write a script or use the amazon CLI tools to do this if that's the only way.

Thanks!

like image 633
Dave Stern Avatar asked Aug 19 '15 21:08

Dave Stern


People also ask

What is the best way to delete multiple objects from S3?

Navigate to the Amazon S3 bucket or folder that contains the objects that you want to delete. Select the check box to the left of the names of the objects that you want to delete. Choose Actions and choose Delete from the list of options that appears. Alternatively, choose Delete from the options in the upper right.

How do I get rid of Delete marker?

To delete a delete marker permanently, you must include its version ID in a DeleteObject versionId request. The following figure shows how a DeleteObject versionId request permanently removes a delete marker.

How do you quickly delete objects in S3 bucket?

If the versioning is disabled, you can run the aws s3 rm CLI command to delete all objects in the S3 bucket. If versioning is enabled, you run the CLI command aws s3api delete-objects to delete all versioned objects in the S3 bucket. Once the S3 bucket is empty you can then proceed to delete it.


1 Answers

Use this to restore the files inside the specific folder. I've used aws cli commands in my script. Provide input as: sh scriptname.sh bucketname path/to/a/folder

**Script:**
#!/bin/bash
#please provide the bucketname and path to destination folder to restore
# Remove all versions and delete markers for each object
 aws s3api list-object-versions --bucket $1 --prefix $2 --output text | 
 grep "DELETEMARKERS" | while read obj
   do
        KEY=$( echo $obj| awk '{print $3}')
        VERSION_ID=$( echo $obj | awk '{print $5}')
        echo $KEY
        echo $VERSION_ID
        aws s3api delete-object --bucket $1 --key $KEY --version-id $VERSION_ID

   done

Edit: put $VERSION_ID in correct position in the script

like image 52
Kc Bickey Avatar answered Oct 06 '22 18:10

Kc Bickey