Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Automatically deleting objects older than n days in Amazon S3 (How ?)

I am storing many images in Amazon S3, using a ruby lib (http://amazon.rubyforge.org/)

I don't care the photos older than 1 week, then to free the space in S3 I have to delete those photos.

I know there is a method to delete the object in a certain bucket:

S3Object.delete 'photo-1.jpg', 'photos'

Is there a way to automatically delete the image older than a week ?

If it does Not exist, I'll have to write a daemon to do that :-(

Thank you


UPDATE: now it is possible, check the Roberto's answer.

like image 415
Alessandro De Simone Avatar asked Jul 13 '10 13:07

Alessandro De Simone


People also ask

Which feature of S3 can automatically delete objects?

Object Expiration allows you to schedule removal of your objects after a defined time period. Using Object Expiration rules to schedule periodic removal of objects eliminates the need to build processes to identify objects for deletion and submit delete requests to Amazon S3.

How do I delete a large number of objects in S3 bucket?

If you are planning to delete large numbers of objects from S3 then you can quickly do so by using Multi-Object Delete. You can also delete object versions (in buckets where S3 object versioning has been enabled) by including Version Ids in the request.

What is the best way to delete multiple objects from S3?

Navigate to the Amazon S3 bucket or folder that contains the objects that you want to delete. Select the check box to the left of the names of the objects that you want to delete. Choose Actions and choose Delete from the list of options that appears. Alternatively, choose Delete from the options in the upper right.


1 Answers

You can use the Amazon S3 Object Expiration policy

Amazon S3 - Object Expiration | AWS Blog

If you use S3 to store log files or other files that have a limited lifetime, you probably had to build some sort of mechanism in-house to track object ages and to initiate a bulk deletion process from time to time. Although our new Multi-Object deletion function will help you to make this process faster and easier, we want to go ever farther.

S3's new Object Expiration function allows you to define rules to schedule the removal of your objects after a pre-defined time period. The rules are specified in the Lifecycle Configuration policy that you apply to a bucket. You can update this policy through the S3 API or from the AWS Management Console.

Object Expiration | AWS S3 Documentation

Some objects that you store in an Amazon S3 bucket might have a well-defined lifetime. For example, you might be uploading periodic logs to your bucket, but you might need to retain those logs for a specific amount of time. You can use using the Object Lifecycle Management to specify a lifetime for objects in your bucket; when the lifetime of an object expires, Amazon S3 queues the objects for deletion.

Ps: Click on the links for more information.

like image 134
Roberto Avatar answered Oct 14 '22 04:10

Roberto