How can I delete files older than seven days in Amazon S3?

Question

I need to delete files in Amazon S3 which are older than seven days. I needed a shell script to do this, but I didn't have any luck with google search. I found the below URL:

http://shout.setfive.com/2011/12/05/deleting-files-older-than-specified-time-with-s3cmd-and-bash/

It is not helpful to us. What would be a script to delete all files older than seven days?

Answer 1

The easiest method is to define Object Lifecycle Management on the Amazon S3 bucket.

You can specify that objects older than a certain number of days should be expired (deleted). The best part is that this happens automatically on a regular basis and you don't need to run your own script.

If you wanted to do it yourself, the best would be to write a script (eg in Python) to retrieve the list of files and delete ones older than a certain date.

Example: GitHub - jordansissel/s3cleaner: Amazon S3 file cleaner - delete things older than a certain age, matching a pattern, etc.

It's somewhat messier to do as a shell script.

Answer 2

We have modified the code a little bit and it is working fine.

   aws s3 ls BUCKETNAME/ | while read -r line;
       do
        createDate=`echo $line|awk {'print $1" "$2'}`
        createDate=`date -d"$createDate" +%s`
        olderThan=`date --date "7 days ago" +%s`
        if [[ $createDate -lt $olderThan ]]
           then
            fileName=`echo $line|awk {'print $4'}`

            if [[ $fileName != "" ]]
            then
                    aws s3 rm BUCKETNAME/$fileName
            fi
       fi

       done;