Disaster recovery for files on Amazon S3?

Question

We outgrew the disc space on our server some time ago, so I made the decision to move the majority of our user generated content onto Amazon S3. Obvious benefits in saving space, reducing bandwidth cost and spreading requests away from our server which also deals with Apache and MySQL.

We have about 150GB of images all in three sizes, small (~2KB), medium (~ 10KB), large (~ 70KB).

We haven't had any problems so far, but I'm aware that if Amazon went down, we would be screwed (my only backup is of the first 50GB or so of those images, many of which will have been deleted / changed etc).

I don't want to download the images onto my server (even if I had the disk space), nor can I download the entire bucket to my local machine (crap connection in the office).

So I need a solution to backup this data remotely and in the worst case scenario quickly restore all the data to the bucket.

Or am I worrying too much and Amazon is backing this up for me?

Answer 1

Amazon's resilience policy for S3 is located here: http://aws.amazon.com/s3/#protecting . Essentially, they store multiple copies within the region in case of failure.

To improve this, you could look at transferring your data to another S3 region, eg. US to EU or use Amazon's Import/Export service to copy the data onto a suitably sized hard disk and store it at the office or at home.

http://aws.amazon.com/importexport/

Answer 2

Enable Cross Region Replication for S3

As of March, 2015, Amazon started offering Cross Region Replication for S3. This would drastically reduce your risk of data loss if a specific region encounters a major disaster.

https://aws.amazon.com/blogs/aws/new-cross-region-replication-for-amazon-s3/

It's worth noting that enabling this functionality is not backwards compatible with objects already stored in your buckets.