Amazon S3 lifecycle retroactive application

Question

Fairly straightforward question. Do Amazon S3 lifecycle rules that I set get applied to data retroactively? If so, what sort of delay might I see before older data begins to be archived or deleted?

By way of example, let's say I have a bucket with 3 years of backed up data. If I create a new lifecycle where that data will be archived after 31 days, and deleted after 365 days, will that new rule be applied to the existing data? How soon will it begin to be enforced?

Answer 1

Yes it's retroactive (i.e. all the things that are already there and match will get the rule).
There may be a slight delay (i.e. rules will have day granularity and run on daily basis), but the rules will take effect immediately. Depending on how much data you have to remove/move it may take a while if you have a lot of pre-existing data.

Source: S3 FAQ here: http://aws.amazon.com/s3/faqs/

Lifecycle policies apply to both existing and new S3 objects, ensuring that you can optimize storage and maximize cost savings for all current data and any new data placed in S3 without time-consuming manual data review and migration.

After an Object Expiration rule is added, the rule is applied to objects that already exist in the bucket as well as new objects added to the bucket. Once objects are past their expiration date, they are identified and queued for removal. You will not be billed for storage for objects on or after their expiration date, though you may still be able to access those objects while they are in queue before they are removed. As with standard delete requests, Amazon S3 doesn’t charge you for removing objects using Object Expiration. You can set Expiration rules for your versioning-enabled or versioning-suspended buckets as well.

Answer 2

S3 bucket lifecycle rules apply to both existing and new data in S3. In addiiton to Mircea answer:

• Lifecycle rules run once a day at midnight Universal Coordinated Time (UTC). For example, if you create a lifecycle rule on October 15 at 4:00 PM Pacific Standard Time (PST) that transitions objects after one day, then the objects are transitioned to the new storage class on October 17 at 12:00 AM UTC.
• New lifecycle rules can take up to 48 hours to complete the first run. If there are millions of objects for the rule, then it might take a few runs for all the objects to be transitioned into another storage class.