Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How long does it take for S3 Lifecycle policies to get applied to the old objects in the bucket?

Tags:

amazon-s3

I've recently define a lifecycle policy for my s3 bucket to transition the objects to Standard IA after 30 days. The bucket contains old files (more than 30 days) and I assume those objects should change to Standard IA. But after a day those objects still show Standard (not standard IA) as their storage class. I'm wondering how long would it take for the old objects to get affected by the newly defined policy?

enter image description here

like image 980
Nisman Avatar asked May 14 '19 15:05

Nisman


People also ask

How do I configure the S3 lifecycle rule?

If you want the S3 Lifecycle rule to apply to all objects in the bucket, specify an empty prefix. In the following configuration, the rule specifies a Transition action that directs Amazon S3 to transition objects to the S3 Glacier storage class 0 days after creation.

How to empty a bucket in Amazon S3 using lifecycle rule?

Open the Amazon S3 console. 2. From the list of buckets, choose the bucket that you want to empty. 3. Choose the Management tab. 4. Choose Create lifecycle rule. 5. For Lifecycle rule name, enter a rule name. 6. For Choose a rule scope, select This rule applies to all objects in the bucket. 7.

How are objects charged for S3 lifecycle?

Note that objects smaller than 128 KB are charged at the Frequent Access tier rates. When S3 Lifecycle processing runs daily, all objects in the bucket that match the rule are marked. The objects are marked for the action defined in the rule, and the expiration or transition of objects occurs asynchronously.

How long do S3 objects expire?

This rule directs Amazon S3 to expire objects 14 days after creation. The Lifecycle configuration is shown following. The policy is fine, but if there is an object with both tags, then S3 has to decide what to do. That is, both rules apply to an object, and in effect you are directing Amazon S3 to perform conflicting actions.


1 Answers

A rule of thumb that seems to hold is to expect the policy to take effect within 48 hours.

More specifically, you should expect to see see things happening by the time midnight UTC has occurred twice since the time you created the policy (which would be 48 hours, worst case, or slightly over 24 hours, best case) -- at the first pass of midnight, S3 starts evaluating the rules and flagging objects internally for transition, and at the second pass of midnight it actually starts transitioning the flagged objects.

The time may be longer when large numbers of objects are involved.

Review your bucket logs. At least some types of transitions generate log entries for each object.

like image 83
Michael - sqlbot Avatar answered Oct 20 '22 14:10

Michael - sqlbot