Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

S3 Object: Difference between 'expires' and 'expiration date'

I am attempting to follow AWS documentation for aws s3 cp, which documents the --expires flag as:

--expires (string) The date and time at which the object is no longer cacheable.

...

Copying a local file to S3 with an expiration date

The following cp command copies a single file to a specified bucket and key that expires at the specified ISO 8601 timestamp:

aws s3 cp test.txt s3://mybucket/test2.txt --expires 2014-10-01T20:30:00Z

So, when I run a command just like the example above, I wind up with a file in S3 which has an "Overview" pane on the right side which looks like this:

overview pane

The overview pane claims there is no "Expiration date". OK. But if I click through to the file, under Properties -> Metadata, I do see this:

S3 Object Metadata

So which one is right? Do the 'Expires' and 'Expiration date' timestamps mean different things somehow? Or are they the same thing, and the display is just buggy?

I have searched StackOverflow for several similar questions (here, here, and here ) but didn't find this question answered.

like image 564
Josh Kupershmidt Avatar asked Dec 24 '18 19:12

Josh Kupershmidt


People also ask

What happens when S3 object expires?

When an object reaches the end of its lifetime based on its lifecycle policy, Amazon S3 queues it for removal and removes it asynchronously. There might be a delay between the expiration date and the date at which Amazon S3 removes an object.

Does S3 delete expired objects?

Expiration deletes the object, and the deleted object cannot be recovered. If the current version is not a delete marker, Amazon S3 creates a delete marker, which becomes the current version, and the existing current version is retained as a noncurrent version.

What is expire current versions of objects?

There are three specific Amazon S3 lifecycle expiration actions that can be leveraged by customers: Expiring current version of the object: This configuration allows users to automatically expire the current version of the Amazon S3 objects stored within the bucket after a specified number of days.

How long do objects stay in S3?

S3 Storage Archive S3 Glacier—data must be stored for at least 90 days and can be restored within 1-5 minutes, with expedited retrieval. S3 Glacier Deep Archive—data must be stored for at least 180 days, and can be retrieved within 12 hours.


1 Answers

So, this is extremely poorly documented, but I think I found an answer here:

To add a Cache-Control or Expires header field to Amazon S3 objects using the Amazon S3 console ...

Choose Add Metadata, and then in the Key menu, choose Cache-Control or Expires. ...

For an Expires field, type a date and time in HTML format.

In other words, the Expires field that one sets via aws s3 cp ... --expires=... is intended to be an HTTP header which influences downstream caches (apparently CloudFront will honor this expires value). The S3 object is not deleted after this time.

Confusingly, this Expires value is completely different from the Expiration of an S3 object, which I gather can only be set by a Lifecycle Policy on the S3 bucket, not per-file at upload time (i.e. not using aws s3 cp ... ).

like image 85
Josh Kupershmidt Avatar answered Sep 16 '22 16:09

Josh Kupershmidt