Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

S3 last-modified timestamp for eventually-consistent overwrite PUTs

The AWS S3 docs state that:

Amazon S3 offers eventual consistency for overwrite PUTS and DELETES in all regions.

http://docs.aws.amazon.com/AmazonS3/latest/dev/Introduction.html#ConsistencyModel

The timespan until full consistency is reached can vary. During this period GET requests may return the previous object or the udpated object.

My question is:

When is the last-modified timestamp updated? Is it updated immediately after the overwrite PUT succeeds but before full consistency is reached, or is it only updated after full consistency is achieved?

I suspect the former but I can't find any documentation which clearly states this.

like image 756
andrasp Avatar asked Nov 19 '16 21:11

andrasp


1 Answers

The Last-Modified timestamp should match the Date value returned in the response headers from the successful PUT request.

To my knowledge, this is not explicitly documented, but it can be derived from what is documented.

When you overwrite an object, it's not the overwriting itself that may be delayed by the eventual consistency model -- it's the availability of the overwritten content at a given S3 node (S3 is replicated to multiple nodes within the S3 region).

The Last-Modified timestamp, like the rest of the metadata, is established at the time of object creation and immutable, thereafter.

It is, in fact, not the "modification" time of the object at all, it is the creation time of the object. The explanation may sound pedantic, but it is accurate in the strictest sense: S3 objects and their metadata cannot in fact be modified at all, they can only be overwritten. When you "overwrite" an object in S3, what you are actually doing is creating a new object, reusing the old object's key (path+file name). The availability of this new object at a given S3 node (replication) is what may be delayed by the eventual consistency model... not the actual creation of the new object that overwrites the old one... hence there would be no reason for Last-Modified to be impacted by the replication delay (assuming there is a replication delay -- eventual consistency can at times be indistinguishable from immediate consistency).

like image 128
Michael - sqlbot Avatar answered Oct 05 '22 00:10

Michael - sqlbot