Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Etag definition changed in Amazon S3

Tags:

amazon-s3

I've used Amazon S3 a little bit for backups for some time. Usually, after I upload a file I check the MD5 sum matches to ensure I've made a good backup. S3 has the "etag" header which used to give this sum.

However, when I uploaded a large file recently the Etag no longer seems to be a md5 sum. It has extra digits and a hyphen "696df35ad1161afbeb6ea667e5dd5dab-2861" . I can't find any documentation about this changing. I've checked using the S3 management console and with Cyberduck.

I can't find any documentation about this change. Any pointers?

like image 813
jjh Avatar asked Jul 06 '11 02:07

jjh


People also ask

What is ETag in S3 object?

ETag. The entity tag is a hash of the object. The ETag reflects changes only to the contents of an object, not its metadata. The ETag may or may not be an MD5 digest of the object data.

Are S3 tags metadata?

If you want to add your own custom metadata to an object in S3, you can add metadata instead of tags. Tags are not the same thing as object metadata in S3. Metadata applies only to that object in S3 and cannot be searched on, as you can search with tags.

Does S3 overwrite file with same name?

By default, when you upload the file with same name. It will overwrite the existing file. In case you want to have the previous file available, you need to enable versioning in the bucket.

How can you protect S3 data from being overwritten?

Using Amazon S3 Object Lock, you can prevent an object from being deleted or overwritten for a fixed amount of time, or until the legal hold is removed. An object version can have either a combination or both a retention period and a legal hold.


1 Answers

If any file is being uploaded with multipart then you will always get such type of ETag. But if you upload whole file as single file then you will get ETag as before.

Bucket Explorer providing you normal ETag till 5Gb upload in multipart operation. But more then it is not providing.

AWS:

The ETag for an object created using the multipart upload api will contain one or more non-hexadecimal characters and/or will consist of less than 16 or more than 16 hexadecimal digits.

Reference: https://forums.aws.amazon.com/thread.jspa?messageID=203510#203510

like image 147
Tej Kiran Avatar answered Sep 20 '22 02:09

Tej Kiran