I am using golang sdk to upload files to a bucket on amazon S3.The response gives no error and the file is uploaded successfully. My problem is when I am listing the objects of the bucket in the same region in which I have uploaded a new .jpg
file. I am getting same files list with no new file added to the Bucket. I don't know what I am doing wrong. This is the code that I am using to list objects inside the bucket.
input := &s3.ListObjectsInput{
Bucket: aws.String("Bucket Name"),
}
result2, err := svc.ListObjects(input)
if err != nil {
if aerr, ok := err.(awserr.Error); ok {
switch aerr.Code() {
case s3.ErrCodeNoSuchBucket:
fmt.Println(s3.ErrCodeNoSuchBucket, aerr.Error())
default:
fmt.Println(aerr.Error())
}
} else {
// Print the error, cast err to awserr.Error to get the Code and
// Message from an error.
fmt.Println(err.Error())
}
return
}
log.Println("Bucket List", result2)
s3 is eventually consistent and may not show an object that has been recently written in a list, even from within the same zone:
Amazon S3 achieves high availability by replicating data across multiple servers within Amazon's data centers. If a PUT request is successful, your data is safely stored. However, information about the changes must replicate across Amazon S3, which can take some time, and so you might observe the following behaviors:
A process writes a new object to Amazon S3 and immediately lists keys within its bucket. Until the change is fully propagated, the object might not appear in the list.
A process replaces an existing object and immediately attempts to read it. Until the change is fully propagated, Amazon S3 might return the prior data.
A process deletes an existing object and immediately attempts to read it. Until the deletion is fully propagated, Amazon S3 might return the deleted data.
A process deletes an existing object and immediately lists keys within its bucket. Until the deletion is fully propagated, Amazon S3 might list the deleted object.
see S3 developer guide
There is a polling Go AWS API call "func (*S3) WaitUntilObjectExists" which keeps looking for a file until it appears or a time out happens
Update s3 is now strongly consistent thanks to @Tensiba for pointing this out!
Amazon S3 provides strong read-after-write consistency for PUT and DELETE requests of objects in your Amazon S3 bucket in all AWS Regions. This behavior applies to both writes to new objects as well as PUT requests that overwrite existing objects and DELETE requests. In addition, read operations on Amazon S3 Select, Amazon S3 access controls lists (ACLs), Amazon S3 Object Tags, and object metadata (for example, the HEAD object) are strongly consistent.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With