Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Uploading a file to a bucket in Amazon S3 failing with "Maximum number of retry attempts reached"

Tags:

c#

.net

amazon-s3

I have been trying to create buckets and upload files to Amazon S3 using their .net SDK. I am able to create the buckets and specify that they be created in the EU region. The code used to create the buckets is as below

PutBucketRequest request = new PutBucketRequest();
request.WithBucketName(bucketName)
       .WithBucketRegion(S3Region.EU);

client.PutBucket(request);

I then proceed to upload a file to the bucket using the following code:

PutObjectRequest request = new PutObjectRequest();
request.WithBucketName(bucketName)
    .WithCannedACL(S3CannedACL.PublicRead)
    .WithKey(remoteFileName)
    .WithInputStream(uploadFileStream);

The file upload code fails with the error "Maximum retry attempts reached."

Can anyone let me know what else I need to be doing in order for the upload to work?

Thanks.

EDIT: Trying to upload a file to the same bucket using the AWS Management Console works fine.

like image 279
Rajeev Shenoy Avatar asked Sep 13 '11 16:09

Rajeev Shenoy


People also ask

What is the max file size that we can upload to S3?

You can upload any file type—images, backups, data, movies, etc. —into an S3 bucket. The maximum size of a file that you can upload by using the Amazon S3 console is 160 GB. To upload a file larger than 160 GB, use the AWS CLI, AWS SDK, or Amazon S3 REST API.

Is there a limit to the number of files in S3 bucket?

There is no max bucket size or limit to the number of objects that you can store in a bucket. You can store all of your objects in a single bucket, or you can organize them across several buckets.

Can Amazon S3 uploads resume on failure or do they need to restart?

You can resume them, if you flag the “resume on failure” option before uploading.


1 Answers

I found the issue, at last.

When targeting buckets in a specific region, the Amazon S3 client object will have to be configured to use a specific endpoint. The code to configure the endpoint is as in the constructor and creation of the client is in the individual methods of the class below:

public class AmazonS3Service : IAmazonS3Service 
{
     private AmazonS3 client;
     private string accessKeyID;
     private string secretAccessKeyID;
     private AmazonS3Config config;

     public AmazonS3Service()
     {
         accessKeyID = ConfigurationManager.AppSettings["AWSAccessKey"];
         secretAccessKeyID = ConfigurationManager.AppSettings["AWSSecretKey"];
         config = new AmazonS3Config();
         config.ServiceURL = ConfigurationManager.AppSettings["AWSEUEndPoint"];
      }

    public void CreateBucket(string bucketName)
    {
        using (client = Amazon.AWSClientFactory.CreateAmazonS3Client(accessKeyID, secretAccessKeyID, config))
        {
            try
            {
                PutBucketRequest request = new PutBucketRequest();
                request.WithBucketName(bucketName)
                       .WithBucketRegion(S3Region.EU);

                 client.PutBucket(request);
             }
             catch (AmazonS3Exception amazonS3Exception)
             {
                if (amazonS3Exception.ErrorCode != null && (amazonS3Exception.ErrorCode.Equals("InvalidAccessKeyId") || amazonS3Exception.ErrorCode.Equals("InvalidSecurity")))
                {
                //log exception - ("Please check the provided AWS Credentials.");
                }
                else
                {
                //log exception - ("An Error, number {0}, occurred when creating a bucket with the message '{1}", amazonS3Exception.ErrorCode, amazonS3Exception.Message);    
                }
            }
        }
    }

     public void UploadFile(string bucketName, Stream uploadFileStream, string remoteFileName)
    {
        using (client = Amazon.AWSClientFactory.CreateAmazonS3Client(accessKeyID, secretAccessKeyID, config))
        {
            try
            {
                PutObjectRequest request = new PutObjectRequest();
                request.WithBucketName(bucketName)
                    .WithCannedACL(S3CannedACL.PublicRead)
                    .WithKey(remoteFileName)
                    .WithInputStream(uploadFileStream);

                 using (S3Response response = client.PutObject(request))
                 {
                    WebHeaderCollection headers = response.Headers;
                    foreach (string key in headers.Keys)
                    {
                        //log headers ("Response Header: {0}, Value: {1}", key, headers.Get(key));
                    }
                }
            }
            catch (AmazonS3Exception amazonS3Exception)
            {
                if (amazonS3Exception.ErrorCode != null && (amazonS3Exception.ErrorCode.Equals("InvalidAccessKeyId") || amazonS3Exception.ErrorCode.Equals("InvalidSecurity")))
                {
                //log exception - ("Please check the provided AWS Credentials.");
                }
                else
                {
                //log exception -("An error occurred with the message '{0}' when writing an object", amazonS3Exception.Message);
                }
            }
        }
    }
 }

The various endpoints of each of Amazon's services can be found at at this URL - http://docs.amazonwebservices.com/general/latest/gr/index.html?rande.html

Hope this helps someone!

like image 55
Rajeev Shenoy Avatar answered Nov 14 '22 23:11

Rajeev Shenoy