Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

aws S3 400 Bad Request

Tags:

amazon-s3

I'm attempting to narrow down the following 400 Bad Request error:

com.amazonaws.services.s3.model.AmazonS3Exception: Bad Request (Service: Amazon S3; Status Code: 400; Error Code: 400 Bad Request; Request ID: 7FBD3901B77A07C0), S3 Extended Request ID: +PrYXDrq9qJwhwHh+DmPusGekwWf+jmU2jepUkQX3zGa7uTT3GA1GlmHLkJjjjO67UQTndQA9PE=
    at com.amazonaws.http.AmazonHttpClient.handleErrorResponse(AmazonHttpClient.java:1343)
    at com.amazonaws.http.AmazonHttpClient.executeOneRequest(AmazonHttpClient.java:961)
    at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:738)
    at com.amazonaws.http.AmazonHttpClient.doExecute(AmazonHttpClient.java:489)
    at com.amazonaws.http.AmazonHttpClient.executeWithTimer(AmazonHttpClient.java:448)
    at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:397)
    at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:378)
    at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4039)
    at com.amazonaws.services.s3.AmazonS3Client.getObjectMetadata(AmazonS3Client.java:1177)
    at com.amazonaws.services.s3.AmazonS3Client.getObjectMetadata(AmazonS3Client.java:1152)
    at com.amazonaws.services.s3.AmazonS3Client.doesObjectExist(AmazonS3Client.java:1212)
    at com.abcnews.apwebfeed.articleresolver.APWebFeedArticleResolverImpl.makeS3Crops(APWebFeedArticleResolverImpl.java:904)
    at com.abcnews.apwebfeed.articleresolver.APWebFeedArticleResolverImpl.resolve(APWebFeedArticleResolverImpl.java:542)
    at sun.reflect.GeneratedMethodAccessor62.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.codehaus.xfire.service.invoker.AbstractInvoker.invoke(AbstractInvoker.java:54)
    at org.codehaus.xfire.service.binding.ServiceInvocationHandler.sendMessage(ServiceInvocationHandler.java:322)
    at org.codehaus.xfire.service.binding.ServiceInvocationHandler$1.run(ServiceInvocationHandler.java:86)
    at java.lang.Thread.run(Thread.java:662)

I'm testing something as imple as this:

boolean exists = s3client.doesObjectExist("aws-wire-qa", "wfiles/in/wire.json");

I manually added the wfiles/in/wire.json file. I get back true when I run this line inside a local app. But inside a separate remote service it throws the error above. I use the same credentials inside the service as I use in my local app. I also set bucket as "Enable website hosting", but no difference. My permissions are set as:

Grantee: Any Authenticated AWS User
y List 
y Upload/DeleteView 
y PermissionsEdit Permissions    

So I thought the error could be related to not having a policy on the bucket and created a policy file on the bucket for GET/PUT/DELETE objects, but I'm still getting the same error. My policy look like this:

{
"Version": "2012-10-17",
"Id": "Policy1481303257155",
"Statement": [
    {
        "Sid": "Stmt1481303250933",
        "Effect": "Allow",
        "Principal": {
            "AWS": "arn:aws:iam::755710071517:user/law"
        },
        "Action": [
            "s3:DeleteObject",
            "s3:GetObject",
            "s3:PutObject"
        ],
        "Resource": "arn:aws:s3:::aws-wire-qa/*"
    }
]
}

I was told it can't be a firewall or a proxy issue. What else I could try? The error is very non-specific. And so far I did only local development, so I have no idea what else can be not set up here. Would much appreciate some help here.

like image 809
user2917629 Avatar asked Dec 14 '16 00:12

user2917629


People also ask

Why is my S3 bucket Access Denied?

The "403 Access Denied" error can occur due to the following reasons: Your AWS Identity and Access Management (IAM) user or role doesn't have permissions for both s3:GetBucketPolicy and s3:PutBucketPolicy. The bucket policy denies your IAM identity permission for s3:GetBucketPolicy and s3:PutBucketPolicy.

When working with S3 through the API you get an error response as 409 conflict What could be the reason for this?

You are working with the S3 API and receive an error: 409 Conflict. What is a possible cause of this error? You're attempting to delete a bucket without first removing the contents in the bucket. Explanation: A 409 HTTP Status Code can indicate a BucketNotEmpty error code.

How many requests can an S3 bucket handle?

You can send 3,500 PUT/COPY/POST/DELETE or 5,500 GET/HEAD requests per second per prefix in an Amazon S3 bucket. There are no limits to the number of prefixes that you can have in your bucket.

What is the maximum file size on S3?

Individual Amazon S3 objects can range in size from a minimum of 0 bytes to a maximum of 5 TB. The largest object that can be uploaded in a single PUT is 5 GB.


2 Answers

I have been getting this error for days, and in every case it was because my temporary access token had expired (or because I'd inadvertently built an instance of hdfs-site.xml containing an old token into a JAR). It had nothing to do with regions.

like image 158
AbuNassar Avatar answered Jan 03 '23 12:01

AbuNassar


curl -XPUT 'http://localhost:9200/_snapshot/repo_s3' -d '{
      "type": "s3",
      "settings": {
        "bucket": "my-bucket",
        "base_path": "/folder/in/bucket",
        "region": "eu-central"     
      }
}'

In my case that was a region issue!
I had to remove the region from the elasticsearch.yml and set in the command. If i don't remove the region from the yml file, elastic won't start (with the latest s3-repository plugin)

Name: repository-s3
Description: The S3 repository plugin adds S3 repositories
Version: 5.2.2
 * Classname: org.elasticsearch.plugin.repository.s3.S3RepositoryPlugin
like image 32
true_gler Avatar answered Jan 03 '23 13:01

true_gler