Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

S3 Presigned URL Issue - file upload successful, 200 statusCode but no response body

This was working up until recently, I am not sure what has changed. I am generating a presignedUrl using S3 SDK and uploading a file to a bucket using it. The file actually uploads and the response returns a statusCode of 200 but weirdly there no response body.

I cannot understand if I'm missing some sort of header, or if they're wrong. The Content-length on the response headers worries me.

Any help would be much appreciated!

Note: I've obscured the values, if they are helpful I can mock them back in

General

Request URL: https://some-bucket.s3.ap-southeast-2.amazonaws.com/some/path/file/picture?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=ASIAXXXap-southeast-2%2Fs3%2Faws4_request&X-Amz-Date=xxx&X-Amz-Expires=300&X-Amz-Security-Token=xxx&X-Amz-Signature=xxx&X-Amz-SignedHeaders=host%3Bx-amz-acl&x-amz-acl=public-read Request Method: PUT Status Code: 200 OK Remote Address: 1.2.3.4:443 Referrer Policy: no-referrer-when-downgrade

Request headers

Accept: application/json, text/plain, */* Accept-Encoding: gzip, deflate, br Accept-Language: en-US,en;q=0.9 Cache-Control: no-cache Connection: keep-alive Content-Length: 31897 Content-Type: image/jpeg Host: some-bucket.s3.ap-southeast-2.amazonaws.com Origin: http://localhost:5000 Pragma: no-cache Referer: http://localhost:5000/some/page User-Agent: Mozilla/5.0 xxx

Response headers

Access-Control-Allow-Methods: GET, PUT, POST, HEAD Access-Control-Allow-Origin: * Content-Length: 0 Date: Thu, 1 May 2017 01:00:00 GMT ETag: "xxx" Server: AmazonS3 Vary: Origin, Access-Control-Request-Headers, Access-Control-Request-Method x-amz-id-2: xxx x-amz-request-id: xxx

Query String Parameters

X-Amz-Algorithm: AWS4-HMAC-SHA256 X-Amz-Credential: ASIAXXXap-southeast-2/s3/aws4_request X-Amz-Date: XXX X-Amz-Expires: 300 X-Amz-Security-Token: XXX X-Amz-Signature: XXX X-Amz-SignedHeaders: host;x-amz-acl x-amz-acl: public-read

Presigning service (NodeJS, Lambda)

... 
const params = {
    Bucket: 'some-bucket',
    Key: 'some/path/file/picture',
    Expires: 60 * 5,
    ACL: 'public-read'
};

s3.getSignedUrl('putObject', params, (err, url) => {
    ...
    callback(null, new Response(200, {url});
});
...

S3 service (TS, Angular)

public putObject(presignedUrl: string, file: File): Observable<any> {
  return this.http.put<any>(presignedUrl, file);
}

Component

this.s3Service.putObject(presignedUrl, file)
  .subscribe(
    (response) => {
      // it gets here as expected
      // but response is null!
    },
    () => {}
  );
like image 339
Matt Rowles Avatar asked May 24 '18 07:05

Matt Rowles


People also ask

How long is S3 Presigned URL valid for?

In the Amazon S3 console, the maximum expiration time for a presigned URL is 12 hours from the time of creation.

Which is the maximum S3 object size for upload in a single PUT operation?

Upload an object in a single operation using the AWS SDKs, REST API, or AWS CLI—With a single PUT operation, you can upload a single object up to 5 GB in size. Upload a single object using the Amazon S3 Console—With the Amazon S3 Console, you can upload a single object up to 160 GB in size.

Does S3 bucket need to be public for Presigned URL?

All objects and buckets are private by default. However, you can use a presigned URL to optionally share objects or allow your customers/users to upload objects to buckets without AWS security credentials or permissions.

How does Presigned URL work S3?

Generating a presigned URL for uploading objects When you use the URL to upload an object, Amazon S3 creates the object in the specified bucket. If an object with the same key that is specified in the presigned URL already exists in the bucket, Amazon S3 replaces the existing object with the uploaded object.


1 Answers

I reproduced it using go SDK to confirm it is how API itself behaves, not something specific from nodejs.

As I can see from my experimentation it is a normal behavior now, it will not return anything from PUT request.

OBS: I masked some sensitive values!

The Code:

package main

import (
    "crypto/tls"
    "fmt"
    "log"
    "net/http"
    "net/url"
    "strings"
    "time"

    "github.com/aws/aws-sdk-go/aws"
    "github.com/aws/aws-sdk-go/aws/session"
    "github.com/aws/aws-sdk-go/service/s3"
)

func main() {
    sess, err := session.NewSession(&aws.Config{
        Region: aws.String("eu-west-1")},
    )

    svc := s3.New(sess)

    req1, _ := svc.PutObjectRequest(&s3.PutObjectInput{
        Bucket: aws.String("bucketversioningenabled"),
        Key:    aws.String("myKey"),
        Body:   strings.NewReader("EXPECTED CONTENTS"),
    })

    presignURL, err := req1.Presign(time.Minute * 1)
    if err != nil {
        log.Println("Error on presign", err)
        return
    }
    fmt.Println("Presign: ", presignURL, err)
    fmt.Println("")

    req2, err := http.NewRequest("PUT", presignURL, strings.NewReader("EXPECTED CONTENTS"))
    if err != nil {
        log.Println("error creating request", err)
        return
    }
    fmt.Println("NewRequest: ", req2, err)
    fmt.Println("")

    proxyURL, err := url.Parse("http://myfiddler.proxy.com:8888")
    if err != nil {
        log.Println("Error on proxy parse", err)
        return
    }

    tr := &http.Transport{
        TLSClientConfig: &tls.Config{
            InsecureSkipVerify: true,
        },
        Proxy: http.ProxyURL(proxyURL),
    }
    client := &http.Client{
        Transport: tr,
        Timeout:   time.Duration(5 * time.Second),
    }

    resp, err := client.Do(req2)
    if err != nil {
        log.Println("error on request put", err)
        return
    }
    fmt.Println("Do: ", resp, err)
}

The execution output:

Please look the Content-Length:[0] on the last line.

$ ./s3put 
Presign:  https://bucketversioningenabled.s3.eu-west-1.amazonaws.com/myKey?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=XXXXX%2F20180531%2Feu-west-1%2Fs3%2Faws4_request&X-Amz-Date=20180531T112707Z&X-Amz-Expires=60&X-Amz-Security-Token=XXXX&X-Amz-SignedHeaders=content-length%3Bhost&X-Amz-Signature=XXXX <nil>

NewRequest:  &{PUT https://bucketversioningenabled.s3.eu-west-1.amazonaws.com/myKey?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=XXXX%2F20180531%2Feu-west-1%2Fs3%2Faws4_request&X-Amz-Date=20180531T112707Z&X-Amz-Expires=60&X-Amz-Security-Token=XXXX&X-Amz-SignedHeaders=content-length%3Bhost&X-Amz-Signature=XXXX HTTP/1.1 1 1 map[] {0xc42010d020} 0x5f2040 17 [] false bucketversioningenabled.s3.eu-west-1.amazonaws.com map[] map[] <nil> map[]   <nil> <nil> <nil> <nil>} <nil>

Do:  &{200 OK 200 HTTP/1.1 1 1 map[X-Amz-Version-Id:[3I4txVUgi4ObULr8EVadA4U3cfvdVwQM] Etag:["952973475e3f4d992fe48578086c1e17"] Content-Length:[0] Server:[AmazonS3] X-Amz-Id-2:[yGUZtjttGKwv0uJxQcG7bIkGRqxhPxKeW71jWIGkmwt73oZY/+r3HWyr2uK07nR8xTDQyzbM3Hw=] X-Amz-Request-Id:[509F1785D0383ADA] Date:[Thu, 31 May 2018 11:27:09 GMT]] 0xc42000c0e0 0 [] false false map[] 0xc42019c400 0xc420199290} <nil>

And below you can see the request and response from Fiddler

Request:

PUT https://bucketversioningenabled.s3.eu-west-1.amazonaws.com/myKey?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=XXXX%2F20180531%2Feu-west-1%2Fs3%2Faws4_request&X-Amz-Date=20180531T112707Z&X-Amz-Expires=60&X-Amz-Security-Token=XXXX&X-Amz-SignedHeaders=content-length%3Bhost&X-Amz-Signature=XXXX HTTP/1.1
Host: bucketversioningenabled.s3.eu-west-1.amazonaws.com
User-Agent: Go-http-client/1.1
Content-Length: 17
Accept-Encoding: gzip

EXPECTED CONTENTS

Response (look the Content-Length):

HTTP/1.1 200 OK
x-amz-id-2: yGUZtjttGKwv0uJxQcG7bIkGRqxhPxKeW71jWIGkmwt73oZY/+r3HWyr2uK07nR8xTDQyzbM3Hw=
x-amz-request-id: 509F1785D0383ADA
Date: Thu, 31 May 2018 11:27:09 GMT
x-amz-version-id: 3I4txVUgi4ObULr8EVadA4U3cfvdVwQM
ETag: "952973475e3f4d992fe48578086c1e17"
Content-Length: 0
Server: AmazonS3
like image 198
Azize Avatar answered Oct 16 '22 17:10

Azize