How to generate a temporary url to upload file to Amazon S3 with boto library?

Question

I know how to download a file in this way: key.generate_url(3600)

But when I tried to upload: key.generate_url(3600, method='PUT'), the url didn't work. I was told: The request signature we calculated does not match the signature you provided. Check your key and signing method.

I cannot find example code on the boto homepage for how to use the function generate_url(method='PUT'). Does anyone here know how to use it for uploading? How to set the params for the path of upload file?

Answer 1

I found some time to experiment with this and here's what I found.

>>> import boto >>> c =boto.connect_s3() >>> fp = open('myfiletoupload.txt') >>> content_length = len(fp.read()) >>> c.generate_url(300, 'PUT', 'test-1332789015', 'foobar', headers={'Content-Length': str(content_length)}, force_http=True) 'http://test-1332789015.s3.amazonaws.com/foobar?Signature=oUARG45mR95utXsiQYRJNiCI4x4%3D&Expires=1333731456&AWSAccessKeyId=AKIAJOTCCJRP4C3NSMYA&Content-Length=16' 

I was then able to use curl to PUT the file to that url like this:

$ curl --request PUT --upload-file myfiletoupload.txt "http://test-1332789015.s3.amazonaws.com/foobar?Signature=oUARG45mR95utXsiQYRJNiCI4x4%3D&Expires=1333731456&AWSAccessKeyId=AKIAJOTCCJRP4C3NSMYA&Content-Length=16" 

This resulted in the file being uploaded to the bucket. So, it appears that it is possible. You might want to see if you can calculate the content-md5 value and include that in the headers but then you also have to figure out how to get curl to send that header, as well. Also, you should be able to make this work over HTTPS rather than HTTP but I haven't tried that.

Answer 2

Here's what it looks like in boto3(tested with version 1.2.3).

First, create a presigned url with s3.generate_presigned_url method:

>>> import boto3 >>> s3 = boto3.client('s3') >>> s3.generate_presigned_url('put_object', Params={'Bucket':'YourBucket', 'Key':'YourKey'}, ExpiresIn=3600, HttpMethod='PUT') u'https://s3-ap-northeast-1.amazonaws.com/YourBucket/YourKey?AWSAccessKeyId=AKIAXXXXXXXXXXXXXXXX&Expires=1451061671&Signature=%2FtyAyCd5vrp13p%2FqLdoPkox7yTM%3D' 

PUT to S3 with a presigned URL

$ curl \   --request PUT \   --upload-file path/to/file \   "https://s3-ap-northeast-1.amazonaws.com/YourBucket/YourKey?AWSAccessKeyId=AKIAXXXXXXXXXXXXXXXX&Expires=1451061671&Signature=%2FtyAyCd5vrp13p%2FqLdoPkox7yTM%3D"