How to upload a file to S3 without creating a temporary local file

Question

Is there any feasible way to upload a file which is generated dynamically to amazon s3 directly without first create a local file and then upload to the s3 server? I use python. Thanks

Answer 1

Here is an example downloading an image (using requests library) and uploading it to s3, without writing to a local file:

import boto from boto.s3.key import Key import requests  #setup the bucket c = boto.connect_s3(your_s3_key, your_s3_key_secret) b = c.get_bucket(bucket, validate=False)  #download the file url = "http://en.wikipedia.org/static/images/project-logos/enwiki.png" r = requests.get(url) if r.status_code == 200:     #upload the file     k = Key(b)     k.key = "image1.png"     k.content_type = r.headers['content-type']     k.set_contents_from_string(r.content) 

Answer 2

You could use BytesIO from the Python standard library.

from io import BytesIO bytesIO = BytesIO() bytesIO.write('whee') bytesIO.seek(0) s3_file.set_contents_from_file(bytesIO)