Upload file from memory to S3

Question

I have downloaded a csv file from S3 into memory and edited the file using Boto3 and Python. How can I can reupload this file to S3 without ever storing it locally?

Answer 1

As per @JamesMchugh, from put_object():

response = client.put_object(
    Body=b'bytes'|file,
    Bucket='string',
    Key='string',
)

Answer 2

In my case, I have a list of dictionaries and I have to create in memory file and save that on S3. Following Code works for me!

import csv
import boto3
from io import StringIO
# input list
list_of_dicts = [{'name': 'name 1', 'age': 25}, {'name': 'name 2', 'age': 26}, {'name': 'name 3', 'age': 27}]

# convert list of dicts to list of lists
file_data = []
header = list(list_of_dicts[0].keys())
file_data = [[d[key] for key in header] for d in list_of_dicts]
file_data = [header] + file_data

# create in memory file and write data to it.
file_to_save = StringIO()
csv.writer(file_to_save).writerows(file_data)
file_to_save = bytes(file_to_save.getvalue(), encoding='utf-8')
file_name_on_s3 = 'my_data.csv'

# save in memory file to S3
client = boto3.client('s3',
                      aws_access_key_id='your access key',
                      aws_secret_access_key='your secret key')
response = client.put_object(
    Body=file_to_save,
    Bucket='your bucket name',
    Key=file_name_on_s3,
)