saving csv file to s3 using boto3

Question

I am trying to write and save a CSV file to a specific folder in s3 (exist). this is my code:

from io import BytesIO
import pandas as pd
import boto3
s3 = boto3.resource('s3')

d = {'col1': [1, 2], 'col2': [3, 4]}
df = pd.DataFrame(data=d)

csv_buffer = BytesIO()

bucket = 'bucketName/folder/'
filename = "test3.csv"
df.to_csv(csv_buffer)
content = csv_buffer.getvalue()

def to_s3(bucket,filename,content):
  s3.Object(bucket,filename).put(Body=content)

to_s3(bucket,filename,content)

this is the error that I get:

Invalid bucket name "bucketName/folder/": Bucket name must match the regex "^[a-zA-Z0-9.\-_]{1,255}$"

I also tried :

bucket = bucketName/folder

and:

bucket = bucketName
key = folder/
s3.Object(bucket,key,filename).put(Body=content)

Any suggestions?

Answer 1

Saving into s3 buckets can be also done with upload_file with an existing .csv file:

import boto3
s3 = boto3.resource('s3')

bucket = 'bucket_name'
filename = 'file_name.csv'
s3.meta.client.upload_file(Filename = filename, Bucket= bucket, Key = filename)