Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to add encryption to boto3.s3.transfer.TransferConfig for s3 file upload

I am trying to upload a file to s3 using boto3 file_upload method. This is pretty straight forward until server side encryption is needed. In the past I have used put_object to achieve this.

Like so:

import boto3
s3 = boto3.resource('s3')
s3.Bucket(bucket).put_object(Key=object_name,
                             Body=data,
                             ServerSideEncryption='aws:kms',
                             SSEKMSKeyId='alias/aws/s3')

I now want to upload files directly to s3 using the file_upload method. I can't find how to add server side encryption to the file_upload method. The file_upload method can take a TransferConfig but I do not see any arguments that set the encryption but I do see them in S3Transfer.

I am looking for something like this:

import boto3
s3 = boto3.resource('s3')
tc = boto3.s3.transfer.TransferConfig(ServerSideEncryption='aws:kms',
                                      SEKMSKeyId='alias/aws/s3')
s3.upload_file(file_name, 
               bucket, 
               object_name,
               Config=tc)

boto3 documentation

  • file_upload
  • TransferConfig
like image 638
Arthur Putnam Avatar asked Oct 06 '17 16:10

Arthur Putnam


1 Answers

I was able to come up with two solutions with jarmod's help.

Using boto3.s3.transfer.S3Transfer

import boto3
client = boto3.client('s3', 'us-west-2')
transfer = boto3.s3.transfer.S3Transfer(client=client)
transfer.upload_file(file_name,
                     bucket, 
                     key_name,
                     extra_args={'ServerSideEncryption':'aws:kms', 
                                 'SSEKMSKeyId':'alias/aws/s3'}
)

Using s3.meta.client

import boto3
s3 = boto3.resource('s3')
s3.meta.client.upload_file(file_name, 
                           bucket, key_name, 
                           ExtraArgs={'ServerSideEncryption':'aws:kms',
                                      'SSEKMSKeyId':'alias/aws/s3'})
like image 91
Arthur Putnam Avatar answered Oct 02 '22 20:10

Arthur Putnam