I'm trying to upload a .zip file to S3 using boto3 for python but the .zip file in my directory is not uploaded correctly. The code downloads all emails of a given user, zips them in the same directory and uploads them to an S3 bucket. The problem is that the file that gets uploaded is not the one I intend to upload. Instead a file with 18kb only appears.
Here's the code:
import sys
import imaplib
import getpass
import email
import shutil
import boto3
import os
username = input("Enter user's first name: ")
surname = input("Enter user's surname: ")
email_address = username + "." + surname + "@gmail.com"
password = getpass.getpass()
directory = username + surname + '/'
def download_emails(server):
result, data = server.uid('search', None, "ALL") #search all email and return their uids
if result == 'OK':
for num in data[0].split():
result, data = server.uid('fetch', num, '(RFC822)') #RFC is a standard for the format of ARPA Internet text messages
if result == 'OK':
email_message = email.message_from_bytes(data[0][1]) #raw email text including headers
file_name = email_message['Subject'] #use dates and file names(can be changed)
if not os.path.exists(directory):
os.makedirs(directory) #create a dir for user's emails
try:
email_file = open(directory + file_name+'.eml', 'wb') #open a file for each email and insert the data.
email_file.write(data[0][1])
email_file.close()
except:
pass
#function to zip all the emails
def archive(zipname, directory):
return shutil.make_archive(zipname, 'zip', root_dir=directory, base_dir=None)
#function to upload zipped emails to AWS bucket
def upload_to_s3(file_name):
s3 = boto3.resource('s3',
aws_access_key_id=accessKey,
aws_secret_access_key=secretKey,
aws_session_token=secretToken,
)
s3.Bucket('user-backups').put_object(Key=username.title() + " " +
surname.title() + "/" + file_name, Body=file_name)
print("Uploaded")
def main():
server = imaplib.IMAP4_SSL("imap.gmail.com", 993) #connect to gmail's imap server
server.login(email_address, password) #enter creds
result, data = server.select('"[Gmail]/All Mail"') #get all emails(inbox, outbox etc)
if result == 'OK':
print("Downloading")
download_emails(server)
server.close()
else:
print("ERROR: Unable to open mailbox ", result)
server.logout()
archive(username + surname, directory)
upload_to_s3(username + surname + ".zip")
#os.remove(email_address + ".zip")
#shutil.rmtree(email_address)
print("Done")
if __name__ == "__main__":
main()
You can check out this article for more information.
There are a number of ways to upload. Check out this boto3 document where I have the methods listed below:
The managed upload methods are exposed in both the client and resource interfaces of boto3:
S3.Client method to upload a file by name: S3.Client.upload_file()
S3.Client method to upload a readable file-like object: S3.Client.upload_fileobj()
S3.Bucket method to upload a file by name: S3.Bucket.upload_file()
S3.Bucket method to upload a readable file-like object: S3.Bucket.upload_fileobj()
S3.Object method to upload a file by name: S3.Object.upload_file()
S3.Object method to upload a readable file-like object: S3.Object.upload_fileobj()
I made it work using s3.client.upload_file.
upload_file(Filename, Bucket, Key, ExtraArgs=None, Callback=None, Config=None) .
Upload a file to an S3 object.
import boto3
s3Resource = boto3.resource('s3')
try:
s3Resource.meta.client.upload_file('/path/to/file', 'bucketName', 'keyName')
except Exception as err:
print(err)
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With