Boto3: Verify if the file has been uploaded using upload_file()

Question

I'm making a backup script using boto3 using this line to upload the item:

bucket.upload_file(backup_file_name,bucket_path+backup_file_name)

How can I verify if this file has really been uploaded to my bucket? I tried to find something using boto3 doc but I didn't found anything. Does this method throws any exception that I can handle?

Answer 1

It will raise boto3.exceptions.S3UploadFailedError. You can also do a head_object request to verify that the object looks like it should. This will raise a botocore.ClientError with the code 404 if the object does not exist.

Answer 2

There is few ways to check. First, you should always make sha256 hash for your file. When you upload, remember to put this info inside the Meta part of the object upload script.

• read the whole file, retrieve the sha256 has from the header meta, and recalculate the has to check it is tally.
• Add an events for the bucket on PUT. When file successfully uploaded, send the events to a queue which you can inspect. So in success upload but your program crash afterwards, you still have a fall back than reupload the file again.