Unzip .zip file and transfer to s3 bucket using python and boto 3

Question

I'm trying to unzip a .zip file in an s3 bucket without downloading it to my computer and copy it to another s3 bucket.

I've already got the file to copy over to the other bucket, now I just need to unzip it.

I'm new to python so if you could explain it as well that would be awesome!

import boto3 

# set buckets
s3bucket = mybucket
otherbucket = prodbucket

# pass the access keys as variables into the boto session
session = Session(aws_access_key_id=awsAccessKeyID,
                  aws_secret_access_key=awsSecretAccessKey)

# specify s3 connection
s3 = session.resource('s3')
your_bucket = s3.Bucket(s3bucket)

# set main bucket as copy source
copy_source = {
      'Bucket': s3bucket,
      'Key': mykey
    }
# set staging bucket as bucket variable
bucket = s3.Bucket(otherbucket)



# copy files from main bucket and set the key
bucket.copy(copy_source, otherkey)

Answer 1

This is explained well here: How to extract files in S3 on the fly with boto3?

S3 itself does not modify files. Which leaves you with the options of download, extract the content locally with code, upload (which you stated isn't preferred), or trigger an AWS Lambda function that extracts the file into a temporary space in the cloud with code and then uploads it to your bucket.

Either way you'll need python code using the zipfile library, it's just a matter of running the code on your computer, or running the code on AWS resources (Lambda).

Getting started with Lambda: https://docs.aws.amazon.com/lambda/latest/dg/getting-started-create-function.html

Extract Zipfile with zipfile examples: https://docs.python.org/3/library/zipfile.html