How can I run the aws-cli in an AWS Lambda Python 3.6 environment?

Question

I would like to call the aws s3 sync command from within an AWS Lambda function with a runtime version of Python 3.6. How can I do this?

Why don't you just use the included boto3 SDK?

• boto3 does not have an equivalent to the sync command
• boto3 does not automatically find MIME types ("If you do not provide anything for ContentType to ExtraArgs, the end content type will always be binary/octet-stream.")
• aws cli does automatically find MIME types ("By default the mime type of a file is guessed when it is uploaded")

Architecturally this doesn't make sense!

For my use case I think it makes sense architecturally and financially, but I'm open to alternatives. My Lambda function:

• downloads Git and Hugo
• downloads my repository
• runs Hugo to generate my small (<100 pages) website
• uploads the generated files to s3

Right now, I'm able to do all of the above on a 1536 MB (the most powerful) Lambda function in around 1-2 seconds. This function is only triggered when I commit changes to my website, so it's inexpensive to run.

Maybe it is already installed in the Lambda environment?

As of the time of this writing, it is not.

Answer 1

From Running aws-cli Commands Inside An AWS Lambda Function:

import subprocess
command = ["./aws", "s3", "sync", "--acl", "public-read", "--delete",
           source_dir + "/", "s3://" + to_bucket + "/"]
print(subprocess.check_output(command, stderr=subprocess.STDOUT))

The AWS CLI isn't installed by default on Lambda, so you have to include it in your deployment. Despite running in a Python 3.6 Lambda environment, Python 2.7 is still available in the environment, so the approach outlined in the article will continue to work.

To experiment on Lambda systems, take a look at lambdash.