Using aws encryption SDK in python AWS lambda

Question

I tried using the aws encryption lib to encrypt/decrypt data with a KMS key in an AWS Lambda ( using python ). However, I get errors when running the lambda ( complaining about shared libs not found, I will update later with exact lib ). I am guessing that the SDK is using shared libs that are not installed in the AWS lambda environment.

This is the SDK I am using: https://docs.aws.amazon.com/encryption-sdk/latest/developer-guide/python.html

When building my package I am doing pip install aws-encryption-sdk and cryptography.

Anyone who can give me pointers how to fix it?

Update:

Here is the error message:

Unable to import module 'lambdaMain': libffi-d78936b1.so.6.0.4: cannot open shared object file: No such file or directory

Update 2: For those wondering about the solution. I built the library on the wrong platform. It's really important to do the pip install etc on an amazon AMI that's compatible with whatever lambda runtime you are using.

Answer 1

The aws-encryption-sdk requires cryptography library:

The SDK requires the cryptography library on all platforms. All versions of pip install and build the cryptography library on Windows. pip 8.1 and later installs and builds cryptography on Linux. If you are using an earlier version of pip and your Linux environment doesn't have the tools needed to build the cryptography library, you need to install them. For more information, see Building cryptography on Linux.

It seems the Lambda environment does not have the required libraries, so you need to add them.

Unfortunately Miserlou's lambda-packages does not have it, so you need to compile yourself.

Couple pointers:

• Using moviepy, scipy and numpy in amazon lambda
• Running Python with compiled code on AWS Lambda
• Using Native Dependencies with AWS Lambda

Answer 2

I had multiple issues when trying to package my libs / requirements and heres the steps that resolved the issue for me:

1) Create a new EC2 instance using Amazon Linux AMI

2) Install any requirements sudo yum install python36 python36-virtualenv python36-pip -y

3) Create new virtual enviroment using python3 virtualenv -p python3 .

4) Install any requirements using pip3 pip3 install aws_encryption_sdk

5) Everything in both lib and lib64 must be added (including hidden files)

zip -r -9 /tmp/export.zip lib64/python3.6/site-packages/* lib64/python3.6/site-packages/.*
zip -r -9 /tmp/export.zip lib/python3.6/site-packages/* lib/python3.6/site-packages/.*
zip -r -9 /tmp/export.zip main.py   

Hope this helps.