AWS Lambda (python2.7): Adding matplotlib as a layer

Question

I am having trouble trying to add matplotlib as a layer to my Python 2.7 AWS Lambda function.

On the Lambda execution environment, I am trying to install the necessary libraries and create a layer as described here.

Things I've tried:

First, I pip installed matplotlib into a virtual environment and copied the contents of the site-packages under lib and lib64. When the lambda function is executed, I get a No module named pkg_resources exception. I also tried installing with the --target option to install all dependancies to the same folder. The result was the same.

I read here that it may be due to outdated setuptools package. When I did an update pip install --upgrade setuptools and then tried to install matplotlib I started getting the following exception:

pkg_resources.DistributionNotFound: The 'pip==9.0.3' distribution was not found and is required by the application 

Finally I thought of installing matplotlib with

sudo yum install python-matplotlib

and then collect the required packages as described here. But this did not make matplotlib importable from within the python shell, so I guess it won't work as a Lambda layer.

Thanks for any help.

P.S: At AWS re:invent, exactly this was demoed but there are no details on the session :/

Answer 1

I encountered similar issues with other modules such as crypto and my own custom modules. I discovered the problem is really a lack of good documentation.

In my case, I was zipping all the dependencies up from the target directory, using the --target option, so all the dependency directories were at the top level of the zip file. That works fine for straight Lambda deployment, but when you want to use a layer, the layer is deployed in to the /opt folder of your Lambda container, so you need to create your zip file with a top-level directory named 'python' so that your dependencies can be located at /opt/python/.

mkdir python && cd python && pip install pyopenssl crypto --target . && cd .. && zip -r9 ./lambda_layer.zip python/

It does appear in the documentation but it is brief and VERY easy to miss. This page helped me: https://medium.com/@adhorn/getting-started-with-aws-lambda-layers-for-python-6e10b1f9a5d

Good luck!