How to install Numpy and Pandas for AWS Lambdas?

Question

Problem: I wanted to use Numpy and Pandas in my AWS lambda function. I am working on Windows 10 with PyCharm. My function compiles and works fine on local machine, however, as soon as package it up and deploy on AWS, it breaks down giving errors in importing the numpy and pandas packages. I tried reinstalling both packages and then redeploying however, error remained the same.

StackOverFlow Solutions: Other people are having similar issues and fellow users have suggested that this is mainly compatibility issue, because Python libraries are compiled on Windows whereas, AWS Lambda runs on linux machines.

Question: What's the best way to create a deployment package for AWS on windows 10? Is there a way I can specify targeted platform while installing packages through PIP. Apparently there is an option in pip with tag --platform but I cannot figure out how to use it. Any helps?

Answer 1

Like often there is more than one way to come to a solution.

The preferred way imho is to use AWS lambda layers, because it separates the functional code from the dependencies. The basics are explained here.

1. Get all your dependencies. Like you mentioned correctly, pandas and numpy have to be compiled for the AMI Linux. This can be done with the tool: "serverless python requirements" or with a docker container based on this image. A more detailed instruction can be found here.
2. Put the dependencies in a folder called python.
3. zip the whole folder e.g. with the preinstalled windows zipping tool.
4. Upload the zip file to AWS as a layer: Go to AWS Lambda, from the left choose Layers and "Create a new layer".
5. After you saved the layer, go to your Lambda Function and choose "Layers". Click "Add a layer" choose your newly created layer and click on save. Now your function should not get import errors anymore.