Import a python module in multiple AWS Lambdas

Question

I have a couple of AWS Lambda functions. All of those functions use some common helper functions. I have placed these helper functions in a separate file called helper_functions.py. I want to import this module in all of my AWS Lambda functions. I am unable to find a place to store this module (helper_functions.py), so when I make a change in this module I don't have to change anything in my Lambda functions.

Some of the options I thought of are:

1. Uploading the module on AWS S3 and then loading it in each Lambda function in the start from S3 and using the functions. (if possible)

2. Writing some script (which I haven't figured out yet) that packages the module along with the Lambda functions' Python file in a zip and uploads it on AWS Lambda

Please suggest a better solution to manage the module and import it in a much more efficient way.

Answer 1

I struggled with this for a long time. Here's my solution (there might be a better way):

setup your helper function in your file system like this:

pathToSomewhere/my_helper/helper_functions.py pathToSomewhere/my_helper/__init__.py pathToSomewhere/setup.py

Where __init__.py is:

from .helper_functions import *

and setup.py is

from setuptools import setup

setup(name='my_helper',
      version='0.10000',
      description='My helper functions',
      url='http://github.com/user/example',
      license='Proprietary',
      author='Null',
      author_email='[email protected]',
      packages=['my_helper'],
      install_requires=['boto3'],
      zip_safe=False)

Now let's package up my_helper. From pathToSomewhere/ run:

python setup.py sdist

I'm assuming you already know how to create and upload a virtual environment for running your lambda function. If not, let me know.

Now let's install my_helper into the virtual env of your lambda function. Let's assume your virtual environment is called worker_env

./worker-env/bin/pip install file://pathToSomewhere/my_helper

Now zip up worker-env and your actual lambda script and upload that.