Is it possible to correctly point to Python Shapely library's LIBGEOS_C in AWS Lambda environment?

Question

I am attempting to write a AWS python Lambda function that utilize's Shapely for simple "point in polygon" operations. I spun up an AWS linux EC2 instance, installed Shapely and had a working script. I then downloaded the linux specific libgeos_c.so.1 binary from my EC2 instance and bundled it (via serverless framework) to exist in the same directory as my lambda function. However once I deploy the script will not execute because it throws a "Could not find library or load any of its variants..." error. I've even tried to explicitly point to libgeos_c.so.1 path via the GEOS_LIBRARY_PATH environment variable to no effect. Is this an impossible deployment?

Here is a code snippet that sets the environment variable and then invokes a secondary script which actually imports and utilizes shapely.

import sys
import os
import subprocess

here = os.path.dirname(os.path.realpath(__file__))

# Import installed packages (in site-packages)
site_pkgs = os.path.join(here, "venv", "lib", "python2.7", "site-packages") 
sys.path.append(site_pkgs)
import json

def hello(event, context):

    command = "GEOS_LIBRARY_PATH={} python test_geo_worker.py".format(here + "/libgeos_c.so.1")
    foo = subprocess.check_output(command, shell=True)
    print foo

Has anyone every successfully deployed shapely in lambda? My fallback plan is to go back to good old postgres/postgis instead of shapely, but I'd sure like to try to build this out in a dynamo/lambda stack.

Answer 1

As I have hit this problem and managed to find a solution, I thought I'd post it here.

Problem is that he necisary compiled libraries don't exist within Lambda, if you include libraries compiled by some other Linux, unless they are built with the same compiler and dependencies are not going to work.

Thankfully a nice chap has figured out the dependencies and built packages for a variety of Python modules that are not included in Lambda, including shapely.

https://github.com/ryfeus/lambda-packs

download the relevant module from there and copy it into the deployment package (removing any you have installed via pip beforehand).