How to use Tensorflow Lite on AWS Lambda

Question

I'm trying to host a small model I have compiled down to a .tflite on AWS Lambda. Using either the python 3.6 or python 3.7 tflite wheel files available on the tensorflow website I zip up my packages/code, and upload to S3 and link to lambda with space to spare. However, when I test my function it crashes when trying to load tflite. Initially, it couldn't load shared object files. This was the error

[ERROR] Runtime.ImportModuleError: Unable to import module 'lambda_predict': No module named '_interpreter_wrapper')

I found this shared object file and moved it up into the local directory, and then got another error

Unable to import module 'lambda_predict': /lib64/libm.so.6: version `GLIBC_2.27' not found (required by /var/task/_interpreter_wrapper.so)

My base system is Ubuntu (Bionic Beaver) Both these errors come from importing tflite

Answer 1

Okay I solved this today.

Complete solution and compiled dependencies for amazonlinux/aws lambda on my github: https://github.com/tpaul1611/python_tflite_for_amazonlinux

So the problem is that aws lambda runs on amazonlinux which apparently needs a different compilation of the tflite _interpreter_wrapper than the one tensorflow is currently providing on their website. https://www.tensorflow.org/lite/guide/python

My solution was to compile it natively on amazonlinux using docker and the script that tensorflow is providing in their git repo. https://github.com/tensorflow/tensorflow/tree/master/tensorflow/lite/tools/pip_package

I created a Dockerfile:

FROM amazonlinux

WORKDIR /tflite

RUN yum groupinstall -y development
RUN yum install -y python3.7
RUN yum install -y python3-devel
RUN pip3 install numpy wheel
RUN git clone --branch v2.2.0-rc0 https://github.com/tensorflow/tensorflow.git
RUN sh ./tensorflow/tensorflow/lite/tools/pip_package/build_pip_package.sh
RUN pip3 install tensorflow/tensorflow/lite/tools/pip_package/gen/tflite_pip/python3/dist/tflite_runtime-2.2.0rc0-cp37-cp37m-linux_x86_64.whl

CMD tail -f /dev/null

and then ran the following commands:

docker build -t tflite_amazonlinux .
docker run -d --name=tflite_amazonlinux tflite_amazonlinux
docker cp tflite_amazonlinux:/usr/local/lib64/python3.7/site-packages .
docker stop tflite_amazonlinux

These commands output a folder called site-packages which contain the corretly compiled tflite python dependencies for amazonlinux and therefore also aws lambda.