How to load big file model in a lambda function

Question

Say, I want a lambda function to predict incoming message category with a trained model. However, the model is over-sized (~ 1GB).

With current architecture, I should upload the trained model to AWS S3 and then load it every time the lambda is triggered. This is not desirable since most of time is loading the model.

Some solution in mind:

1. Don't use lambda. Have a dedicated ec2 instance to work
2. Keep in warm by periodically sending dummy request

Or, I suspect AWS will cache the file, so the next loading time could be shorter?

Answer 1

I think reading about container reuse in lambda could be helpful here.

https://aws.amazon.com/blogs/compute/container-reuse-in-lambda/

You can add the model as a global cached variable by declaring and initialising it outside the handler function. And if Lambda is reusing the same container for subsequent requests the file won't be re-downloaded.

But it's entirely up to Lambda whether to reuse the container or start a new one. Since this is Lambda's prerogative you can't depend on this behaviour.

If you want to minimise the number of downloads from S3 maybe using an external managed caching solution (Elasticache, Redis) in the same AZ as your function is a possible alternative you can look at.