Menu
I have developed a rest API using Flask to expose a Python Keras Deep Learning model (CNN for text classification). I have a very simple script that loads the model into memory and outputs class probabilities for a given text input. The API works perfectly locally.
However, when I git push heroku master, I get Compiled slug size: 588.2M is too large (max is 500M). The model is 83MB in size, which is quite small for a Deep Learning model. Notable dependencies include Keras and its tensorflow backend.
I know that you can use GBs of RAM and disk space on Heroku. But the bottleneck seems to be the slug size. Is there a way to circumvent this? Or is Heroku just not the right tool for deploying Deep Learning models?
599
You can see all the build libraries loaded and application deployment steps in the logs. These logs are more useful for debugging if you have any errors/failures during deployment. And that's it! Using these simple steps, any machine learning model can be deployed on Heroku to get real-time predictions.
Heroku executes applications by running a command you specified in the Procfile, on a dyno that's been preloaded with your prepared slug (in fact, with your release, which extends your slug and a few items not yet defined: config vars and add-ons).
Heroku is a very good cloud platform to deploy your apps but if you have a Deep Learning model i.e. an app that needs to predict using large CNN / Deep Learning models then this cloud is not suitable. You can try other cloud platforms like AWS, Amazon Sagemaker, MS Azure, IBM Watson.
I was facing the same issue and after spending several days I came to know it was tensorflow library that was causing this slug overhead.
I solved it using 1 line in the requirements.txt file:
tensorflow-cpu==2.5.0
Instead of
tensorflow==2.5.0
You can use any updated tensorflow library version. Read more about tensorflow-cpu here
120
The first thing I would check, as suggested by others, is to find out why your repo is so big given that the model size is only 83MB.
Given that you cannot reduce the size there is the option of offloading parts of the repo, but to do this you will still need an idea of which files are taking up the space. Offloading is suggested in the heroku docs. Slug size is limited to 500MB as stated here: https://devcenter.heroku.com/articles/slug-compiler#slug-size and I believe this has to do with the time it takes to spin up a new instance if a change in resources is needed. However, you can use offloading if you have particularly large files. More info on offloading here: https://devcenter.heroku.com/articles/s3
44
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
