PyTorch model deployment in AI Platform

Question

I'm deploying a Pytorch model in Google Cloud AI Platform, I'm getting the following error:

ERROR: (gcloud.beta.ai-platform.versions.create) Create Version failed. Bad model detected with error: Model requires more memory than allowed. Please try to decrease the model size and re-deploy. If you continue to have error, please contact Cloud ML.

Configuration:

setup.py

from setuptools import setup

REQUIRED_PACKAGES = ['torch']

setup(
    name="iris-custom-model",
    version="0.1",
    scripts=["model.py"],
    install_requires=REQUIRED_PACKAGES
)

Model version creation

MODEL_VERSION='v1'
RUNTIME_VERSION='1.15'
MODEL_CLASS='model.PyTorchIrisClassifier'

!gcloud beta ai-platform versions create {MODEL_VERSION} --model={MODEL_NAME} \
            --origin=gs://{BUCKET}/{GCS_MODEL_DIR} \
            --python-version=3.7 \
            --runtime-version={RUNTIME_VERSION} \
            --package-uris=gs://{BUCKET}/{GCS_PACKAGE_URI} \
            --prediction-class={MODEL_CLASS}

Answer 1

You need to use Pytorch compiled packages compatible with Cloud AI Platform Package information here

This bucket contains compiled packages for PyTorch that are compatible with Cloud AI Platform prediction. The files are mirrored from the official builds at https://download.pytorch.org/whl/cpu/torch_stable.html

From documentation

In order to deploy a PyTorch model on Cloud AI Platform Online Predictions, you must add one of these packages to the packageURIs field on the version you deploy. Pick the package matching your Python and PyTorch version. The package names follow this template:

Package name = torch-{TORCH_VERSION_NUMBER}-{PYTHON_VERSION}-linux_x86_64.whl where PYTHON_VERSION = cp35-cp35m for Python 3 with runtime versions < 1.15, cp37-cp37m for Python 3 with runtime versions >= 1.15

For example, if I were to deploy a PyTorch model based on PyTorch 1.1.0 and Python 3, my gcloud command would look like:

gcloud beta ai-platform versions create {VERSION_NAME} --model {MODEL_NAME} 
 ...
--package-uris=gs://{MY_PACKAGE_BUCKET}/my_package-0.1.tar.gz,gs://cloud->ai-pytorch/torch-1.1.0-cp35-cp35m-linux_x86_64.whl

In summary:

1) Remove torch from your install_requires dependencies in setup.py

2) Include torch package when creating your version model.

!gcloud beta ai-platform versions create {VERSION_NAME} --model {MODEL_NAME} \
 --origin=gs://{BUCKET}/{MODEL_DIR}/ \
 --python-version=3.7 \
 --runtime-version={RUNTIME_VERSION} \
 --package-uris=gs://{BUCKET}/{PACKAGES_DIR}/text_classification-0.1.tar.gz,gs://cloud-ai-pytorch/torch-1.3.1+cpu-cp37-cp37m-linux_x86_64.whl \
 --prediction-class=model_prediction.CustomModelPrediction