How to load a model saved in joblib file from Google Cloud Storage bucket

Question

I want to load a model which is saved as a joblib file from Google Cloud Storage bucket. When it is in local path, we can load it as follows (considering model_file is the full path in system):

loaded_model = joblib.load(model_file)

How can we do the same task with Google Cloud Storage?

Answer 1

For anyone googling around for an answer to this. Here are two more options besides the obvious, to use Google AI platform for model hosting (and online predictions).

Option 1 is to use TemporaryFile like this:

from google.cloud import storage
from sklearn.externals import joblib
from tempfile import TemporaryFile

storage_client = storage.Client()
bucket_name=<bucket name>
model_bucket='model.joblib'

bucket = storage_client.get_bucket(bucket_name)
#select bucket file
blob = bucket.blob(model_bucket)
with TemporaryFile() as temp_file:
    #download blob into temp file
    blob.download_to_file(temp_file)
    temp_file.seek(0)
    #load into joblib
    model=joblib.load(temp_file)
#use the model
model.predict(...)

Option 2 is to use BytesIO like this:

from google.cloud import storage
from sklearn.externals import joblib
from io import BytesIO

storage_client = storage.Client()
bucket_name=<bucket name>
model_bucket='model.joblib'

bucket = storage_client.get_bucket(bucket_name)
#select bucket file
blob = bucket.blob(model_bucket)
#download blob into an in-memory file object
model_file = BytesIO()
blob.download_to_file(model_file)
#load into joblib
model=joblib.load(model_local)