Attaching class labels to a Keras model

Question

I am using the Keras Sequential model to train a number of multiclass classifiers.

On evaluation, Keras outputs a vector of confidences and I can infer the correct class id from that using argmax. I can then use a lookup table to receive the actual class label (e.g. a string).

So far the solution is to load the trained model, and then to load a lookup table separately. Since I have quite a number of classifiers I would prefer to keep both structures in one file.

So what I am looking for is a way to integrate the actual label lookup vector into the Keras model. That would allow me to have a single classifier file that is capable of taking some input data and returning the correct class label for that data.

One way to solve this would be to store both the model and the lookup table in a tuple and write that tuple into a pickle, but this doesn't seem very elegant.

Answer 1

So I tried my hand at a solution myself and this seems to work. I was hoping for something simpler though.

Opening the model file a second time is not really optimal I think. If anyone can do better, by all means, do.

import h5py

from keras.models import load_model
from keras.models import save_model


def load_model_ext(filepath, custom_objects=None):
    model = load_model(filepath, custom_objects=None)
    f = h5py.File(filepath, mode='r')
    meta_data = None
    if 'my_meta_data' in f.attrs:
        meta_data = f.attrs.get('my_meta_data')
    f.close()
    return model, meta_data
   

def save_model_ext(model, filepath, overwrite=True, meta_data=None):
    save_model(model, filepath, overwrite)
    if meta_data is not None:
        f = h5py.File(filepath, mode='a')
        f.attrs['my_meta_data'] = meta_data
        f.close()

Since h5 files do not accept python containers, you should consider converting the meta data into a string. Assuming that your meta data exists in the form of a dictionary or a list, you can use json to do the conversion. This would also allow you to store more complex data structures within your model.

Full usage example:

import json
import keras

# prepare model and label lookup
model = keras.Sequential();
model.add(keras.layers.Dense(10, input_dim=8, activation='relu'));
model.add(keras.layers.Dense(3, activation='softmax'))
model.compile()

filepath = r".\mymodel.h5"

labels = ["dog", "cat", "automobile"]

# save
labels_string = json.dumps(labels)
save_model_ext(model, filepath, meta_data=labels_string)

# load
loaded_model, loaded_labels_string = load_model_ext(filepath)
loaded_labels = json.loads(loaded_labels_string)

# label of class 0: "dog"
print(loaded_labels[0])

If you prefer to have a dictionary for your classes, be aware that json will convert numeric dictionary keys to strings, so you will have to convert them back to numbers after loading.

Answer 2

It is possible to save a "list" of labels in keras model directly. You need to use lambda. You just replace the output of lambda with a string tensor containing your labels. Here is a dummy example of how one can perform an "injection" of labels

# assume we get labels as list
labels = ["cat","dog","horse","tomato"]
# here we start building our model with input image 299x299 and one output layer
xx = Input(shape=(299,299,3))
flat = Flatten()(xx)
output = Dense(shape=(4))(flat)
# here we perform injection of labels
tf_labels = tf.constant([labels],dtype="string")
# adding ? dimension to tf tensor
tf_labels = tf.tile(labels,[tf.shape(xx)[0],1])
output_labels = Lambda(lambda x: tf_labels,name="label_injection")(xx)
#and finaly creating a model
model=tf.keras.Model(xx,[output,output_labels])

This model now stores the labels and returns them as well. All this mess with tf.tile is necessary because a keras layer of shape (N) is actually a tf tensor of shape (?,N) and we add this ? dimension to the label tensor.