Keras - How to run loaded model on CPU

Question

I have keras with tensorflow backend that runs on GPU. However, I am training an LSTM so instead I am training on the CPU.

with tf.device('/cpu:0'):
    model = Sequential()
    model.add(Bidirectional(LSTM(50, return_sequences=True), input_shape=(50, len(train_x[0][0]))))
    model.add(TimeDistributed(Dense(1, activation='sigmoid')))
    model.compile(loss='binary_crossentropy', optimizer='Adam', metrics=['acc'])

The problem I have is that when I save and load the model, the predict function for the loaded model performs very slowly. After some timed tests I believe what is happening is that the loaded model is running on the GPU rather than the CPU, so it is slow. I tried compiling the loaded model on the CPU however this does not speed things up:

model.save('test_model.h5')
new_model = load_model('test_model.h5')
with tf.device('/cpu:0'):
    new_model.compile(loss='binary_crossentropy', optimizer='Adam', metrics=['acc'])

Is there a way to achieve the same speeds with the loaded model as with the newly trained model? The newly trained model is almost five times faster. Thanks for your help.

Answer 1

Load the model with the device you want to use:

with tf.device('/cpu:0'):
    new_model = load_model('test_model.h5')