Extract encoder and decoder from trained autoencoder

Question

I want to divide the autoencoder learning and applying into two parts following https://blog.keras.io/building-autoencoders-in-keras.html and using the fashion-mnist data for testing purposes:

1. Load the images, do the fitting that may take some hours or days and use a callback to save the best autoencoder model. That process can be some weeks before the following part.
2. Use this best model (manually selected by filename) and plot original image, the encoded representation made by the encoder of the autoencoder and the prediction using the decoder of the autoencoder. I have problems (see second step) to extract the encoder and decoder layers from the trained and saved autoencoder.

For step one I have the very simple network as follows:

input_img = Input(shape=(784,))
# encoded representation
encoded = Dense(encoding_dim, activation='relu')(input_img)
# lossy reconstruction
decoded = Dense(784, activation='sigmoid')(encoded)

# full AE model: map an input to its reconstruction
autoencoder = Model(input_img, decoded)

# encoder: map an input to its encoded representation
encoder = Model(input_img, encoded)
# placeholder for an encoded input
encoded_input = Input(shape=(encoding_dim,))
# last layer of the autoencoder model
decoder_layer = autoencoder.layers[-1]
# decoder
decoder = Model(encoded_input, decoder_layer(encoded_input))

The networks are:

autoencoder.summary()
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
input_5 (InputLayer)         (None, 784)               0         
_________________________________________________________________
dense_5 (Dense)              (None, 32)                25120     
_________________________________________________________________
dense_6 (Dense)              (None, 784)               25872     
=================================================================

and

encoder.summary()
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
input_5 (InputLayer)         (None, 784)               0         
_________________________________________________________________
dense_5 (Dense)              (None, 32)                25120     
=================================================================

So I train the model and save it by autoencoder.save('fashion-autoencoder.hdf5'). In my real example I save it with a callback so a workaround by saving the encoder and decoder does not seem a real solution. Later, I load the images (not shown) and do the predictions like

# encode and decode some images from test set
encoded_imgs = encoder.predict(x_test)
decoded_imgs = decoder.predict(encoded_imgs)
# test the shape
print(encoded_imgs[0].shape)

and get a shape of (32,0).

So lets go to step 2 where I have my problems. I load the model using

encoder= K.models.load_model('fashion-autoencoder.hdf5')
# delete the last layers to get the encoder
encoder.layers.pop()
encoder.summary() # show model data

and the encoder looks the same as the original in step one what makes me think the extraction has worked well:

Layer (type)                 Output Shape              Param #   
=================================================================
input_5 (InputLayer)         (None, 784)               0         
_________________________________________________________________
dense_5 (Dense)              (None, 32)                25120     
=================================================================
Total params: 50,992
Trainable params: 50,992
Non-trainable params: 0

But I also get the warning

training.py:478: UserWarning: Discrepancy between trainable weights and collected trainable weights, did you set `model.trainable` without calling `model.compile` after ?
'Discrepancy between trainable weights and collected trainable'

that I understand in a kind of way but do not know how important it is. Then I load images again (not shown) and use the encoder

encoded_imgs = encoder.predict(x_test)

# test the shape
print(encoded_imgs[0].shape)

but shape is not right with (784,).

So, my extraction for the encoder did not work since the dimensions are not correct. I even have less success extracting the decoder (from the saved autoencoder) since I cannot use push() and tried stuff like decoder = decoder.layers[-1:-2] but it did not work.

So, my general question is how to extract parts of loaded models.

Answer 1

Since you are using functional API for creating the autoencoder, the best way to reconstruct the encoder and decoder is to use the functional API and the Model class again:

autoencoder= K.models.load_model('fashion-autoencoder.hdf5')

encoder = Model(autoencoder.input, autoencoder.layers[-2].output)

decoder_input = Input(shape=(encoding_dim,))
decoder = Model(decoder_input, autoencoder.layers[-1](decoder_input))

encoder.summary()
decoder.summary()

The models summary:

Layer (type)                 Output Shape              Param #   
=================================================================
input_4 (InputLayer)         (None, 784)               0         
_________________________________________________________________
dense_3 (Dense)              (None, 32)                25120     
=================================================================
Total params: 25,120
Trainable params: 25,120
Non-trainable params: 0
_________________________________________________________________


Layer (type)                 Output Shape              Param #   
=================================================================
input_6 (InputLayer)         (None, 32)                0         
_________________________________________________________________
dense_4 (Dense)              (None, 784)               25872     
=================================================================
Total params: 25,872
Trainable params: 25,872
Non-trainable params: 0
_________________________________________________________________

The solution involving pop() on layers attribute does not work since you need to update some of the internal attributes of the model. Although, for sequential models a built-in pop() method has been implemented.