How to decode encoded data from deep autoencoder in Keras (unclarity in tutorial)

Question

I have followed the tutorial "Building Autoencoders in Keras": https://blog.keras.io/building-autoencoders-in-keras.html

The first, simple, solution works fine. But in the section "Deep autoencoder" the code provided in the tutorial does not seem to work fully.

Here is my code (just until where the problem appears), which is just copied from the turorial:

from keras.layers import Input, Dense
from keras.models import Model

encoding_dim = 32

input_img = Input(shape=(784,))
encoded = Dense(128, activation='relu')(input_img)
encoded = Dense(64, activation='relu')(encoded)
encoded = Dense(32, activation='relu')(encoded)  # Multiple encoding
decoded = Dense(64, activation='relu')(encoded)  # and decoding layers.
decoded = Dense(128, activation='relu')(decoded)
decoded = Dense(784, activation='sigmoid')(decoded)

autoencoder = Model(input_img, decoded)

encoder = Model(input_img, encoded)

encoded_input = Input(shape=(encoding_dim,))
decoder_layer = autoencoder.layers[-1]
decoder = Model(encoded_input, decoder_layer(encoded_input))  # Crash happens here.

I get this error:

Traceback (most recent call last):
  File "keras_test.py", line 20, in <module>
    decoder = Model(encoded_input, decoder_layer(encoded_input))  # Crash happens here
  File "/Users/paulmagnus/Library/Enthought/Canopy/edm/envs/User/lib/python2.7/site-packages/keras/engine/topology.py", line 569, in __call__
    self.assert_input_compatibility(inputs)
  File "/Users/paulmagnus/Library/Enthought/Canopy/edm/envs/User/lib/python2.7/site-packages/keras/engine/topology.py", line 479, in assert_input_compatibility
    ' but got shape ' + str(x_shape))
ValueError: Input 0 is incompatible with layer dense_6: expected axis -1 of input shape to have value 128 but got shape (None, 32)

I am guessing that decoderis connected to the wrong decoding layer and/or that the shape of its input or output is wrong. But what should I do about it?

The decoder is not necessary for the autoencoder to work. I can perform the learning and encode the images following the rest of the tutorial. But without decoder I am not able to decode the images back to original format to see if they actually look good. The tutorial doesn't mention anything about this and just shows the decoded images without a word. I guess the author assumed whatever change he did to decoder to achieve this was trivial.

To clarify: the single-layer version works fine, where instead of 3 encoding and 3 decoding layers we just have

encoded = Dense(32, activation='relu')(input_img)
decoded = Dense(784, activation='sigmoid')(encoded)

and everything else is equal as above. Then there is no error and I can use decoder to recreate the images.

Answer 1

decoder_layer = autoencoder.layers[-1]
decoder = Model(encoded_input, decoder_layer(encoded_input))

This code works for single-layer because only last layer is decoder in this case and

decoder_layer = autoencoder.layers[-1]

this line calls for last layer.

for 3-layer encoders and decoders, you have to call all 3 layers for defining decoder. i was doing the same tutorial so i have written the code like this.

encoded_input = Input(shape=(encoding_dim,))

deco = autoencoder.layers[-3](encoded_input)
deco = autoencoder.layers[-2](deco)
deco = autoencoder.layers[-1](deco)
# create the decoder model
decoder = Model(encoded_input, deco).

it works fine now.