Keras: fix "IndexError: list index out of range" error when using model.fit

Question

I'm trying to build a Variational Autoencoder with several Conv2d layers that works with cifar-10. It seems all right, but when I run the training I get this error:

Train on 50000 samples, validate on 10000 samples
  100/50000 [..............................] - ETA: 2:19

---------------------------------------------------------------------------

IndexError                                Traceback (most recent call last)

<ipython-input-8-a9198aa155a7> in <module>()
      3         epochs=1,
      4         batch_size=batch_size,
----> 5         validation_data=(x_test, None))

20 frames

/tensorflow-2.0.0-rc2/python3.6/tensorflow_core/python/keras/engine/training_eager.py in _model_loss(model, inputs, targets, output_loss_metrics, sample_weights, training)
    164 
    165         if hasattr(loss_fn, 'reduction'):
--> 166           per_sample_losses = loss_fn.call(targets[i], outs[i])
    167           weighted_losses = losses_utils.compute_weighted_loss(
    168               per_sample_losses,

IndexError: list index out of range

I've tried to reset the kernel and I've also tried with both tensorflow 2.0 and 1.14.0 but nothing changes. I'm a newbie to keras and tf, so I have probably made some mistakes.

Here's the architecture of my VAE:

(x_train, _), (x_test, y_test) = cifar10.load_data()
x_train = x_train.astype('float32') / 255.
x_train = x_train.reshape((x_train.shape[0],) + original_img_size)
x_test = x_test.astype('float32') / 255.
x_test = x_test.reshape((x_test.shape[0],) + original_img_size)

latent_dim = 128
kernel_size = (4,4)
original_img_size = (32,32,3)

#Encoder
x_in = Input(shape=original_img_size)
x = x_in
x = Conv2D(128, kernel_size=kernel_size, strides=2, padding='SAME', input_shape=original_img_size)(x)
x = BatchNormalization()(x)
x = layers.ReLU()(x)
x = Conv2D(256, kernel_size=kernel_size, strides=2, padding='SAME')(x)
x = BatchNormalization()(x)
x = layers.ReLU()(x)
x = Conv2D(512, kernel_size=kernel_size, strides=2, padding='SAME')(x)
x = BatchNormalization()(x)
x = layers.ReLU()(x)
x = Conv2D(1024, kernel_size=kernel_size, strides=2, padding='SAME')(x)
x = BatchNormalization()(x)
x = layers.ReLU()(x)

flat = Flatten()(x)
hidden = Dense(128, activation='relu')(flat)

#mean and variance
z_mean = hidden
z_log_var = hidden

#Decoder
decoder_input = Input(shape=(latent_dim,))

decoder_fc3 = Dense(8*8*1024) (decoder_input)
decoder_fc3 = BatchNormalization()(decoder_fc3)
decoder_fc3 = Activation('relu')(decoder_fc3)

decoder_reshaped = layers.Reshape((8,8,1024))(decoder_fc3)

decoder_ConvT1 = layers.Conv2DTranspose(512, kernel_size=(4,4), strides=(2,2), padding='SAME', input_shape=(8,8,1024))(decoder_reshaped)
decoder_ConvT1 = BatchNormalization()(decoder_ConvT1)
decoder_ConvT1 = Activation('relu')(decoder_ConvT1)

decoder_ConvT2 = layers.Conv2DTranspose(256, kernel_size=(4,4), strides=(2,2), padding='SAME')(decoder_ConvT1)
decoder_ConvT2 = BatchNormalization()(decoder_ConvT2)
decoder_ConvT2 = Activation('relu')(decoder_ConvT2)

decoder_ConvT3 = layers.Conv2DTranspose(3,kernel_size=(4,4), strides=(1,1), padding='SAME')(decoder_ConvT2)

y = decoder_ConvT3

decoder = Model(decoder_input, y)

x_out = decoder(encoder(x_in))

vae = Model(x_in, x_out)
vae.compile(optimizer='adam', loss=vae_loss) #custom loss 
vae.fit(x_train,
        shuffle=True,
        epochs=1,
        batch_size=batch_size,
        validation_data=(x_test, None))

Here's my custom loss function:

def vae_loss(x, x_decoded_mean):
  xent_loss = losses.binary_crossentropy(x, x_decoded_mean)
  kl_loss = - 0.5 * K.mean(1 + z_log_var - K.square(z_mean) - K.exp(z_log_var), axis=-1)
  return xent_loss + kl_loss

As suggested by qmeeus I tried to add a target output, but now I get this error:

Train on 50000 samples, validate on 10000 samples
  100/50000 [..............................] - ETA: 12:33

---------------------------------------------------------------------------

TypeError                                 Traceback (most recent call last)

/tensorflow-2.0.0-rc2/python3.6/tensorflow_core/python/eager/execute.py in quick_execute(op_name, num_outputs, inputs, attrs, ctx, name)
     60                                                op_name, inputs, attrs,
---> 61                                                num_outputs)
     62   except core._NotOkStatusException as e:

TypeError: An op outside of the function building code is being passed
a "Graph" tensor. It is possible to have Graph tensors
leak out of the function building context by including a
tf.init_scope in your function building code.
For example, the following function will fail:
  @tf.function
  def has_init_scope():
    my_constant = tf.constant(1.)
    with tf.init_scope():
      added = my_constant * 2
The graph tensor has name: dense/Identity:0


During handling of the above exception, another exception occurred:

_SymbolicException                        Traceback (most recent call last)

11 frames

/tensorflow-2.0.0-rc2/python3.6/tensorflow_core/python/eager/execute.py in quick_execute(op_name, num_outputs, inputs, attrs, ctx, name)
     73       raise core._SymbolicException(
     74           "Inputs to eager execution function cannot be Keras symbolic "
---> 75           "tensors, but found {}".format(keras_symbolic_tensors))
     76     raise e
     77   # pylint: enable=protected-access

_SymbolicException: Inputs to eager execution function cannot be Keras symbolic tensors, but found [<tf.Tensor 'dense/Identity:0' shape=(None, 128) dtype=float32>]

If you need more details let me know.

Answer 1

I had a similar error but with a normal supervised model (not AE). It might not be the problem in your case, but might be relevant to others with the same error: Make sure that your validation_data is a tuple.

Answer 2

Can you try with this instead:

vae.fit(x_train, x_train
        shuffle=True,
        epochs=1,
        batch_size=batch_size,
        validation_data=(x_test,x_test))

Keras is expecting a target output (for example y_train in supervised learning, x_train in autoencoders) that you did not provide. From the doc:

You can either pass the name of an existing loss function, or pass a TensorFlow/Theano symbolic function that returns a scalar for each data-point and takes the following two arguments:

y_true: True labels. TensorFlow/Theano tensor.
y_pred: Predictions. TensorFlow/Theano tensor of the same shape as y_true.

The actual optimized objective is the mean of the output array across all datapoints.

The way I usually do is simply to provide the same target as the input to the fit method, as shown in the code above...

[EDIT]: The error comes from your definition of kld that uses methods from tf.keras.backend. I'm no expert in tensorflow 2 but this is definitely the reason of the error. Refer to this tutorial to know how to build your loss.

Another workaround is to build a model with two outputs and create two loss functions, one for each output, eg

model = Model(x_in, [hidden, y])
model.compile(loss=[custom_kld, binary_crossentropy], optimizer=optimizer)