Dimension Error while using LSTM after Convolutional neural network in Keras

Question

In keras 1.2.2, I have made a dataset which has these dimensions:

X_train: (2000, 100, 32, 32, 3) 
y_train: (2000,1)

Here, 2000 is the number of instances (batches of data), 100 is the number of samples in each batch, 32 is the image rows and cols, and 3 is the number of channels (RGB).

I have written this code which applies an LSTM after a CNN, however, I get this error:

ValueError: Input 0 is incompatible with layer lstm_layer: expected ndim=3, found ndim=2

This is my code:

import keras
from keras.layers import Input ,Dense, Dropout, Activation, LSTM
from keras.layers import Convolution2D, MaxPooling2D, Flatten, Reshape
from keras.models import Sequential
from keras.layers.wrappers import TimeDistributed
from keras.layers.pooling import GlobalAveragePooling1D
from keras.optimizers import SGD
from keras.utils import np_utils
from keras.models import Model


import numpy as np

timesteps=100;
number_of_samples=2500;
nb_samples=number_of_samples;
frame_row=32;
frame_col=32;
channels=3;

nb_epoch=1;
batch_size=timesteps;

data= np.random.random((2500,timesteps,frame_row,frame_col,channels))
label=np.random.random((2500,timesteps,1))

X_train=data[0:2000,:]
y_train=label[0:2000]

X_test=data[2000:,:]
y_test=label[2000:,:]

#%%

model=Sequential();                          

model.add(Convolution2D(32, 3, 3, border_mode='same',
                        input_shape=X_train.shape[2:]))
model.add(Activation('relu'))
model.add(Convolution2D(32, 3, 3))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))

model.add(Convolution2D(64, 3, 3, border_mode='same'))
model.add(Activation('relu'))
model.add(Convolution2D(64, 3, 3))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))

model.add(Flatten())
model.add(Dense(512))
model.add(Activation('relu'))
model.add(Dropout(0.5))


model.add(Dense(35, input_shape=(timesteps,512), name="first_dense" ));
#model.add(Dense(1, name="test_dense"));         

model.add(LSTM(20, return_sequences=True, name="lstm_layer"));

#%%
model.add(TimeDistributed(Dense(1), name="time_distr_dense_one"))
model.add(GlobalAveragePooling1D(name="global_avg"))

#%%

model.compile(loss='binary_crossentropy',
              optimizer='adam',
              metrics=['accuracy'])
#%%
model.fit(X_train, y_train,
          batch_size=batch_size,
          nb_epoch=nb_epoch,
          validation_data=(X_test, y_test))

Answer 1

Try exchanging each Convolution2D(...) to:

TimeDistributed(Convolution2D(...))

You need to let your model know that your data is sequential and you want to apply some layer to each element in your sequence. This is what TimeDistributed wrapper is for.