How to stack Convolutional Layer and LSTM using Tensorflow2.0 alpha?

Question

I am trying to implement a neural network for an NLP task with a convolutional layer followed up by an LSTM layer. I am currently experimenting with the new Tensorflow 2.0 to do this. However, when building the model, I've encountered an error that I could not understand.

# Input shape of training and validation set
(1000, 1, 512), (500, 1, 512)

The model

model = keras.Sequential()
model.add(keras.layers.InputLayer(input_shape=(None, 512)))
model.add(keras.layers.Conv1D(128, 1, activation="relu"))
model.add(keras.layers.MaxPooling1D((2)))
model.add(keras.layers.LSTM(64, activation="tanh"))
model.add(keras.layers.Dense(6))
model.add(keras.layers.Activation("softmax"))

The error

InvalidArgumentError: Tried to stack elements of an empty list with non-fully-defined element_shape: [?,64]
 [[{{node unified_lstm_16/TensorArrayV2Stack/TensorListStack}}]] [Op:__inference_keras_scratch_graph_26641]

At first, I tried to check if there are any issues regarding implementing a Conv1D layer with an LSTM layer. I found this post, that suggested so that I reshaped the layer between the convolutional layer and lstm layer. But that still did not work and I got a different error instead. This post seems similar but it does not use Tensorflow 2.0 and not answer so far. I also found this post that has the same intention of stacking a convolutional and lstm layers. But it uses Conv2D instead of Conv1D. This post also suggests to use reshaped the output of the convolutional layer with a built-in layer called Reshape. Yet, I still got the same error.

I also tried to specify the input_shape in the LSTM layer.

model = keras.Sequential()
model.add(keras.layers.InputLayer(input_shape=(None, 512)))
model.add(keras.layers.Conv1D(128, 1, activation="relu"))
model.add(keras.layers.MaxPooling1D((2)))
model.add(keras.layers.LSTM(64, activation="tanh", input_shape=(None, 64)))
model.add(keras.layers.Dense(6))
model.add(keras.layers.Activation("softmax"))

And I still got the same error in the end.

I am not sure if I understand how to stack the 1-dimensional convolutional layer and lstm layer. I know that TF2.0 is still an Alpha, but did can someone point out what I was missing? Thanks in advance

Answer 1

The issue is a dimensionality issue. Your feature is of shape [..., 1, 512]; therefore, MaxPooling1D pooling_size 2 is bigger than 1 causing the issue.

Adding padding="same" will solve the issue.

model = tf.keras.Sequential()
model.add(tf.keras.layers.InputLayer(input_shape=(None, 512)))
model.add(tf.keras.layers.Conv1D(128, 1, activation="relu"))
model.add(tf.keras.layers.MaxPooling1D(2, padding="same"))

model.add(tf.keras.layers.LSTM(64, activation="tanh"))
model.add(tf.keras.layers.Flatten())
model.add(tf.keras.layers.Dense(6))
model.add(tf.keras.layers.Activation("softmax"))