Recurrentshop and Keras: multi-dimensional RNN results in a dimensions mismatch error

Question

I have an issue with Recurrentshop and Keras. I am trying to use Concatenate and multidimensional tensors in a Recurrent Model, and I get dimension issue regardless of how I arrange the Input, shape and batch_shape.

Minimal code:

from keras.layers import *
from keras.models import *
from recurrentshop import *
from keras.layers import Concatenate

input_shape=(128,128,3)

x_t = Input(shape=(128,128,3,))
h_tm1 = Input(shape=(128,128,3, ))

h_t1 = Concatenate()([x_t, h_tm1])
last = Conv2D(3, kernel_size=(3,3), strides=(1,1), padding='same',     name='conv2')(h_t1)

# Build the RNN
rnn = RecurrentModel(input=x_t, initial_states=[h_tm1], output=last,     final_states=[last], state_initializer=['zeros'])

x = Input(shape=(128,128,3, ))
y = rnn(x)

model = Model(x, y)

model.predict(np.random.random((1, 128, 128, 3)))

ErrorCode:

ValueError: Shape must be rank 3 but it is rank 4 for 'recurrent_model_1/concatenate_1/concat' (op:ConcatV2) with input shapes: [?,128,3], [?,128,128,3], [].

Please help.

Answer 1

Try this (the changed lines are commented):

from recurrentshop import *
from keras.layers import Concatenate

x_t = Input(shape=(128, 128, 3,))
h_tm1 = Input(shape=(128, 128, 3,))

h_t1 = Concatenate()([x_t, h_tm1])
last = Conv2D(3, kernel_size=(3, 3), strides=(1, 1), padding='same', name='conv2')(h_t1)

rnn = RecurrentModel(input=x_t,
                     initial_states=[h_tm1],
                     output=last,
                     final_states=[last],
                     state_initializer=['zeros'])

x = Input(shape=(1, 128, 128, 3,))  # a series of 3D tensors -> 4D
y = rnn(x)

model = Model(x, y)
model.predict(np.random.random((1, 1, 128, 128, 3)))  # a batch of x -> 5D