concatenate (merge) layer keras with tensorflow

Question

I'd like to make a model as following.

input data    input data
    |             | 
 convnet1      convet2
    |             |
maxpooling    maxpooling
    |             |
    - Dense layer -
           |
      Dense layer

So, I've wrote following code.

model1 = Sequential()
model1.add(Conv2D(32, (3, 3), activation='relu', input_shape=(bands, frames, 1)))
print(model1.output_shape)
model1.add(MaxPooling2D(pool_size=(2, 2)))
model1.add(Flatten())

model2 = Sequential()
model2.add(Conv2D(32, (9, 9), activation='relu', input_shape=(bands, frames, 1)))
print(model2.output_shape)
model2.add(MaxPooling2D(pool_size=(4, 4)))
model2.add(Flatten())

modelall = Sequential()
modelall.add(concatenate([model1, model2], axis=1))
modelall.add(Dense(100, activation='relu'))

modelall.add(Dropout(0.5))
modelall.add(Dense(10, activation='softmax')) #number of class = 10
print(modelall.output_shape)

modelall.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])

modelall.fit([X_train, X_train], y_train, batch_size=batch_size, nb_epoch=training_epochs)
score = modelall.evaluate(X_test, y_test, batch_size=batch_size)

However, I got an error.

AttributeError: 'Sequential' object has no attribute 'get_shape'

The whole error traceback as follows.

  Traceback (most recent call last):
  File "D:/keras/cnn-keras.py", line 54, in <module>
    model.add(concatenate([modelf, modelt], axis=1))
  File "C:\Users\Anaconda3\lib\site-packages\keras\layers\merge.py", line 508, in concatenate
    return Concatenate(axis=axis, **kwargs)(inputs)
  File "C:\Users\Anaconda3\lib\site-packages\keras\engine\topology.py", line 542, in __call__
    input_shapes.append(K.int_shape(x_elem))
  File "C:\Users\Anaconda3\lib\site-packages\keras\backend\tensorflow_backend.py", line 411, in int_shape
    shape = x.get_shape()
AttributeError: 'Sequential' object has no attribute 'get_shape'

Is the error caused by tensorflow? Any idea on how to fix it?

Answer 1

You cannot use a Sequential model for creating branches, that doesn't work.

You must use the functional API for that:

from keras.models import Model    
from keras.layers import *

It's ok to have each branch as a sequential model, but the fork must be in a Model.

#in the functional API you create layers and call them passing tensors to get their output:

conc = Concatenate()([model1.output, model2.output])

    #notice you concatenate outputs, which are tensors.     
    #you cannot concatenate models


out = Dense(100, activation='relu')(conc)
out = Dropout(0.5)(out)
out = Dense(10, activation='softmax')(out)

modelall = Model([model1.input, model2.input], out)

It wasn't necessary here, but usually you create Input layers in the functional API:

inp = Input((shape of the input))
out = SomeLayer(blbalbalba)(inp)
....
model = Model(inp,out)