ValueError: A `Concatenate` layer should be called on a list of at least 2 inputs

Question

I'm trying to use a sigmoid to join the output of two models with different embedding matrix. but I keep getting the error at the concatenate line. I have tried other suggestions from similar questions but it keeps giving the same error. I feel I'm missing something but I can't find it. please help explain. Thanks

############################            MODEL   1      ######################################
input_tensor=Input(shape=(35,))
input_layer= Embedding(vocab_size, 300, input_length=35, weights=[embedding_matrix],trainable=True)(input_tensor)
conv_blocks = []
filter_sizes = (2,3,4)
for fx in filter_sizes:
    conv_layer= Conv1D(100, kernel_size=fx, activation='relu', data_format='channels_first')(input_layer)   #filters=100, kernel_size=3
    maxpool_layer = MaxPooling1D(pool_size=4)(conv_layer)
    flat_layer= Flatten()(maxpool_layer)
    conv_blocks.append(flat_layer)
conc_layer=concatenate(conv_blocks, axis=1)
graph = Model(inputs=input_tensor, outputs=conc_layer)
model = Sequential()
model.add(graph)
model.add(Dropout(0.2))

############################            MODEL    2     ######################################
input_tensor_1=Input(shape=(35,))
input_layer_1= Embedding(vocab_size, 300, input_length=35, weights=[embedding_matrix_1],trainable=True)(input_tensor_1)
conv_blocks_1 = []
filter_sizes_1 = (2,3,4)
for fx in filter_sizes_1:
    conv_layer_1= Conv1D(100, kernel_size=fx, activation='relu', data_format='channels_first')(input_layer_1)   #filters=100, kernel_size=3
    maxpool_layer_1 = MaxPooling1D(pool_size=4)(conv_layer_1)
    flat_layer_1= Flatten()(maxpool_layer_1)
    conv_blocks_1.append(flat_layer_1)
conc_layer_1=concatenate(conv_blocks_1, axis=1)
graph_1 = Model(inputs=input_tensor_1, outputs=conc_layer_1)
model_1 = Sequential()
model_1.add(graph_1)
model_1.add(Dropout(0.2))


fused = concatenate([graph, graph_1], axis=-1)
prediction = Dense(3, activation='sigmoid')(fused)
model = Model(inputs=[input_tensor,input_tensor_1], outputs=[prediction])
model.compile(loss='sparse_categorical_crossentropy',optimizer='Adagrad', metrics=['accuracy'])
model.summary()

This is the error trace

Traceback (most recent call last):
  File "DL_Ensemble.py", line 145, in <module>
    fused = concatenate([graph, graph_1], axis= 1 )
  File "/usr/pkg/lib/python3.8/site- 
   packages/tensorflow_core/python/keras/layers/merge.py", line 705, in concatenate
    return Concatenate(axis=axis, **kwargs)(inputs)
  File "/usr/pkg/lib/python3.8/site-packages/tensorflow_core/python/keras/engine/base_layer.py", line 887, in __call__
    self._maybe_build(inputs)
  File "/usr/pkg/lib/python3.8/site-packages/tensorflow_core/python/keras/engine/base_layer.py", line 2141, in _maybe_build
    self.build(input_shapes)
   File "/usr/pkg/lib/python3.8/site- 
   packages/tensorflow_core/python/keras/utils/tf_utils.py", line 306, in wrapper
output_shape = fn(instance, input_shape)
  File "/usr/pkg/lib/python3.8/site- 
   packages/tensorflow_core/python/keras/layers/merge.py", line 378, in build
    raise ValueError('A `Concatenate` layer should be called '
ValueError: A `Concatenate` layer should be called on a list of at least 2 inputs

UPDATE: I have reflected the answers given by @VivekMehta, however, I have this error.

File "DL_Ensemble.py", line 165, in <module> model.fit([train_sequences,train_sequences], train_y, epochs=10, verbose=False, batch_size=32, class_weight={0: 6.0, 1: 1.0, 2: 2.0}) File "/usr/pkg/lib/python3.8/site- packages/tensorflow_core/python/keras/engine/training.py", line 709, in fit return func.fit( File "/usr/pkg/lib/python3.8/site- packages/tensorflow_core/python/keras/engine/training_v2.py", line 313, in fit training_result = run_one_epoch( File "/usr/pkg/lib/python3.8/site- packages/tensorflow_core/python/keras/engine/training_v2.py", line 123, in run_one_epoch batch_outs = execution_function(iterator) File "/usr/pkg/lib/python3.8/site- packages/tensorflow_core/python/keras/engine/training_v2_utils.py", line 86, in execution_function distributed_function(input_fn)) File "/usr/pkg/lib/python3.8/site- packages/tensorflow_core/python/eager/def_function.py", line 457, in __call__ result = self._call(*args, **kwds) File "/usr/pkg/lib/python3.8/site- packages/tensorflow_core/python/eager/def_function.py", line 520, in _call return self._stateless_fn(*args, **kwds) File "/usr/pkg/lib/python3.8/site- packages/tensorflow_core/python/eager/function.py", line 1823, in __call__ return graph_function._filtered_call(args, kwargs) # pylint: disable=protected-access File "/usr/pkg/lib/python3.8/site- packages/tensorflow_core/python/eager/function.py", line 1137, in _filtered_call return self._call_flat( File "/usr/pkg/lib/python3.8/site- packages/tensorflow_core/python/eager/function.py", line 1223, in _call_flat flat_outputs = forward_function.call( File "/usr/pkg/lib/python3.8/site- packages/tensorflow_core/python/eager/function.py", line 506, in call outputs = execute.execute( File "/usr/pkg/lib/python3.8/site- packages/tensorflow_core/python/eager/execute.py", line 67, in quick_execute six.raise_from(core._status_to_exception(e.code, message), None) File "<string>", line 3, in raise_from tensorflow.python.framework.errors_impl.InvalidArgumentError:
Conv2DCustomBackpropInputOp only supports NHWC. [[node Conv2DBackpropInput (defined at /usr/pkg/lib/python3.8/site- packages/tensorflow_core/python/framework/ops.py:1751) ]] [Op:__inference_distributed_function_2250]

Function call stack:
distributed_function

I also wanted to add that when the code is run on a GPU as opposed to a CPU, the error occurs on the same line as before but the message changes to :

File "DL_Ensemble.py", line 166, in <module>
model.fit([train_sequences,train_sequences], train_y, epochs=10, verbose=False, batch_size=32, class_weight={0: 6.0, 1: 1.0, 2: 2.0})
  File "/home/kosimadukwe/.local/lib/python3.7/site-packages/tensorflow/python/keras/engine/training.py", line 880, in fit
validation_steps=validation_steps)
  File "/home/kosimadukwe/.local/lib/python3.7/site-packages/tensorflow/python/keras/engine/training_arrays.py", line 329, in model_iteration
batch_outs = f(ins_batch)
  File "/home/kosimadukwe/.local/lib/python3.7/site-packages/tensorflow/python/keras/backend.py", line 3073, in __call__
self._make_callable(feed_arrays, feed_symbols, symbol_vals, session)
  File "/home/kosimadukwe/.local/lib/python3.7/site-packages/tensorflow/python/keras/backend.py", line 3019, in _make_callable
callable_fn = session._make_callable_from_options(callable_opts)
  File "/home/kosimadukwe/.local/lib/python3.7/site-packages/tensorflow/python/client/session.py", line 1471, in _make_callable_from_options
return BaseSession._Callable(self, callable_options)
  File "/home/kosimadukwe/.local/lib/python3.7/site-packages/tensorflow/python/client/session.py", line 1425, in __init__
session._session, options_ptr, status)
  File "/home/kosimadukwe/.local/lib/python3.7/site-packages/tensorflow/python/framework/errors_impl.py", line 528, in __exit__
c_api.TF_GetCode(self.status.status))
tensorflow.python.framework.errors_impl.InvalidArgumentError: Conv2DCustomBackpropInputOp only supports NHWC.
     [[{{node training/Adagrad/gradients/conv1d_5/conv1d/Conv2D_grad/Conv2DBackpropInput}}]]
Exception ignored in: <function BaseSession._Callable.__del__ at 0x7fe4dd06a730>
Traceback (most recent call last):
  File "/home/kosimadukwe/.local/lib/python3.7/site-packages/tensorflow/python/client/session.py", line 1455, in __del__
self._session._session, self._handle, status)
  File "/home/kosimadukwe/.local/lib/python3.7/site-packages/tensorflow/python/framework/errors_impl.py", line 528, in __exit__
c_api.TF_GetCode(self.status.status))
tensorflow.python.framework.errors_impl.InvalidArgumentError: No such callable handle: 94697914208640

Answer 1

So from you stack trace, code is throwing error at:

fused = concatenate([graph, graph_1], axis= 1 )
print(type(graph))
# output: <class 'tensorflow.python.keras.engine.training.Model'>

This error is coming because concatenate expects list of tensors to be concatenated. While you are passing graph and graph_1 which is not tensor but a Model instance.

So from your code I assume that you want to concatenate output of these two models. In that case you'll have to change above line to:

fused = concatenate([graph.outputs[0], graph_1.outputs[0]], axis=-1)

Here, graph.outputs gives list of outputs by given by Model. Since each model is giving us one output, we will take 0th index from each output.

Change this part and you'll get model summary as you are expecting.