Input multiple datasets to tensorflow model

Question

Hi I'm trying to input multiple datasets in a model. This is an example of my problem, however in my case one of my models has 2 input parameters while the other one has one. The error I get in my case is :

Failed to find data adapter that can handle input: (<class 'list'> containing values of types {"<class 'tensorflow.python.data.ops.dataset_ops.BatchDataset'>", "<class 'tensorflow.python.data.ops.dataset_ops.TakeDataset'>"}), <class 'NoneType'>

Code:

import tensorflow as tf

# Create first model
model1 = tf.keras.Sequential()
model1.add(tf.keras.layers.Dense(1))
model1.compile()
model1.build([None,3])

# Create second model
model2 = tf.keras.Sequential()
model2.add(tf.keras.layers.Dense(1))
model2.compile()
model2.build([None,3])


# Concatenate
fusion_model = tf.keras.layers.Concatenate()([model1.output, model2.output])
t = tf.keras.layers.Dense(1, activation='tanh')(fusion_model)
model = tf.keras.models.Model(inputs=[model1.input, model2.input], outputs=t)
model.compile()

#Datasets
ds1 = tf.data.Dataset.from_tensors(([1,2,3],1))
ds2 = tf.data.Dataset.from_tensors(([1,2,3], 2))

print(ds1)
print(ds2)
# Fit
model.fit([ds1,ds2])

Running this example code produces that:

Failed to find data adapter that can handle input: (<class 'list'> containing values of types {"<class 'tensorflow.python.data.ops.dataset_ops.TensorDataset'>"}), <class 'NoneType'>

I need to use the dataset modules because they provide in built lazy loading of the data.

Answer 1

As noted in the comment, the TensorFlow .fit function in TensorFlow models does not support a list of Datasets.

If you really want to use Datasets, you could use a dictionary as the input, and have named input layers to match to the dict.

Here's how you do it:

model1 = tf.keras.Sequential(name="layer_1")
model2 = tf.keras.Sequential(name="layer_2")

model.summary()

ds1 = tf.data.Dataset.from_tensors(({"layer_1": [[1,2,3]],
                                     "layer_2": [[1,2,3]]}, [[2]]))

model.fit(ds1)

An easier option is to simply use tensors instead of datasets. .fit supports a list of tensors as input so just use that.

model = tf.keras.models.Model(inputs=[model1.input, model2.input], outputs=t)
model.compile(loss='mse')

model.summary()

a = tf.constant([[1, 2, 3]])
b = tf.constant([[1, 2, 3]])

c = tf.constant([[1]])

model.fit([a, b], c)