Multiple Embedding layers for Keras Sequential model

Question

I am using Keras (tensorflow backend) and am wondering how to add multiple Embedding layers into a Keras Sequential model.

More specifically, I have several columns in my dataset which have categorical values and I have considered using one-hot encoding but have determined that the number of categorical items is in the hundreds leading to a large and far too sparse set of columns. Upon looking for solutions I have found that Keras' Embedding layer appears to solve the problem very elegantly. However, most of the examples (and Keras documentation) illustrate a very simple situation with one Embedding layer.

Unfortunately, I do not know how to integrate the multiple Embedding layers as input into a single model.

My code looks like this, but it does not work, and I am guessing that the multiple Embedding layers act sequentially (first Embedding layer is input the the second and so on) rather than be a multiple input sources to the model:

model = Sequential()
model.add(Embedding(500, 64, input_length=10))  # categorical col 1
model.add(Embedding(100, 64, input_length=10))  # categorical col 2
model.add(Embedding(500, 64, input_length=10))  # categorical col 3
model.add(Flatten... 
model.add(Dense...

My question is how would I establish a Keras Sequential model such that I would be able to use the three Embedding layers shown above. What specifically goes in between the first and last layers:

model = Sequential()
#
# What goes here?
#
model.add(Dense...

Am I on the right track, or is my approach incorrect and I need to establish the model in a different manner? Any suggestions/examples are appreciated!

Answer 1

This can be done easily if you switch to the functional API, first have a read. Then you can build a model with multiple inputs that represent different columns:

col1, col2, col3 = Input(shape=(10,)), Input(shape=(10,)), ...
col1_embeded = Embedding(500, 64)(col1)
col2_embedded = Embedding(100, 64)(col2)
# ...

The gist of this the layers are callable objects that build the computation graph. You can also for example share embedding layers between columns by simply using the same Embedding layer.