How to use additional features along with word embeddings in Keras ?

Question

I am training a LSTM model with Keras on the dataset which looks like following. The variable "Description" is a text field and "Age" and "Gender" are categorical and continuous fields.

Age, Gender, Description
22, M, "purchased a phone"
35, F, "shopping for kids"

I am using word-embedding to convert the text fields to word vectors and then input it in the keras model. The code is given below:

model = Sequential()
model.add(Embedding(word_index, 300, weights=[embedding_matrix], input_length=70, trainable=False))

model.add(LSTM(300, dropout=0.3, recurrent_dropout=0.3))
model.add(Dropout(0.6))
model.add(Dense(1))
model.add(Activation('sigmoid'))
model.compile(loss='binary_crossentropy', optimizer='adam', metrics['accuracy'])

This model is running successfully but I want to input "age" and "gender" variables as features as well. What changes are required in the code to use these features as well ?

Answer 1

You want to add more input layers which is not possible with Sequential Model, you have to go for functional model

from keras.models import Model

which allows you to have multiple inputs and indirect connections.

embed = Embedding(word_index, 300, weights=[embedding_matrix], input_length=70, trainable=False)
lstm = LSTM(300, dropout=0.3, recurrent_dropout=0.3)(embed)
agei = Input(shape=(1,))
conc = Concatenate()(lstm, agei)
drop = Dropout(0.6)(conc)
dens = Dense(1)(drop)
acti = Activation('sigmoid')(dens)

model = Model([embed, agei], acti)
model.compile(loss='binary_crossentropy', optimizer='adam', metrics['accuracy'])

You cannot concatenate before LSTM layer as it doesn't make sense and also you will have 3D Tensor after embedding layer and input is a 2D Tensor.

Answer 2

I wrote about how to do this in keras. It's basically a functional multiple input model, which concatenates both feature vectors like this:

nlp_input = Input(shape=(seq_length,), name='nlp_input')
meta_input = Input(shape=(10,), name='meta_input')
emb = Embedding(output_dim=embedding_size, input_dim=100, input_length=seq_length)(nlp_input)
nlp_out = Bidirectional(LSTM(128))(emb)
x = concatenate([nlp_out, meta_input])
x = Dense(classifier_neurons, activation='relu')(x)
x = Dense(1, activation='sigmoid')(x)
model = Model(inputs=[nlp_input , meta_input], outputs=[x])