Menu
I am training a LSTM model with Keras on the dataset which looks like following. The variable "Description" is a text field and "Age" and "Gender" are categorical and continuous fields.
Age, Gender, Description
22, M, "purchased a phone"
35, F, "shopping for kids"
I am using word-embedding to convert the text fields to word vectors and then input it in the keras model. The code is given below:
model = Sequential()
model.add(Embedding(word_index, 300, weights=[embedding_matrix], input_length=70, trainable=False))
model.add(LSTM(300, dropout=0.3, recurrent_dropout=0.3))
model.add(Dropout(0.6))
model.add(Dense(1))
model.add(Activation('sigmoid'))
model.compile(loss='binary_crossentropy', optimizer='adam', metrics['accuracy'])
This model is running successfully but I want to input "age" and "gender" variables as features as well. What changes are required in the code to use these features as well ?
522
Again, we can do this with a built in Keras function, in this case the pad_sequences() function. We are now ready to define our Embedding layer as part of our neural network model. The Embedding has a vocabulary of 50 and an input length of 4.
An embedding layer is faster, because it is essentially the equivalent of a dense layer that makes simplifying assumptions. A Dense layer will treat these like actual weights with which to perform matrix multiplication.
We load this embedding matrix into an Embedding layer. Note that we set trainable=False to prevent the weights from being updated during training. An Embedding layer should be fed sequences of integers, i.e. a 2D input of shape (samples, indices) .
You want to add more input layers which is not possible with Sequential Model, you have to go for functional model
from keras.models import Model
which allows you to have multiple inputs and indirect connections.
embed = Embedding(word_index, 300, weights=[embedding_matrix], input_length=70, trainable=False)
lstm = LSTM(300, dropout=0.3, recurrent_dropout=0.3)(embed)
agei = Input(shape=(1,))
conc = Concatenate()(lstm, agei)
drop = Dropout(0.6)(conc)
dens = Dense(1)(drop)
acti = Activation('sigmoid')(dens)
model = Model([embed, agei], acti)
model.compile(loss='binary_crossentropy', optimizer='adam', metrics['accuracy'])
You cannot concatenate before LSTM layer as it doesn't make sense and also you will have 3D Tensor after embedding layer and input is a 2D Tensor.
91
I wrote about how to do this in keras. It's basically a functional multiple input model, which concatenates both feature vectors like this:
nlp_input = Input(shape=(seq_length,), name='nlp_input')
meta_input = Input(shape=(10,), name='meta_input')
emb = Embedding(output_dim=embedding_size, input_dim=100, input_length=seq_length)(nlp_input)
nlp_out = Bidirectional(LSTM(128))(emb)
x = concatenate([nlp_out, meta_input])
x = Dense(classifier_neurons, activation='relu')(x)
x = Dense(1, activation='sigmoid')(x)
model = Model(inputs=[nlp_input , meta_input], outputs=[x])
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
