Menu
I am trying to build a text LSTM autoencoder in Keras. I want to use an embedding layer but I'am not sure how to implement this. The code looks like this.
inputs = Input(shape=(timesteps, input_dim))
embedding_layer = Embedding(numfeats + 1,
EMBEDDING_DIM,
weights=[data_gen.get_embedding_matrix()],
input_length=maxlen,
trainable=False)
embedded_sequence = embedding_layer(inputs)
encoded = LSTM(num_units)(inputs)
decoded = RepeatVector(timesteps)(encoded)
decoded = LSTM(???, return_sequences=True)(decoded)
sequence_autoencoder = Model(inputs, decoded)
sequence_autoencoder.compile(loss='binary_crossentropy', optimizer='adam')
I am not sure how to decode the output into the target sequence (which is obviously the input sequence).
306
An LSTM network is a type of recurrent neural network (RNN) that can learn long-term dependencies between time steps of sequence data. A word embedding layer maps a sequence of word indices to embedding vectors and learns the word embedding during training. This layer requires Deep Learning Toolbox™.
Auto-encoders, are a type of architecture, where embedding layers are used. Using these architectures, one can calculate word2vec. word2vec values are calculated when words are fed into the auto-encoders.
LSTM autoencoder is an encoder that makes use of LSTM encoder-decoder architecture to compress data using an encoder and decode it to retain original structure using a decoder.
Each autoencoder learns to represent input sequences as lower-dimensional, fixed-size vectors. This can be useful for finding patterns among sequences, clustering sequences, or converting sequences into inputs for other algorithms.
You can first convert the word to embeddings and pass them to the fit()
expected_output = np.array([[embedding_matrix[word_index] for word_index in encoded_sequence] for encoded_sequence in padded_sequences])
history = lstm_autoencoder.fit(padded_sequences, expected_output, epochs=15, verbose=1)
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
