Implementing a many-to-many LSTM in TensorFlow?

Question

I am using TensorFlow to make predictions on time-series data. So it is like I have 50 tags and I want to find out the next possible 5 tags.

As shown in the following picture, I want to make it like the 4th structure.

I went through the tutorial demo: Recurrent Neural Networks

But I found it can provide like the 5th one in the above picture, which is different.

I am wondering which model could I use? I am thinking of the seq2seq models, but not sure if it is the right way.

Answer 1

You are right that you can use a seq2seq model. For brevity I've written up an example of how you can do it in Keras which also has a Tensorflow backend. I've not run the example so it might need tweaking. If your tags are one-hot you need to use cross-entropy loss instead.

from keras.models import Model
from keras.layers import Input, LSTM, RepeatVector

# The input shape is your sequence length and your token embedding size
inputs = Input(shape=(seq_len, embedding_size))

# Build a RNN encoder
encoder = LSTM(128, return_sequences=False)(inputs)

# Repeat the encoding for every input to the decoder
encoding_repeat = RepeatVector(5)(encoder)

# Pass your (5, 128) encoding to the decoder
decoder = LSTM(128, return_sequences=True)(encoding_repeat)

# Output each timestep into a fully connected layer
sequence_prediction = TimeDistributed(Dense(1, activation='linear'))(decoder)

model = Model(inputs, sequence_prediction)
model.compile('adam', 'mse')  # Or categorical_crossentropy
model.fit(X_train, y_train)