How to input new text for prediction in keras while using an inbuilt dataset

Question

I am going through the examples in keras and I ran the example for using an LSTM for classifying sentiment on the inbuilt imdb dataset (https://github.com/fchollet/keras/blob/master/examples/imdb_lstm.py).

On inspecting the data, each review is depicted as an array of numbers which I assume is their index from a vocabulary built using this dataset.

My question however is how do I input a new piece of text(something that I make up) into this model to get a prediction? How would I get access to this vocabulary of words?

After that I could preprocess by input text into an array of numbers and feed it in. Thanks!

Answer 1

When Predicting new text, you have to follow the same step you have done for training.

• Preprocess this new sentence.
• Convert the text to vector using word_index
• Pad the vector's with same length as you specified during training
• Flatten the array and pass it as an input to your model

sentences = clean_text(text)

word_index = imdb.get_word_index()

x_test = [[self.word_index[w] for w in sentences if w in self.word_index]]

x_test = pad_sequences(x_test, maxlen=maxlen) # Should be same which you used for training data

vector = np.array([x_test.flatten()])

model.predict_classes(vector)