how to train a model for item sequence segmentation

Question

If got a sequence of items I want to split into multiple segments. To do this I want to train a statistical model.

e.g.

Fruit:     Apple Banana Peach | Apple Banana | Apple Banana Peach | Banana
Segment:   1     1      1     | 2     2      | 3     3      3     | 4

In this example the Model should learn that apple always starts a new segment and banana as well if it is not preceded by an apple.

I thought of the following approaches:

1. Train a Neuronal Network to predict if an item starts a new segment based on the items before and after it (maybe LSTM?) which would basically be a 2-class Classification

2. Train a Neuronal Network to predict the sequence numbers based on the a part of the item sequence. This would be a sequence-2-sequence model.

Maybe someone of you had a similar problem and can share his experiences. Thanks!

Answer 1

You don't need complex methods if all you need to see is if the sequence starts. Two ifs will be enough.

Though, if you have n number of these sequence starts, you can still store them in an array and update their numbers and check according to that.

You don't need to jump to ML until you have simple solutions.

If I am missing something about the complexity of your problem, do tell.

Maybe if you didn't know when the sequence, starts then clustering would have told you.

Answer 2

Yes, I would also use some sort of recurrent neural net. I'm not sure you have to encode the inputs therefore I wouldn't recommend using a seq2seq network. You could maybe just do a GRU (faster LSTM), and then a dense layer that would predict the probability of finding an apple based on previous input.
You can maybe even just use the decoder from the seq2seq as your entire model.

Pay attention to that don't make the model super complex because there is not many features. But when you get your example to work, you could maybe predict what a user will buy based on what the user also bought, so if the user buys apples then predict a pear and a banana but if the user buys a mango the recommend something more exotic

Happy researching!