How to handle extremely long LSTM sequence length?

Question

I have some data that is sampled at at a very high rate (on the order of hundreds of times per second). This results in a sequence length that is huge (~90,000 samples) on average for any given instance. This entire sequence has a single label. I am trying to use an LSTM neural network to classify new sequences as one of these labels (multiclass classification).

However, using an LSTM with a such a large sequence length results in a network that is quite large.

What are some methods to effectively 'chunk' these sequences so that I could reduce the sequence length of the neural network, yet still maintain the information captured in the entire instance?

Answer 1

Three years later, we have what seems to be the start of solutions for this type of problem: sparse transformers.

See

https://arxiv.org/abs/1904.10509

https://openai.com/blog/sparse-transformer/

Answer 2

This post is from some time ago, but I thought I would chime in here. For this specific problem that you are working on (one-dimensional continuous-valued signal with locality, composition-ality, and stationarity), I would highly recommend a CNN convolutional neural network approach, as opposed to using an LSTM.