When to use GlobalAveragePooling1D and when to use GlobalMaxPooling1D while using Keras for an LSTM model?

Question

I have to make LSTM classification model for some text and I am confused between GlobalAveragePooling1D and GlobalMaxPooling1D in the pooling layer while using keras. Which one should I use and what are the things to consider while deciding a particular choice.

Answer 1

It depends a lot on your data and what you want from it. This is one of those cases where testing would be the only sure answer.

For instance, if you want to detect the presence of something in your sequences, max pooling seems a good option.

But if the contribution of the entire sequence seems important to your result, then average pooling sounds reasonable.

Now, since you're using LSTM layers, perhaps you should use return_sequences=False in the last LSTM layer. This is also a possibility instead of pooling. This will keep the last step of the sequence only. There might be the advantage of having the entire sequence processed, perhaps with cumulative effects in your result.

In all cases, testing is the only definitive answer.