How to input mask value to Convolution1D layer

Question

I need to feed variable length sequences into my model.

My model is Embedding + LSTM + Conv1d + Maxpooling + softmax.

When I set mask_zero = True in Embedding, I fail to compile at Conv1d.

How can I input mask value in Conv1d or is there another solution?

Answer 1

The Masking layer expects every downstream layer to support masking, which is not the case of the Conv1D layer. Fortunately, there is another way to apply masking, using the Functional API:

inputs = Input(...)
mask = Masking().compute_mask(inputs) # <= Compute the mask
embed = Embedding(...)(inputs)
lstm = LSTM(...)(embed, mask=mask) # <= Apply the mask
conv = Conv1D(...)(lstm)
...
model = Model(inputs=[inputs], outputs=[...])

Answer 2

Conv1D layer does not support masking at this time. Here is an open issue on the keras repo.

Depending on the task you might be able to get away with embedding the mask_value just like the other values in the sequence and apply global pooling (as you're doing now).