Does LSTM in Keras support dynamic sentence length or not?

Question

Since I see people arguing on some platform that the LSTM in Keras does not support dynamic sentence length, I wrote the following code.

embedding_size = 100
model = Sequential()
model.add(LSTM(32, return_sequences=True, input_shape=(None, embedding_size)))

And it works perfectly given two input val1 and val2 I fed (those input are in shape batch_size * sentence length * embedding size),

val1 = np.random.random((5,20,embedding_size))
val2 = np.random.random((5,10,embedding_size))
input = model.input
output = model.output

with tf.Session() as sess:
    sess.run(tf.global_variables_initializer())
    k1 = sess.run(output, feed_dict={input:val1})
    k2 = sess.run(output, feed_dict={input:val2})
    print k1.shape
    print k2.shape

And I have the following output, which meets my expectation that the LSTM in Keras is dynamic in input length if we set the input_shape to be (None, embedding_size), am I understanding right?

(5, 20, 32)

(5, 10, 32)

Answer 1

Of course - as you mentioned it's possible to feed a varying length sequences to Keras model. The case is that in a single batch all sequences should have the same length. In this sense - it's impossible to implement a dynamic sequence length support.

If you consider this case more deeply - you'll see that even though you could have a varying length sequences fed in different batches - this could actually introduce some bias to your training process - as sequences of the same / similiar length are always fed together to a training process.