Understanding lstm input shape in keras with different sequence

Question

I'm very new to keras and also to python. I have a time series dataset with different sequence lengths (for example 1st sequence is 484000x128, 2nd sequence is 563110x128, etc) I've put the sequences in 3D array.

My question is how to define the input shape, because I'm confused. I was using DL4J but the concept is different in defining the network configuration.

Here is my first trial code:

import numpy as np
from keras.models import Sequential
from keras.layers import Embedding,LSTM,Dense,Dropout


## Loading dummy data
sequences = np.array([[[1,2,3],[1,2,3]], [[4,5,6],[4,5,6],[4,5,6]]])
y = np.array([[[0],[0]], [[1],[1],[1]]])
x_test=np.array([[2,3,2],[4,6,7],[1,2,1]])
y_test=np.array([0,1,1])

n_epochs=40

# model configration
model = Sequential()
model.add(LSTM(100, input_shape=(3,1), activation='tanh', recurrent_activation='hard_sigmoid')) # 100 num of LSTM units
model.add(LSTM(100, activation='tanh', recurrent_activation='hard_sigmoid'))
model.add(Dense(1, activation='softmax'))
model.compile(loss='binary_crossentropy',
              optimizer='adam',
              metrics=['accuracy'])

print(model.summary())

## training with batches of size 1 (each batch is a sequence)
for epoch in range(n_epochs):
    for seq, label in zip(sequences, y):
        model.train(np.array([seq]), [label]) # train a batch at a time..
        scores=model.evaluate(x_test, y_test) # evaluate batch at a time..

Answer 1

Here is the docs on input shapes for LSTMs:

Input shapes

3D tensor with shape (batch_size, timesteps, input_dim), (Optional) 2D tensors with shape (batch_size, output_dim).

Which implies that you you're going to need timesteps with a constant size for each batch.

The canonical way of doing this is padding your sequences using something like keras's padding utility

then you can try:

# let say timestep you choose: is 700000 and dimension of the vectors are 128

timestep = 700000
dims = 128 

model.add(LSTM(100, input_shape=(timestep, dim),
         activation='tanh', recurrent_activation='hard_sigmoid'))

I edited the answer to remove the batch_size argument. With this setup the batch size is unspecified, you could set that when you fitting the model (in model.fit()).