I'm trying to build an RNN using python / keras. I understand how it's done with one feature (with t+1 being the output), but how is done with multiple features?
What if I had a regression problem and a dataset with a few different features, one expected output, and I wanted to have the time steps / window set to 30 (so a month if each step represents a day) - what would the shape of the data be? In this example I'd want to be able to predict the output n time periods in the future.
See below for an example of what this data would look like:
I'm having a hard time intuitively understanding the best shape / format the data needs to be for RNNs.
In addition, how well do RNNs handle data sets with, say, 500 features and a few thousand records?
Hopefully someone could help answer or point me in the right direction to get one - so far I've posted on Reddit and Cross Validated with no luck :(
If a code data example is preferred:
# random df
df = pd.DataFrame({'date': np.array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9]),
'feature_1': np.random.randint(10, size=10),
'feature_2': np.random.randint(10, size=10),
'feature_3': np.random.randint(10, size=10),
'feature_4': np.random.randint(10, size=10),
'output': np.random.randint(10, size=10)}
)
# set date as index
df.index = df.date
df = df.drop('date', 1)
Let's say you have 2 timeseries X and Y and you want to predict X using both timeseries.
If we choose a timestep of 3 and suppose that we have at our disposal, (X1,...,Xt)
and (Y1,...,Yt)
, the first sample would be :
[[X1,X2,X3],[Y1,Y2,Y3]]
and the associated output : X4
.
The second one would be [[X2,X3,X4],[Y2,Y3,Y4]]
with X5
as output.
And the last one : [[Xt-3,Xt-2,Xt-1],[Yt-3,Yt-2,Yt-1]]
with Xt
as output.
For example, in the first sample : first you will feed to the network (X1,Y1)
, then (X2,Y2)
and (X3,Y3)
.
Here is a code to create the input and output and then use a LSTM net to do prediction :
import pandas as pd
import numpy as np
import keras.optimizers
from keras.models import Sequential
from keras.layers import Dense,Activation
from keras.layers import LSTM
#random df
df = pd.DataFrame({'date': np.array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9]),
'feature_1': np.random.randint(10, size=10),
'feature_2': np.random.randint(10, size=10),
'feature_3': np.random.randint(10, size=10),
'feature_4': np.random.randint(10, size=10),
'output': np.random.randint(10, size=10)}
)
# set date as index
df.index = df.date
df = df.drop('date', 1)
nb_epoch = 10
batch_size = 10
learning_rate = 0.01
nb_units = 50
timeStep = 3
X = df[['feature_'+str(i) for i in range(1,5)]].values # Select good columns
sizeX = X.shape[0]-X.shape[0]%timeStep # Choose a number of observations that is a multiple of the timstep
X = X[:sizeX]
X = X.reshape(X.shape[0]/timeStep,timeStep,X.shape[1]) # Create X with shape (nb_sample,timestep,nb_features)
Y = df[['output']].values
Y = Y[range(3,len(Y),3)] #Select the good output
model = Sequential()
model.add(LSTM(input_dim = X.shape[2],output_dim = nb_units,return_sequences = False)) # One LSTM layer with 50 units
model.add(Activation("sigmoid"))
model.add(Dense(1)) #A dense layer which is the final layer
model.add(Activation('linear'))
KerasOptimizer = keras.optimizers.RMSprop(lr=learning_rate, rho=0.9, epsilon=1e-08, decay=0.0)
model.compile(loss="mse", optimizer=KerasOptimizer)
model.fit(X,Y,nb_epoch = nb_epoch,batch_size = batch_size)
prediction = model.predict(X)
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With