How to organize the Recurrent Neural Network?

Question

I want to model the following:

y(t)=F(x(t-1),x(t-2),...x(t-k))

or lets say a function that its current output is depended on the last k inputs.

1- I know one way is to have a classic Neural Network with k inputs as {x(t-1),x(t-2),...x(t-k)} for each y(t) and train it. Then what's the benefit of using a RNN to solve that problem?

2- Assuming using RNN, should i use only the x(t) (or x(t-1)) and assume the hidden layer(s) can find the relation of y(t) to the past k inputs through having the in its memory (hidden layer)?

3- using deep nets like Deep RNN or LSTM has any superior benefit for such problem considering we want to estimate the output based on the last k inputs?

Answer 1

1. I would not advice you to use a classic vanilla RNN. Theorethicaly it has an ability to store informations from previous inputs in its memory but practically it requires an expotentially large number of nodes.
2. Assuming classic vanilla implementations as long as modern architectures (like e.g. LSTM or GRU) - it depends on if you want to use one directional or bidirectional model. If you want to predict next step - usually one directional architecture is better. If you want to better analyze sequences given - I advice you to apply bidirectional one.
3. LSTMs and GRUs makes usage of additional memory cells which helps you in keeping long time dependiencies between inputs in memory. They are considered as the best architectures right now. Deep RNNs - are usually deep networks with recurrent topologies - they make use of its depth in the same manner as feedforward neural nets.