Menu
I'm new to RNN, and I'm trying to figure out the specifics of LSTM cells and they're relation to TensorFlow: Colah GitHub
Does the GitHub website's example uses the same LSTM cell compared to TensorFlow? The only thing I got on the TensorFlow site was that basic LSTM cells uses the following architecture: Paper If it's the same architecture then I can hand compute the numbers for a LSTM cell and see if it matches.
Also when we set a basic LSTM cell in tensorflow, it takes in a num_units according to: TensorFlow documentation
tf.nn.rnn_cell.GRUCell.__init__(num_units, input_size=None, activation=tanh)
Is this number of hidden state (h_t)) and cell state (C_t)?
According to the GitHub website, there isn't any mention the number of cell state and hidden states. I'm assuming they have to be the same number?
577
Disadvantages Of RNNThe computation of this neural network is slow. Training can be difficult. If you are using the activation functions, then it becomes very tedious to process long sequences. It faces issues like Exploding or Gradient Vanishing.
RNN have a “memory” which remembers all information about what has been calculated. It uses the same parameters for each input as it performs the same task on all the inputs or hidden layers to produce the output. This reduces the complexity of parameters, unlike other neural networks.
RNNs suffer from the problem of vanishing gradients. The gradients carry information used in the RNN, and when the gradient becomes too small, the parameter updates become insignificant. This makes the learning of long data sequences difficult.
Definitely you can have multiple hidden layers in RNN.
Implementation looks the same as GRUCell class doc also points the same paper (specifically for gated) with link given in Colah's article. Parameter num_units is the number of cells (assuming that is the hidden layer) corresponds to output_size due property definition.
157
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
