Can someone explain to me the difference between activation and recurrent activation arguments passed in initialising keras lstm layer?

Question

Can someone explain to me the difference between activation and recurrent activation arguments passed in initialising keras lstm layer?

According to my understanding LSTM has 4 layers. Please explain what are th e default activation functions of each layer if I do not pass any activation argument to the LSTM constructor?

Answer 1

On code

Line from 1932

i = self.recurrent_activation(z0)
f = self.recurrent_activation(z1)
c = f * c_tm1 + i * self.activation(z2)
o = self.recurrent_activation(z3)
h = o * self.activation(c)

recurrent_activation is for activate input/forget/output gate.

activation if for cell state and hidden state.

Answer 2

An LSTM Unit has 3 gates called the input, forget, and output gates, in addition to a candidate hidden state (g), and an output hidden state (c).

The build method in the LSTMCell class contains the implementation where these activations are called (https://github.com/keras-team/keras/blob/master/keras/layers/recurrent.py#L1892).

The recurrent_activation argument applies to the input, forget, and output gates. The default value for this argument is a hard-sigmoid function. The activation argument applies to the candidate hidden state and output hidden state. The default value for this argument is a hyperbolic tangent function.