Tensorflow: show or save forget gate values in LSTM

Question

I am using the LSTM model that comes by default in tensorflow. I would like to check or to know how to save or show the values of the forget gate in each step, has anyone done this before or at least something similar to this?

Till now I have tried with tf.print but many values appear (even more than the ones I was expecting) I would try plotting something with tensorboard but I think those gates are just variables and not extra layers that I can print (also cause they are inside the TF script)

Any help will be well received

Answer 1

If you are using tf.rnn_cell.BasicLSTMCell , the variable you are looking for will have the following suffix in its name : <parent_variable_scope>/BasicLSTMCell/Linear/Matrix . This is a concatenated matrix for all the four gates. Its first dimension matches the sum of the second dimensions of the input matrix and the state matrix (or output of the cell to be exact). The second dimension is 4 times the number of cell size.

The other complementary variable is <parent_variable_scope>/BasicLSTMCell/Linear/Bias that is a vector of the same size as the second dimension of the abovementioned tensor (for obvious reasons).

You can retrieve the parameters for the four gates by using tf.split() along dimension 1. The split matrices would be in the order [input], [new input], [forget], [output]. I am referring to the code here form rnn_cell.py.

Keep in mind that the variable represents the parameters of the Cell and not the output of the respective gates. But with the above info, I am sure you can get that too, if you so desire.

Edit:
Added more specific information about the actual tensors Matrix and Bias