Print layer outputs in Keras during training

Question

I am new to Keras. How can I print the outputs of a layer, both intermediate or final, during the training phase?

I am trying to debug my neural network and wanted to know how the layers behave during training. To do so I am trying to exact input and output of a layer during training, for every step.

The FAQ (https://keras.io/getting-started/faq/#how-can-i-obtain-the-output-of-an-intermediate-layer) has a method to extract output of intermediate layer for building another model but that is not what I want. I don't need to use the intermediate layer output as input to other layer, I just need to print their values out and perhaps graph/chart/visualize it.

I am using Keras 2.1.4

Answer 1

I think I have found an answer myself, although not strictly accomplished by Keras.

Basically, to access layer output during training, one needs to modify the computation graph by adding a print node.

A more detailed description can be found in this StackOverflow question:
How can I print the intermediate variables in the loss function in TensorFlow and Keras?

I will quote an example here, say you would like to have your loss get printed per step, you need to set your custom loss function as:

for Theano backend:

diff = y_pred - y_true
diff = theano.printing.Print('shape of diff', attrs=['shape'])(diff)
return K.square(diff)

for Tensorflow backend:

diff = y_pred - y_true
diff = tf.Print(diff, [tf.shape(diff)])
return K.square(diff)

Outputs of other layers can be accessed similarly.

There is also a nice vice tutorial about using tf.Print() from Google
Using tf.Print() in TensorFlow