Visualize CNN layer or pooling layer in tflearn

Question

Is there any way I can visualize the output of CNN or pooling layer while training or even testing in tflearn? I have seen visualization code of tensorflow but since session and feeddict are involved their and I keep getting error like "unhashable numpy.ndarray" but my dimensions of the image are same so I decided to ask whether there is a way I can visualize the output of any layer. Below is my tflearn layers code:-

X_train, X_test, y_train, y_test=cross_validation.train_test_split(data,labels,test_size=0.1)

    tf.reset_default_graph()
    convnet=input_data(shape=[None,50,50,3],name='input')
    convnet=conv_2d(convnet,32,5,activation='relu')
    convnet=max_pool_2d(convnet,5)
    convnet=conv_2d(convnet,64,5,activation='relu')
    convnet=max_pool_2d(convnet,5)

    convnet=conv_2d(convnet,32,5,activation='relu')
    convnet=max_pool_2d(convnet,5)

    convnet=fully_connected(convnet,128,activation='relu')
    convnet=dropout(convnet,0.4)
    convnet=fully_connected(convnet,6,activation='softmax')
    convnet=regression(convnet,optimizer='adam',learning_rate=0.005,loss='categorical_crossentropy',name='MyClassifier')
    model=tflearn.DNN(convnet,tensorboard_dir='log',tensorboard_verbose=0)
    model.fit(X_train,y_train, n_epoch=20,validation_set=(X_test,y_test), snapshot_step=20,show_metric=True,run_id='MyClassifier')
    print("Saving the model")
    model.save('model.tflearn')

How can I visualize output from any layer either while training or testing anyway will work? By output I mean the distorted image detecting edges or other low level features. Thank you.

Answer 1

As mentioned here, you can see the output produced by an intermediate layer by simply defining a new model that has the observed layer as output. First, declare your original model (but keep a reference to the intermediate layers you want to observe):

convnet = input_data(shape=[None, 50, 50, 3], name='input')
convnet = conv_2d(convnet, 32, 5, activation='relu')
max_0 = max_pool_2d(convnet, 5)
convnet = conv_2d(max_0, 64, 5, activation='relu')
max_1 = max_pool_2d(convnet, 5)
...
convnet = regression(...)
model = tflearn.DNN(...)
model.fit(...)

Now just create a model to each layer and predict the input data:

observed = [max_0, max_1, max_2]
observers = [tflearn.DNN(v, session=model.session) for v in observed]
outputs = [m.predict(X_test) for m in observers]
print([d.shape for d in outputs])

Which outputs the following evaluated tensors' shapes for your model:

[(2, 10, 10, 32), (2, 2, 2, 64), (2, 1, 1, 32)]

With this, you will be able to look at the outputs during test. As for training, maybe you can use a callback?

class PlottingCallback(tflearn.callbacks.Callback):
    def __init__(self, model, x,
                 layers_to_observe=(),
                 kernels=10,
                 inputs=1):
        self.model = model
        self.x = x
        self.kernels = kernels
        self.inputs = inputs
        self.observers = [tflearn.DNN(l) for l in layers_to_observe]

    def on_epoch_end(self, training_state):
        outputs = [o.predict(self.x) for o in self.observers]

        for i in range(self.inputs):
            plt.figure(frameon=False)
            plt.subplots_adjust(wspace=0.1, hspace=0.1)
            ix = 1
            for o in outputs:
                for kernel in range(self.kernels):
                    plt.subplot(len(outputs), self.kernels, ix)
                    plt.imshow(o[i, :, :, kernel])
                    plt.axis('off')
                    ix += 1
            plt.savefig('outputs-for-image:%i-at-epoch:%i.png'
                        % (i, training_state.epoch))

model.fit(X_train, y_train,
          ...
          callbacks=[PlottingCallback(model, X_test, (max_0, max_1, max_2))])

This will save images similar to this on your disk, at each epoch: