How does the unpooling and deconvolution work in DeConvNet

Question

I have been trying to understand how unpooling and deconvolution works in DeConvNets.

Unpooling

While during the unpooling stage, the activations are restored back to the locations of maximum activation selections, which makes sense, but what about the remaining activations? Do those remaining activations need to be restored as well or interpolated in some way or just filled as zeros in unpooled map.

Deconvolution

After the convolution section (i.e., Convolution layer, Relu, Pooling ), it is common to have more than one feature map output, which would be treated as input channels to successive layers ( Deconv.. ). How could these feature maps be combined together in order to achieve the activation map with same resolution as original input?

Answer 1

Unpooling

As etoropov wrote, you can read about unpooling in Visualizing and Understanding Convolutional Networks by Zeiler and Ferguson:

Unpooling: In the convnet, the max pooling operation is non-invertible, however we can obtain an approximate inverse by recording the locations of the maxima within each pooling region in a set of switch variables. In the deconvnet, the unpooling operation uses these switches to place the reconstructions from the layer above into appropriate locations, preserving the structure of the stimulus. See Fig. 1(bottom) for an illustration of the procedure.

Deconvolution

Deconvolution works like this:

• You add padding around each pixel
• You apply a convolution

For example, in the following illustration the original blue image is padded with zeros (white), the gray convolution filter is applied to get the green output.

Source: What are deconvolutional layers?

Answer 2

1 Unpooling.

In the original paper on unpooling, remaining activations are zeroed.

2 Deconvolution.

A deconvolutional layer is just the transposed of its corresponding conv layer. E.g. if conv layer's shape is [height, width, previous_layer_fms, next_layer_fms], than the deconv layer will have the shape [height, width, next_layer_fms, previous_layer_fms]. The weights of conv and deconv layers are shared! (see this paper for instance)