Menu
I have partially understood Max-pooling, after reading Convolutional Neural Networks (LeNet):
Another important concept of CNNs is max-pooling, which is a form of non-linear down-sampling. Max-pooling partitions the input image into a set of non-overlapping rectangles and, for each such sub-region, outputs the maximum value.
What about Sum-pooling? I couldn't find any easy to understand article.
533
There are two types of Pooling: Max Pooling and Average Pooling . Max Pooling returns the maximum value from the portion of the image covered by the Kernel. On the other hand, Average Pooling returns the average of all the values from the portion of the image covered by the Kernel.
max-pooling layer gave the largest value in a certain subarea as an output, while the global max-pooling did this in the whole area.
Maximum pooling, or max pooling, is a pooling operation that calculates the maximum, or largest, value in each patch of each feature map. The results are down sampled or pooled feature maps that highlight the most present feature in the patch, not the average presence of the feature in the case of average pooling.
Pooling mainly helps in extracting sharp and smooth features. It is also done to reduce variance and computations. Max-pooling helps in extracting low-level features like edges, points, etc. While Avg-pooling goes for smooth features.
Sum pooling (which is proportional to Mean pooling) measures the mean value of existence of a pattern in a given region. UPDATE: The subregions for Sum pooling / Mean pooling are set exactly the same as for Max pooling but instead of using max function you use sum / mean.
Max pooling is used much more often than average pooling with one exception which is sometimes very deep in the neural network you might use average pooling to collapse your representation from say 7x7x1000 and average over all the spatial experiments you get 1x1x1000.
This makes the model more robust to variations in the position of the features in the input image. Max pooling is a pooling operation that selects the maximum element from the region of the feature map covered by the filter.
The input to the max-over-time pooling is a feature map c = [c (1), ..., c (n-h+1)], which is computed over a sentence of length n with a filter of size h. The convolution operation is very similar to one with images, but in this case it's applied to 1-dimensional vector of words. This is the formula (3) in the paper.
Convolutional Neural Networks do a great job in dealing with high dimensional data. Restricting the number of weights only to kernels weights makes learning easier due to invariance properties of images or sound. But if you look carefully at what's going on you may notice that the after first convolutional layer the dimension of your data might severely increase if you don't do the tricks like pooling.
Max pooling decreases the dimension of your data simply by taking only the maximum input from a fixed region of your convolutional layer. Sum pooling works in a similiar manner - by taking the sum of inputs instead of it's maximum.
The conceptual difference between these approaches lies in the sort of invariance which they are able to catch. Max pooling is sensitive to existence of some pattern in pooled region. Sum pooling (which is proportional to Mean pooling) measures the mean value of existence of a pattern in a given region.
UPDATE:
The subregions for Sum pooling / Mean pooling are set exactly the same as for Max pooling but instead of using max function you use sum / mean. You can read about here in the paragraph about pooling.
151
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
