Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Understanding what weight sharing is in convolutional neural network

I am currently trying understand what weight sharing actually is in convolutional neural network.

From what I can read was CNN first introduced as a way to reduce the number of connection required to connect input and output given an input had 3 dimensions.

Following that logic it made sense for me that convolution reduces of one of these dimensions, and connect the convoluted image to an output neuron.

Whether the weight connecting the convoluted image and the output are the weight that not shared? And if not what are the weights that are not shared?

Or if yes does back propagation then view it as one weight, and update it as one unit?

An illustration of what I think weight sharing:

like image 261
Agumon Avatar asked Oct 18 '25 19:10

Agumon


1 Answers

A CNN has multiple layers. Weight sharing happens across the receptive field of the neurons (filters) in a particular layer. Weights are the numbers within each filter. So essentially we are trying to learn a filter. These filters act on a certain receptive field/small section of the image. When the filter moves through the image, the filter does not change. The idea being, if an edge is important to learn in a particular part of an image, it is important in other parts of the image too.

like image 93
Vin Avatar answered Oct 22 '25 05:10

Vin