Menu
What is the difference between a "Local" layer and a "Dense" layer in a convolutional neural network? I am trying to understand the CIFAR-10 code in TensorFlow, and I see it uses "Local" layers instead of regular dense layers. Is there any class in TF that supports implementing "Local" layers?
708
asked Dec 21 '15 10:12
Dense Layer is simple layer of neurons in which each neuron receives input from all the neurons of previous layer, thus called as dense. Dense Layer is used to classify image based on output from convolutional layers. Working of single neuron. A layer contains multiple number of such neurons.
As known, the main difference between the Convolutional layer and the Dense layer is that Convolutional Layer uses fewer parameters by forcing input values to share the parameters. The Dense Layer uses a linear operation meaning every output is formed by the function based on every input.
Local layering allows artists to stack layers like objects in the real world. In a conventional 2d painting or compositing program, graphical objects are stacked in a user-specified global order, as if each were printed on an image-sized sheet of transparent film.
Convolutional layers are technically locally connected layers. To be precise, they are locally connected layers with shared weights. We run the same filter for all the (x,y) positions in the image. In other words, all the pixel positions “share” the same filter weights.
Quoting from cuda-convnet:
Locally-connected layer with unshared-weight: This kind of layer is just like a convolutional layer, but without any weight-sharing. That is to say, a different set of filters is applied at every (x, y) location in the input image. Aside from that, it behaves exactly as a convolutional layer.
In the TensorFlow CIFAR-10 example, although the two layers are named local3 and local4, they are actually fully-connected layer, not locally-connected layer as specified in cuda-convnet (you can see that the output from pool2 is flattened into the input of local3 layer).
112
answered Sep 20 '22 02:09
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With