Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Should an input layer include a bias neuron?

I was wondering: in a multi-layer feed-forward neural network should the input layer include a bias neuron, or this is just useful in hidden layers? If so, why?

like image 366
tunnuz Avatar asked Aug 30 '11 10:08

tunnuz


1 Answers

No, an input layer doesn't need a connection to the bias neuron, since any activation it received from the bias neuron would be completely overridden by the actual input.

For example, imagine a network that's trying to solve the classic XOR problem, using this architecture (where the neuron just marked 1 is the bias):

enter image description here

To run this network on input (1,0), you simply clamp the activation of neurons X1=1 and X2=0. Now, if X1 or X2 had also received input from the bias, then that input would be overridden anyways, thus making such a connection pointless.

like image 127
zergylord Avatar answered Sep 21 '22 02:09

zergylord