PyBrain: When creating network from ground up how and where do you create a bias?

Question

Following the PyBrain documentation, Building Networks with Modules and Connections, I'm building a neural network piecewise (in contrast to using the buildNetwork shortcut). I'm constructing a simple 3-layer (input, hidden, output) neural network. How do I properly add a bias unit?

I'm guessing I construct a BiasUnit module as in:

b = BiasUnit(name='bias')
network.addModule(b)

Is this the right way? Do I have to create FullConnection object? If so, what should I be connecting?

Answer 1

Realized PyBrain is open source and I have the source code sitting in my Python directory. I opened the C:\Python27\Lib\site-packages\pybrain\tools\shortcuts.py file. Inside this file I located the buildNetwork function and saw how it adds BiasUnit's. The relevant code is here:

...
n = Network()
# linear input layer
n.addInputModule(LinearLayer(layers[0], name='in'))
# output layer of type 'outclass'
n.addOutputModule(opt['outclass'](layers[-1], name='out'))
if opt['bias']:
    # add bias module and connection to out module, if desired
    n.addModule(BiasUnit(name='bias'))
    if opt['outputbias']:
        n.addConnection(FullConnection(n['bias'], n['out']))
# arbitrary number of hidden layers of type 'hiddenclass'
for i, num in enumerate(layers[1:-1]):
    layername = 'hidden%i' % i
    n.addModule(opt['hiddenclass'](num, name=layername))
    if opt['bias']:
        # also connect all the layers with the bias
        n.addConnection(FullConnection(n['bias'], n[layername]))
# connections between hidden layers
...

Basically it looks like it creates a single BiasUnit and connects it to each hidden layer and optionally to the output layer as well.