XOR Neural Network - unexpected results

Question

I am trying to implement Daniel Shiffman's XOR Neural Network in swift, I have all the parts, but after training, the results are unexpected.

Part of me thinks it's the actual training system trying to learn multiple things at once.

I have linked my playground in case anyone can spot anything wrong: https://www.dropbox.com/s/9rv8ku3d62h03ip/Neural.playground.zip?dl=0

Daniels code:

https://github.com/shiffman/The-Nature-of-Code-Examples/blob/master/chp10_nn/xor/code/src/Network.java

Answer 1

There are a couple of errors in your code. The first (and most important) is a subtlety in the way you're creating your networks.

Right now you're using

inputs = [Neuron](repeating: Neuron(), count:2+1)
hidden = [Neuron](repeating: Neuron(), count:4+1)

But this creates all the inputs with the same Neuron and also all the hidden with the same Neuron, so there are only 4 Neurons: 2 for input (the regular repeated 2 times and a bias neuron) and 2 for hidden (the regular repeated 4 times and 1 for bias).

You can solve it by simply using a for loop:

public class Network
{
    var inputs:[Neuron] = []
    var hidden:[Neuron] = []
    var output:Neuron!

    public init()
    {
        for _ in 1...2 {
            inputs.append(Neuron())
        }

        for _ in 1...4 {
            hidden.append(Neuron())
        }

        //print("inputs length: \(inputs.count)")

        inputs.append(Neuron(bias: true))
        hidden.append(Neuron(bias: true))

        output = Neuron()

        setupInputHidden()
        setupHiddenOutput()
    }

    ...
}

The other (minor) thing is when you calculate the output of a Neuron you're adding the bias instead of replacing it (bias = from.output*c.weight), I don't know if that was on purpose but the result seems to be unaffected.