Connect 4 with neural network: evaluation of draft + further steps

Question

I would like to build a Connect 4 engine which works using an artificial neural network - just because I'm fascinated by ANNs.

I'be created the following draft of the ANN structure. Would it work? And are these connections right (even the cross ones)?

Could you help me to draft up an UML class diagram for this ANN?

I want to give the board representation to the ANN as its input. And the output should be the move to chose.

The learning should later be done using reinforcement learning and the sigmoid function should be applied. The engine will play against human players. And depending on the result of the game, the weights should be adjusted then.

What I'm looking for ...

... is mainly coding issues. The more it goes away from abstract thinking to coding - the better it is.

Answer 1

The below is how I organized my design and code when I was messing with neural networks. The code here is (obviously) psuedocode and roughly follows Object Oriented conventions.

Starting from the bottom up, you'll have your neuron. Each neuron needs to be able to hold the weights it puts on the incoming connections, a buffer to hold the incoming connection data, and a list of its outgoing edges. Each neuron needs to be able to do three things:

• A way to accept data from an incoming edge
• A method of processing the input data and weights to formulate the value this neuron will be sending out
• A way of sending out this neuron's value on the outgoing edges

Code-wise this translates to:

// Each neuron needs to keep track of this data
float in_data[]; // Values sent to this neuron
float weights[]; // The weights on each edge
float value; // The value this neuron will be sending out
Neuron out_edges[]; // Each Neuron that this neuron should send data to

// Each neuron should expose this functionality
void accept_data( float data ) {
    in_data.append(data); // Add the data to the incoming data buffer
}
void process() {
    value = /* result of combining weights and incoming data here */;
}
void send_value() {
    foreach ( neuron in out_edges ) {
        neuron.accept_data( value );
    }
}

Next, I found it easiest if you make a Layer class which holds a list of neurons. (It's quite possible to skip over this class, and just have your NeuralNetwork hold a list of list of neurons. I found it to be easier organizationally and debugging-wise to have a Layer class.) Each layer should expose the ability to:

• Cause each neuron to 'fire'
• Return the raw array of neurons that this Layer wraps around. (This is useful when you need to do things like manually filling in input data in the first layer of a neural network.)

Code-wise this translates to:

//Each layer needs to keep track of this data.
Neuron[] neurons;

//Each layer should expose this functionality.
void fire() {
    foreach ( neuron in neurons ) {
        float value = neuron.process();
        neuron.send_value( value );
    }
}
Neuron[] get_neurons() {
    return neurons;
}

Finally, you have a NeuralNetwork class that holds a list of layers, a way of setting up the first layer with initial data, a learning algorithm, and a way to run the whole neural network. In my implementation, I collected the final output data by adding a fourth layer consisting of a single fake neuron that simply buffered all of its incoming data and returned that.

// Each neural network needs to keep track of this data.
Layer[] layers;

// Each neural network should expose this functionality
void initialize( float[] input_data ) {
    foreach ( neuron in layers[0].get_neurons() ) {
        // do setup work here
    }
}
void learn() {
    foreach ( layer in layers ) {
        foreach ( neuron in layer ) {
            /* compare the neuron's computer value to the value it
             * should have generated and adjust the weights accordingly
             */
        }
    }
}
void run() {
    foreach (layer in layers) {
        layer.fire();
    }
}

I recommend starting with Backwards Propagation as your learning algorithm as it's supposedly the easiest to implement. When I was working on this, I had great difficulty trying to find a very simple explanation of the algorithm, but my notes list this site as being a good reference.

I hope that's enough to get you started!

Answer 2

There are a lot of different ways to implement neural networks that range from simple/easy-to-understand to highly-optimized. The Wikipedia article on backpropagation that you linked to has links to implementations in C++, C#, Java, etc. which could serve as good references, if you're interested in seeing how other people have done it.

One simple architecture would model both nodes and connections as separate entities; nodes would have possible incoming and outgoing connections to other nodes as well as activation levels and error values, whereas connections would have weight values.

Alternatively, there are more efficient ways to represent those nodes and connections -- as arrays of floating point values organized by layer, for example. This makes things a bit trickier to code, but avoids creating so many objects and pointers to objects.

One note: often people will include a bias node -- in addition to the normal input nodes -- that provides a constant value to every hidden and output node.