How to determine which neurons to connect between layers in an artificial neural network?

Question

Say for my first input layer I have 10 input nodes/neurons. Say my hidden layer has 10 neurons as well. My third and final layer is one output neuron.

How do I connect the layers? Is there a technique for determining the best way to do this or do you just connect each input neuron to each of the hidden layer neurons for a total of 100 edges between the two layers?

This is probably a really basic question but I haven't seen too many concrete examples. The examples I did find either seem to connect all the neurons or the connections seem randomized.

Answer 1

If anything, this is based on intuition and empirical results. I've seen people use recursive neural networks.

With a feedforward neural network, it makes sense to connect all neurons from layer n to all neurons in layer n+1.

Here is an example from my latest usage (to demonstrate the enormous amount of edges):

Answer 2

I think that if you cut some links between input nodes and hidden layer, you're artificially affecting the training phase. Basically you're giving more importance to the remaining links.

Even if you connect one neuron with only a fraction of the neurons in the next layer, it's like having them all connected but the communication weight is 0 for non connected neurons. The training algorithm may actually reach this situation.

What it really matters is the architecture of the neural network and the weights assigned to each input / output links of the neuron.

So connect each of the 10 input nodes of the input layer to all the 10 nodes in the hidden layer, and let the training algorithm do its job. If you have enough training and testing data, it will produce expected results.