Understanding a multilayer perceptron network

Question

I'm trying to understand how to train a multilayer; however, I'm having some trouble figuring out how to determine a suitable network architecture--i.e., number of nodes/neurons in each layer of the network.

For a specific task, I have four input sources that can each input one of three states. I guess that would mean four input neurons firing either 0, 1 or 2, but as far as I'm told, input should be kept binary?

Furthermore am I having some issues choosing on the amount of neurons in the hidden layer. Any comments would be great.

Thanks.

Answer 1

I disagree with doug's answer above on a few points.

You have 4 discrete (3 way categorical) inputs. You should (unless you have a strong reason not to) represent that as 12 binary inputs using a 1-of-3 encoding for each of your four conceptual inputs. So if you input is [2,0,1,1] then your network should be given: 0 0 1 1 0 0 0 1 0 0 1 0 If your network implementation requires a manual bias, then you should add another always on bit for the bias, but most sensible neural net implementations don't require that.

Try a few different numbers of hidden units. You don't need to restrict yourself to a hidden layer size smaller than the input layer size, but if you make it larger you should be careful to regularlize your weights, perhaps with L2 or L1 weight decay and maybe even also doing early-stopping in training (stop training when your error on a held out validation set stops improving).