What are the advantages or disadvantages of having multiple output nodes compared to a few within a neural network

Question

Are there any advantages or disadvantages of having many output nodes in a neural network compared to having a few?

For example if a scenario could be represented with 10, 3, 2 or 1 output nodes, which is considered better? - obviously it depends how you want to represent the outputs but say that doesn't matter for now.

Or does the number of output nodes not make a difference to the accuracy of the network, just the computational time needed to train it?

Answer 1

The number of output nodes should match the number of values you want to compute. For simple regression or binary classification, you need only one output node. For multiclass classification or multiple regression, you need multiple output nodes.

In particular, binary classification using a feedforward neural net is done by computing the activation of a single output node, then checking whether it is larger than some threshold (commonly 0 or .5). For multiclass classification with k classes, you compute the values of k output nodes, then select the index i of the largest value to predict class i.

Also, with multiple output nodes you can do multilabel classification, where you again have a single output node per class/label and predict "true" for all nodes exceeding the threshold. Multilabel classification with k classes can also be performed by k binary classifiers, but you'd need to train all of those separately, which can be time-consuming with neural networks.