How can we prune the neural network in R?

Question

I have 30 independent variables in my model. I wanted to prune the neural network based on importance of variables. I have tried using mlp function of RSNNS package, but I don't know what arguments could be given to "pruneFunc" and "pruneFuncParams" ?
Is there exists any other way for pruning the neural network?

Answer 1

The "pruneFunc" and "pruneFuncParams" arguments in mlp() (RSNNS package) are used to prune the number of hidden layer neurons. I don't think you can use that to reduce your input variables.

You can use 'NeuralNetTools' package to run the Garson algorithm on your net. The functions in this package run on most neural network objects created by RSNNS (and other packages as well). The Garson algorithm will give you the importance of each variable based on the weights. You can then exclude those variables and retrain your network.

Refer to these links for more details on the Garson algorithm... https://beckmw.wordpress.com/2013/08/12/variable-importance-in-neural-networks/ https://beckmw.wordpress.com/tag/neural-network/

Alternately you can try other out many dimensionality reduction techniques (like PCA) before you begin training your neural network.

You have mentioned in the comment that "Reducing the variables and giving only the important variables to neural network gives better prediction"... but this isn't accurate...

An MLP with 2 hidden layers can approximate any function given sufficient training, and usually MLP with 1 hidden layer is enough to approximate most functions. If there is any input variable which has very low importance (or even absolutely no role to play in your model), sufficient amount of training will make the weights linked to that variable very small (close to zero) so as not to have any significant impact on your predictions. The only significant benefit of reducing the number of variables is the lower computation time for predictions.