neuralnet in R: what are the difference between stepmax and rep parameters?

Question

What are the difference between stepmax and rep parameters in neuralnet package? Am I correct that stepmax it's maximal count of all gradient steps, which neural net make on all training samples? Am I correct that rep it's number of how many times neural net can learn from 1 example?

Answer 1

Your understanding of stepmax is essentially correct. If I am not mistaken, the neuralnet package only uses gradient descent using the entire data set, calculating the gradients, updating the weights, and repeating until either convergence (defined by threshold) or stepmax is reached.

The rep parameter, as far as I can tell is nothing more than a wrapper for looping over creating a neural network. There is some inherent randomness in creating a neural network so by setting rep > 1 the function will create multiple starting weights and fit both. If you do this, for example:

library(neuralnet)
data(infert, package="datasets")
net.infert <- neuralnet(case~parity+induced+spontaneous, infert, 
                        err.fct="ce", linear.output=FALSE, likelihood=TRUE,
                        rep = 3)

length(nn.infert$startweights)
[1] 3

length(nn.infert$weights)
[1] 3

Whereas the lengths of both would be 1 otherwise. This is intended to make it easier to evaluate each repetition with compute by again specifying the rep parameter which simply selects with list elements to use.

This whole thing could be down with a simple for loop but it packages it within the function objects to make it more 'convenient'. The point is to make sure the model you create wasn't found by random chance (i.e. likely overfit).