Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

neuralnet package in R - how to obtain weights prior to training convergence?

I want to plot a learning curve to see the progress of a neural net during the course of its training. The horizontal axis is to be the total number of iterations, with the vertical axis representing the error rate. I wanted to see both the test and training set error rates as the network training progresses.

nn <- neuralnet(f, 
            data = train, 
            hidden = 2, 
            linear.output = F, 
            threshold = 0.01,
            stepmax = 10,
            lifesign = "full",
            learningrate = .1,
            algorithm='backprop')

By setting stepmax=10 (or 50 or?) I was hoping to be able to examine the network prior to convergence, see what the error rates are on the test and training set, and then continue the training for another 10 steps. The (partially) trained neural net is named nn, and I was planning to set the startweights to the weights obtained in the interrupted training as follows:

# Try to further train alerady trained net
nn <- neuralnet(f, 
            data = train, 
            hidden = 2, 
            linear.output = F, 
            threshold = 0.01,
            lifesign = "full",
            learningrate = .1,
            startweights = nn$weights,
            algorithm='backprop')

However, the training gave a warning that "algorithm did not converge in 1 of 1 repetition(s) within the stepmax". I wasn't expecting it to converge, but those 10 completed training steps should have modified the initially random weights. Alas, nn$weights is NULL.

Does anyone know of a way to accomplish this using neuralnet?

like image 609
Rafael_Espericueta Avatar asked Feb 08 '14 22:02

Rafael_Espericueta


1 Answers

I wrote directly to one of the authors of the neuralnet package, Frauke Guenther, and received his definitive answer:

"Unfortunately at the moment, the trained weights are only stored if the network is converged. It is not yet implemented that you can access the weight during the training process or if the network does not converge."

like image 182
Rafael_Espericueta Avatar answered Oct 06 '22 12:10

Rafael_Espericueta