Line search fails in training ksvm prob.model

Question

Following up from Invalid probability model for large support vector machines using ksvm in R:

I am training an SVM using ksvm from the kernlab package in R. I want to use the probability model, but during the sigmoid fitting I get the following error message:

line search fails -1.833726 0.5772808 5.844462e-05 5.839508e-05 -1.795008e-08 
-1.794263e-08 -2.096847e-12

When this happens, the resulting value of prob.model(m) is a vector of all probabilities, rather than the expected parameters of a sigmoid function fitted over these probabilities. What causes this error and how can I prevent it? Searching for the error message yielded no results.

Reproducible example:

load(url('http://roelandvanbeek.nl/files/df.rdata'))
ksvm(label~value,df[1:1000],C=10,prob.model=TRUE)->m 
prob.model(m) # works as it should, prints a list containing one named list

# the below, non-working problem, unfortunately takes an hour due to the large
# sample size
ksvm(label~value,df,C=10,prob.model=TRUE)->m # line search fails  
prob.model(m) # just a vector of values

Answer 1

Looking at the source code, this is the line that throws that error.

It's on the method .probPlatt using the Newton method to optimize the function, in this case Platt's scaling. If you check line 3007 though you'll see some parameters pertaining to the method.

One of such parameters is minstep basically the minimal numeric step the method should keep trying to optimize the function. You see, this is exactly the condition of the error in line 3090: if (stepsize < minstep). So, basically, the function is not converging, even when reaching the minimum step size.

You can try changing minstep to lower values to circumvent it. Alexandros even commented these parameters should probably be in the interface.