what is the difference between xtol and ftol to use fmin() of scipy.optimize?

Question

I'm starting to use the function fmin with a very simple example and I try to get the values ​​of a vector that minimizes the value of their multiplication:

def prueba(x,y):
    print "valor1:",x[0],"\n"
    print "valor2:",x[1],"\n"
    print "valor3:",x[2],"\n"
    print "valor4:",x[3],"\n"
    min=x[0]*x[1]*x[2]*x[3]
    print min
    return min

sal = fmin(prueba,x0=array([1, 2, 3,4]),args="1",retall=1,xtol=0.5,ftol=0.5)#maxfun=1,maxiter=1,retall=1,args="1")

but if I dont define xtol and ftol appears:

"Warning: Maximum number of function evaluations has been exceeded."

For this reason I have defined the convergence of the algorithm using the parameters xtol and ftol,but i still don't understand what is the difference between them, I look the same, but if I delete one of the two I get the warning again.

What exactly is the difference of xtol and ftol?, Which should use in this case?.

I have read the documentation:

OtherParameters

xtol : number
acceptable relative error in xopt for convergence.
ftol : number
acceptable relative error in func(xopt) for convergence.

I still do not understand

Answer 1

Here's my understanding. It's similar to the mathworks function fminsearch. They define these values:

TolFun: Termination tolerance on the function value TolX: Termination tolerance on x

As the search proceeds in its iterative fashion. The difference in the values of x from one iteration to another become smaller and smaller, until it doesn't matter any more and you might as well be done. Same goes for the function tolerance. In your example, prueba is evaluated and the difference between its return value from iteration to iteration gets smaller and smaller, until it doesn't matter either. You asked which you should use. This can bit a bit of an experimental approach. In the past I have often used:

xtol = 1e-6;
ftol = 1e-6;

It seems to scale well to many problems, and is a good place to start. You will likely find that if one needs to be tweaked, it will be obvious. Like horrid convergence times. Poor goodness of fit in the data, etc. Hope this helps.