Scipy.optimize.minimize method='SLSQP' ignores constraint

Question

I'm using SciPy for optimization and the method SLSQP seems to ignore my constraints.

Specifically, I want x[3] and x[4] to be in the range [0-1]

I'm getting the message: 'Inequality constraints incompatible'

Here is the results of the execution followed by an example code (uses a dummy function):

  status: 4
  success: False
njev: 2
nfev: 24
 fun: 0.11923608071680103
   x: array([-10993.4278558 , -19570.77080806, -23495.15914299, -26531.4862831 ,
     4679.97660534])
message: 'Inequality constraints incompatible'
 jac: array([ 12548372.4766904 ,  12967696.88362279,  39928956.72239509,
    -9224613.99092537,   3954696.30747453,         0.        ])
 nit: 2

Here is my code:

from random import random
from scipy.optimize import minimize

def func(x):
   """ dummy function to optimize """
   print 'x'+str(x)
   return random()

my_constraints = ({'type':'ineq', 'fun':lambda(x):1-x[3]-x[4]},
                  {'type':'ineq', 'fun':lambda(x):x[3]},
                  {'type':'ineq', 'fun':lambda(x):x[4]},
                  {'type':'ineq', 'fun':lambda(x):1-x[4]},
                  {'type':'ineq', 'fun':lambda(x):1-x[3]})

minimize(func, [57.9499 ,-18.2736,1.1664,0.0000,0.0765],
         method='SLSQP',constraints=my_constraints)

EDIT - The problem persists when even when removing the first constraint.

The problem persists when I try to use the bounds variables. i.e.,

bounds_pairs = [(None,None),(None,None),(None,None),(0,1),(0,1)]
minimize(f,initial_guess,method=method_name,bounds=bounds_pairs,constraints=non_negative_prob)

Answer 1

I know this is a very old question, but I was intrigued.

When does it happen?

This problem occurs when the optimisation function is not reliably differentiable. If you use a nice smooth function like this:

opt = numpy.array([2, 2, 2, 2, 2])

def func(x):
   return sum((x - opt)**2)

The problem goes away.

How do I impose hard constraints?

Note that none of the constrained algorithms in scipy.minimize have guarantees that the function will never be evaluated outside the constraints. If this is a requirement for you, you should rather use transformations. So for instance to ensure that no negative values for x[3] are ever used, you can use a transformation x3_real = 10^x[3]. This way x[3] can be any value but the variable you use will never be negative.

Deeper analysis

Investigating the Fortran code for slsqp yields the following insights into when this error occurs. The routine returns a MODE variable, which can take on these values:

C*        MODE = -1: GRADIENT EVALUATION, (G&A)                        *
C*                0: ON ENTRY: INITIALIZATION, (F,G,C&A)               *
C*                   ON EXIT : REQUIRED ACCURACY FOR SOLUTION OBTAINED *
C*                1: FUNCTION EVALUATION, (F&C)                        *
C*                                                                     *
C*                   FAILURE MODES:                                    *
C*                2: NUMBER OF EQUALITY CONTRAINTS LARGER THAN N       *
C*                3: MORE THAN 3*N ITERATIONS IN LSQ SUBPROBLEM        *
C*                4: INEQUALITY CONSTRAINTS INCOMPATIBLE               *
C*                5: SINGULAR MATRIX E IN LSQ SUBPROBLEM               *
C*                6: SINGULAR MATRIX C IN LSQ SUBPROBLEM               *

The part which assigns mode 4 (which is the error you are getting) is as follows:

C   SEARCH DIRECTION AS SOLUTION OF QP - SUBPROBLEM

      CALL dcopy_(n, xl, 1, u, 1)
      CALL dcopy_(n, xu, 1, v, 1)
      CALL daxpy_sl(n, -one, x, 1, u, 1)
      CALL daxpy_sl(n, -one, x, 1, v, 1)
      h4 = one
      CALL lsq (m, meq, n , n3, la, l, g, a, c, u, v, s, r, w, iw, mode)

C   AUGMENTED PROBLEM FOR INCONSISTENT LINEARIZATION

      IF (mode.EQ.6) THEN
          IF (n.EQ.meq) THEN
              mode = 4
          ENDIF
      ENDIF

So basically you can see it attempts to find a descent direction, if the constraints are active it attempts derivative evaluation along the constraint and fails with a singular matrix in the lsq subproblem (mode = 6), then it reasons that if all the constraint equations were evaluated and none yielded successful descent directions, this must be a contradictory set of constraints (mode = 4).