Element-wise constraints in scipy.optimize.minimize

Question

I'm using scipy.optimize.minimize's COBYLA method to find a matrix of parameters for a categorical distribution. I need to impose the constraint that each parameter is greater than zero, and that the sum of the rows of the parameter matrix is a column of ones.

It's not clear to me how to implement this in scipy.minimize, because the constraints are checked for non-negativity rather than truth. The minimization raises an exception if I just pass the arrays as the constraint.

Does anyone know how to go about implementing these kinds of constraints?

Answer 1

The first constraint x > 0 can be expressed very simply:

{'type':'ineq', 'fun': lambda x: x}

The second constraint is an equality constraint, which COBYLA doesn't natively support. However, you could express it as two separate inequality constraints instead:

{'type':'ineq', 'fun': lambda x: np.sum(x, 0) - 1}  # row sum >= 1
{'type':'ineq', 'fun': lambda x: 1 - np.sum(x, 0)}  # row sum <= 1

Otherwise you could try SLSQP instead, which does support equality constraints.

Answer 2

You need equality constraints that enforce np.sum(x, 1) == 1 and inequality constraints for x >= 0.

However, the COBYLA method can only handle inequality constraints, as explained in the documentation of minimize (see the section that explains the constraints argument). Instead, you can use Sequential Least SQuares Programming (SLSQP), which supports both types of constraints. The minimize function should automatically choose the correct solver for you, based on the constraints you specify.

The constraints you need can be implemented like this:

def ineq_constraint(x):
    """constrain all elements of x to be >= 0"""
    return x

def eq_constraint(x):
    """constrain the sum of all rows to be equal to 1"""
    return np.sum(x, 1) - 1


constraints = [{'type': 'ineq', 'fun': ineq_constraint},
               {'type': 'eq', 'fun': eq_constraint}]

result = minimize(objective_function, x0, constraints=constraints)