How to combine multiple objectives for optimization?

Question

I don't know why this is so hard for me to figure out.

For example, I have two functions, f(x, y) and g(x, y). I want to find the values of x and y such that:

• f(x, y) is at a target value (minimize the difference from the target)
• g(x, y) is minimized (can be negative, doesn't stop at 0)
• x and y are bounded (so g's minimum doesn't necessarily have a gradient of 0)

So if I were just finding a solution for f, I could minimize abs(f(x, y) - target), for instance, and it will hit zero when it's found a solution. But there are multiple such solutions and I also want to find the one that minimizes g.

So how do I combine these two functions into a single expression that I can then minimize (using a Newton-like method)?

My first attempt was 100*abs(f(x, y) - target) + g(x, y) to strongly emphasize hitting the target first, and it worked for some cases of the target value, but failed for others, since g(x, y) could go so negative that it dominated the combination and the optimizer stopped caring about f. How do I guarantee that f hitting the target is always dominant?

Are there general rules for how to combine multiple objectives into a single objective?

Answer 1

There is a rich literature about multi-objective optimization. Two popular methods are weighted objective and a lexicographic approach.

A weighted objective could be designed as:

min w1 * [f-target]^2 + w2 * g

for some weights w1, w2 >= 0. Often we have w1+w2=1 so we can also write:

min w1 * [f-target]^2 + (1-w1) * g

Set w1 to a larger value than w2 to put emphasis on the f objective.

The lexicographic method assumes an ordering of objectives. It can look like:

1. Solve with first objective z = min [f-target]^2. Let z* be the optimal objective.
2. Solve with the second objective while staying close to z*: min g subject to [f-target]^2-z* <= tolerance

To measure the deviation between the target and f I used a quadratic function here. You can also use an absolute value.