1

Assuming that constraints are satisfiable.

In general, does adding constraints make an optimization problem harder since the optimal solution does not necessarily satisfy all constraints? Or does it make it easier by reducing the search space for a feasible solution?

Perhaps "in general" doesn't make sense at all, in which case, under what conditions does adding constraints make the problem easier and when does it make it harder?

Rufus
  • 215

2 Answers2

2

Nobody adds or subtracts constraints from an optimization problem to make it easier or harder. A real-life optimization problem has a well-defined set $S$ of feasible states and a certain object function. Now the set $S$ can be described in different ways, often with too many variables which then are kept under control by constraints. I don't think there is a general philosophy that would dictate in which way to proceed in such circumstances: eliminating superfluous variables can introduce undesirable boundary conditions, or similar, etcetera.

Maybe you better show us an example where you are really in doubt.

  • "Nobody adds or subtracts constraints from an optimization problem to make it easier or harder." Oh, but they do. Whether or not they should is another matter. – Mark L. Stone Oct 05 '19 at 18:18
1

Perhaps to answer my own question now that I have a better understanding of optimization (in particular convex optimization).

Here, I'll define "hardness" as the amount of computation required to reach a global optimum.

The "hardness" of an optimization problem depends less on whether or not there are constraints, but more on the form of the constraint and also the cost function.

A major threshold that makes a problem significantly harder is when the new constraint / cost function converts the program from convex to non-convex (with non-convex problems being significantly harder to solve).

Convex problems in general have very efficient algorithms for reaching global optimums of which the number of constraints plays a rather insignificant role (of course, less constraints makes it easier, but adding more convex constraints does not impact the solve time much).

In the realm of non-convex optimization problems, the problem gets more complicated and usually depends on the cost function and constraints themselves.

Removing constraints obviously can help by making the problem less convex but adding constraints may also help push the solution to a better (i.e. from infeasible to feasible or from higher cost to lower cost) local optima. So the answer is, it depends and YMMA.

Of note, for non-convex problems, providing an initial guess (even if infeasible) can greatly help steer the optimization to a better local optima.

Rufus
  • 215