0

To perform constrained maximization, we can construct the generalized Lagrange function of $-f(x)$, which leads to this optimization problem: $$ \min _{x} \max _{\lambda} \max _{\alpha, \alpha \geq 0}-f(x)+\sum_{i} \lambda_{i} g^{(i)}(x)+\sum_{j} \alpha_{j} h^{(j)}(x) . $$ We may also convert this to a problem with maximization in the outer loop: $$ \max _{x} \min _{\lambda} \min _{\alpha, \alpha \geq 0} f(x)+\sum_{i} \lambda_{i} g^{(i)}(x)-\sum_{j} \alpha_{j} h^{(j)}(x) . $$ *Deep Learning*, Bengio and al. page 93

I don't get why when converting only the original $f(x)$ function and the inequality constraint/term ($\sum_{j} \alpha_{j} h^{(j)}(x)$) change signs.

1 Answers1

0

You can view it as being the case that the $\sum_{j} \alpha_{j} h^{(j)}(x)$ term also changes sign. But because the Lagrange multipliers on equality constraints are not sign-restricted, this is the same as not changing the sign of that term, except that the Lagrange multiplier will be the negative of what it would have been had the sign of that term been changed.

I suppose it would have been less confusing if the author had changed the sign.