5

Convex Optimization Snippet

In showing the convex conjugate of log-sum-exp function, $f(x) = \log(\sum_{i=1}^n e^{x_i})$, Boyd argues that the domain of the convex conjugate, $$f^*(y) = \sup_{x \in D} \{y^Tx - f(x)\}$$ is $y \succeq 0$ since the conjugate would be unbounded otherwise.

I follow along by setting the derivative inside the supremum to zero, we get:

$$y_i = \frac{e^{x_i}}{\sum_{j=1}^n e^{x_j}}$$

Applying this to the convex conjugate, when I work out the domain in a similar manner to Boyd, supposing that one of the $y_k < 0$ then we have an unbounded range, by setting the corresponding $x_k$ to approach negative infinite and all other $x_i = $ for $i \neq k$.

But this is not the most general restriction, as we only need to have that $y_k \geq -1$, since in which case $y_kx_k - x_k = (y_k - 1)x_k$ bounds the supremum as $x_k$ approaches negative infinite?

  • What is $D$ here? – zhoraster Feb 20 '16 at 09:46
  • A classmate answered:

    I think you made a tiny error and accidentally computed $\log(\sum_i \exp(x_i)) = \log(\exp(x_k)) = x_k$ which is why you end up with the expression $y_k x_k - x_k$. (I am not sure how else one would end up with such an expression.)

    Anyhow, Boyd claims that if some $y_k < 0$ then $y^\top x - f(x)$ is unbounded above so let us check this carefully. Similar to Boyd, let us set $x_k = -t$ where $t > 0$. In which case $y^\top x - f(x) = -y_k t - \log((n-1) + e^{-t}) \geq -y_k t - \log (n)$. But $y_k < 0$ so $-y_k t - \log(n) \to \infty$ as $t \to \infty$.

    – Michael Nguyen Mar 04 '16 at 07:18

1 Answers1

1

The Conjugate Function is a Concave Function in its domain.
You can easily see that the derivative implies:

$$ {y}_{i} = \frac{ {e}^{ {x}_{i} } }{ \sum_{j = 1}^{n} {e}^{ {x}_{j} } } $$

This requires $ {y}_{i} > 0 $ (Which can be later relaxes by the definition of $ 0 \ln \left( 0 \right) $).

Royi
  • 8,711
  • 1
    "concave function in its domain" It's convex rather than concave, right? [Because any supremum of convex functions is convex.] – littleO Jul 06 '21 at 04:18