Proving concavity of a function in multiple variables

Question

How do I prove that $f(\vec{x}) = \vec{b}.\vec{x} - \log \left (1+e^{\vec{a}.\vec{x}} \right )$ is concave? where $\vec{a}$ and $\vec{b}$ are constant vectors.

My steps are as follows: The first term is a linear function in $\vec{x}$ so the first term is concave in $\vec{x}$.

If I can prove that the second term is concave in $\vec{x}$, then the sum of two concave functions is concave.

In order to show that the second term is concave, I have to show that $\log \left (1+e^{\vec{a}.\vec{x}} \right )$ is convex, right? How to do that?

How do you define the difference of a vector and a scalar? – Haris Gušić Feb 23 '19 at 22:03 — Haris Gušić, Feb 23 '19 at 22:03

angryavian · Accepted Answer · 2019-02-23T23:35:51.797

0

Let $f(x) = \log(1 + e^{a^\top x})$. It suffices to show that the Hessian is positive semi-definite.

The gradient is $$\frac{1}{1+e^{-a^\top x}} a$$ and the Hessian is therefore $$\frac{e^{-a^\top x}}{(1+e^{-a^\top x})^2} a a^\top.$$

The Hessian is a rank one matrix whose only nonzero eigenvalue is $\frac{e^{-a^\top x}}{(1+e^{-a^\top x})^2} \|a\|^2 \ge 0$, so the Hessian is positive semi-definite.

edited Feb 23 '19 at 23:35

answered Feb 23 '19 at 22:19

angryavian

89,882

The expression for your gradient seems wrong. Should it be = $\frac{e^{a^T x}}{1 + e^{a^T x} } a$ . Nevermind, I did not see the minus sign – rims Feb 23 '19 at 22:27
@AdeelMahmood Multiply your expression by $e^{-a^\top x}/e^{-a^\top x}$ to obtain mine. – angryavian Feb 23 '19 at 22:28
Correct me if I am wrong. But shouldn't the numerator in the expression for Hessian have $e^{-a^T x}$? Thanks! – rims Feb 23 '19 at 23:28
@AdeelMahmood yes, sorry about that. – angryavian Feb 23 '19 at 23:36

score 0 · Answer 2 · answered Feb 23 '19 at 22:20

Differentiate the function $x \mapsto \log(1+e^x)$ twice to show that it is strictly convex.

If $f$ is convex and $g$ is affine then it is straightforward to show that $f \circ g$ is convex.

The map $x \mapsto \langle a, x \rangle$ is linear hence affine.

Proving concavity of a function in multiple variables

2 Answers2