3

According to wikipedia this is Jensen's inequality:

If X is a random variable and φ is a convex function, then:

$$\varphi\left(\mathbb{E}[X]\right) \leq \mathbb{E}\left[\varphi(X)\right].$$

Which stated as an implication reads as follows:

If Convex $\implies $ $\varphi\left(\mathbb{E}[X]\right) \leq \mathbb{E}\left[\varphi(X)\right].$

However, I was wondering if the converse was also true.

$\varphi\left(\mathbb{E}[X]\right) \leq \mathbb{E}\left[\varphi(X)\right]$ $\implies $ Convex.

I thought it would be an iff because of the definition of convexity. Recall what convexity means (and let me explain why I thought iff for jensen's):

f is called convex if: $$\forall x_1, x_2 \in X, \forall t \in [0, 1]: \qquad f(tx_1+(1-t)x_2)\leq t f(x_1)+(1-t)f(x_2).$$

Since its a definition, the the word convex and its definition statement are logically equivalent. Therefore, its an iff, i.e.

f is called convex $ \Leftrightarrow \forall x_1, x_2 \in X, \forall t \in [0, 1]: \qquad f(tx_1+(1-t)x_2)\leq t f(x_1)+(1-t)f(x_2).$

right? Does that logic generalize to Jensen's inequality or am I wrong for both?

Context:

The reason that I care about this is because I wanted to proof that a multivariable function f is convex. My plan of attack was to show Jensen's inequality "worked" on it and since Jensen's inequality and convexity are logically equivalent, then I can conclude that f is convex. But that only works if its an iif. Also, I absolutely wanted to avoid taking the Hessian or derivatives to prove it was convex, I strictly wanted to stick with the "original" definition of convexity to prove this (i.e. the inequality definition).

Charlie Parker
  • 6,546
  • 2
    If the inequality holds for all random variables (with expectation), then $\varphi$ is convex. – Daniel Fischer Nov 17 '14 at 15:57
  • @DanielFischer how is that not a contradiction to Seyhmus Güngören answer bellow? It seems he said that Jensen's cannot imply convexity but you say it does. I am confused right now. – Charlie Parker Nov 17 '14 at 16:15
  • 2
    That answer says that the inequality $\psi(\mathbb{E}[X]) \leqslant \mathbb{E}[\psi(X)]$ can hold for some random variables $X$ even if $\psi$ is not convex. [Note that in that answer, the chosen $\psi$ is convex on the range of the chosen $U$.] – Daniel Fischer Nov 17 '14 at 16:17
  • Is the random variable for that answer the continuous random variable that takes values in $U$ with any probability distribution? – Charlie Parker Nov 17 '14 at 16:23

3 Answers3

5

The convexity of $f$ is equivalent to Jensen's inequality holding for all random variables $X$. (If it holds for all $X$, then in particular it holds for the random variable with $P(X=x_1)=t$ and $P(X=x_2)=1-t$, where Jensen reduces to the definition of convexity.) So if somehow it's easier to prove Jensen for all $X$ (perhaps because the generality of that statement clears away distractions), then sure, your method is sound.

1

That is not true take $U$ uniform in $[0,1]$ and consider the function $f(x)=x^3$. Then one can see that $E[\psi(X)]\geq \psi(E[X])$ but $f$ is non-convex.

In general convexity is a much stronger argument than Jensen's inequality.

Seyhmus Güngören
  • 7,848
  • Sorry if this is a very easy question, but what does it mean that "convexity is a much stronger argument/statement". Are you using the word stronger in a precise way or is it just intuitive? Or both. Either way not sure what it means. Btw, thanx for your response. – Charlie Parker Nov 17 '14 at 15:58
  • 1
    Stronger in the sense that in order to imply covexity one needs to do a little bit more that Jensens inequality. More formally, it means that Jensens inequality is not strong enough to imply convexity. – Seyhmus Güngören Nov 17 '14 at 16:01
  • Seyhmus, I apologize but a comment on my question confused me. Daniel Fischer in the comments section said: "If the inequality holds for all random variables (with expectation), then φ is convex.", however, you said that jensen's inequality is not strong enough to imply convexity, but he said it can. Currently, I am confused, so can it or can it not imply convexity? Thanks for your time :) – Charlie Parker Nov 17 '14 at 16:18
  • there is no contradiction. Hear what he is saying. – Seyhmus Güngören Nov 17 '14 at 16:24
  • Oh ok, I think I get it, so its an iff only when its true for all r.vs X. Right? Thanks for your patience :) – Charlie Parker Nov 17 '14 at 16:27
  • I think this example is not that good, even though $f$ is not convex on $\mathbb R$ but it is convex on the interval $[0,1]$. I guess it's better to use $X \sim \text{Uniform}[-\epsilon, 1]$ for some $\epsilon >0$. – Matata Jul 27 '18 at 16:32
0

It depends on the underlying probability space.

For example, if the measure is $\delta_{1\over2}$ on $[0,1]$, then $E X= X({1 \over 2})$, $E \phi(X) = \phi(X({1 \over 2}))$, so Jensen's inequality holds whether or not $\phi$ is convex.

For all $t\in [0,1]$ you need to be able to find a set $A$ such that $pA = t$.

If you can do this, let $X=x_1 \cdot 1_A + x_2 \cdot 1_{X \setminus A}$. Then $EX = t x_1 + (1-t) x_2$ and $E \phi(X) = t \phi(x_1) + (1-t) \phi(x_2)$, which implies that $\phi$ is convex if Jensen's inequality holds.

copper.hat
  • 172,524