What can Jensen's inequality tell me about bias of an estimator? I don't really get how to use it and I would like someone to explain what convexity and Jensen's inequality has to do with bias.
In particular, I am trying to learn to use Jensen's inequality to decide whether the MLE of an exponential distribution is biased. I don't really understand how the inequality works. I couldn't really find anything that explained it easily, so I decided to work backwards by guessing how to use it and see if I learned something.
So far my current thinking is as follows. Let $g(x) = \theta e^{-\theta x}$. Then $$\Bbb E[g(x)] = \frac{1}{\theta}$$ Also, $$ g(\Bbb E[X]) = \theta e^{-\theta \Bbb E[X]}$$
So then Jensen's inequality says if $X$ is a rv and $g$ is a convex function, then $$\frac{1}{\theta} \geq \theta e^{-\theta \Bbb E[X]}$$
But I'm not really sure what this has to do with the bias of the MLE for an exponential distribution.
I can also rearrange this to get
$$e^{\theta \Bbb E[X]} \geq \theta^2 $$ $$\theta \Bbb E[X] \geq ln(\theta^2) $$ $$\Bbb E[X] \geq \frac{ln(\theta^2)}{\theta} $$
I don't know if that gets me anything.
