1

The maximum likelihood estimator of an exponential distribution $f(x, \lambda) = \lambda e^{-\lambda x}$ is $\lambda_\text{MLE} = \frac n {\sum x_i}$; I know how to derive that by taking the derivative of the log likelihood and setting it equal to zero.

I then read in an article that "Unfortunately this estimator is clearly biased since $\left\langle \sum_i x_i\right\rangle$ is indeed $1/\lambda$ but $\langle 1/\sum_i x_i \rangle \neq \lambda$."

Why does $\left\langle\sum_i x_i\right\rangle = 1/\lambda$? If I am correct in deducing the $\langle \cdot\rangle$ operator means expected value, then I thought $E(x_i) = 1/\lambda$ - that is, the expected value of one such $x_i$, is $1/\lambda$, not the sum of all $x_i$'s. And can someone explain the second of the statement and how these two statements demonstrate the MLE is biased?

Michael Hardy
  • 1
fool
  • 79

2 Answers2

2

$\newcommand{\e}{\operatorname{E}}$You have $$ \Pr\left( \sum_{i=1}^n x_i \in A \right) = \int\limits_A \frac 1 {\Gamma(n)} (\lambda u)^{n-1} e^{-\lambda u} (\lambda\,du), $$ i.e. the sum has a gamma distribution.

So \begin{align} \e\left( \frac n {\sum_{i=1}^n x_i} \right) & = \int_0^\infty \frac n u \cdot \frac 1 {\Gamma(n)} (\lambda u)^{n-1} e^{-\lambda u} (\lambda\,du) \\[10pt] & = \frac {n\lambda} {\Gamma(n)} \int_0^\infty (\lambda u)^{n-2} e^{-\lambda u} (\lambda\,du) \\[10pt] & = \frac{n\lambda}{\Gamma(n)} \cdot\Gamma(n-1) = \frac n {n-1} \lambda. \end{align}

Michael Hardy
  • 1
1

This should read: "This estimator is biased since hence $E(1/\lambda_{MLE})=1/\lambda$ but $E(\lambda_{MLE})\ne\lambda$".

The fact that $E(1/\lambda_{MLE})=1/\lambda$ is direct since $E(x_i)=1/\lambda$ for every $i$ and $1/\lambda_{MLE}=(x_1+\cdots+x_n)/n$.

And, by convexity, $E(\lambda_{MLE})\gt1/E(1/\lambda_{MLE})=\lambda$.

Did
  • 279,727