7

Let $X_1,\ldots,X_n$ to be sample distributed geometric with parameter $p$. Find MLE. Is it unbiased?

The distribution for each is $p(1-p)^{x_i-1}$ so the function is $$L(p)=\displaystyle\prod_{i=1}^np(1-p)^{X_i-1}.$$ After taking lns on both sides I got $$l(p)=\ln(L(p))=n\log(p)+\sum_{i=1}^n(X_i-1)\cdot \log(1-p).$$ I derivatied and found maximum in $p_m=\dfrac{n}{n+\sum_{i=1}^n(X_i-1)}$. Now I need to calculate $E[p_m]$: $$E[p_m]=nE\left[\frac{1}{\sum X_i}\right]$$ How can proceed?

Michael Hardy
  • 1
  • First note that you expression simplifies, which you can see if you put parentheses where needed: $\sum_i (X_i - 1)$. This is an exponential family, and you will find that the MLE is the same as the method of moments estimator $\hat{p} = 1/\bar{X}$ where $\bar{X} = \frac1n \sum_i X_i$. – passerby51 Aug 22 '13 at 00:02
  • Now, consider the case $n=1$. Is it true that $E[\frac{1}{X}] = p$? – passerby51 Aug 22 '13 at 00:04
  • Why $E[\frac 1 X]=p$? –  Aug 22 '13 at 00:06
  • 3
    For $n=1$, the estimator is $\hat{p} = 1/X_1$. Being unbiased means $E[\hat{p}] = p$. I should have said that as: "Is $E[\frac{1}{X_1}] = p$ true, in which case the estimator is unbiased?" – passerby51 Aug 22 '13 at 00:10
  • but as far as I know when we talk about discrete variable )X can get values $x_1... x_k$) $E[X]=\sum_{i=1}^k x*P(X=x_i)$ but here we have various xs. I still don't understand how can we answer the question you asked. –  Aug 22 '13 at 00:17
  • 1
    What I mean is this, when they say an estimator is unbiased, it means that it is unbiased for any number of samples, that is for any $n$. If you can show that it is not unbiased for a particular $n$, the simplest being $n=1$, then you have shown that it is not unbiased. Assume that you just have one sample $X_1$, what is the estimator in this case, and what is its expectation? (You might not be able compute the expectation, but you might be able to show that it is not equal to $p$.) – passerby51 Aug 22 '13 at 00:38

1 Answers1

5

Here is one way to answer this. Consider the case $n = 1$. The estimator in this case is $\hat{p} = 1/X_{1}$. Let us try to see what it is expectation is, $$ E[\hat{p}] = E\Big[ \frac1{X_1}\Big] = \sum_{k=1}^\infty \frac{1}{k} P(X_1 = k) = \sum_{k=1}^\infty \frac1k p(1-p)^{k-1} $$

Hint: Note that for $ \alpha \in (-1,1)$, we have $\sum_{k=1}^\infty \frac{\alpha^k}{k} = - \log(1-\alpha)$.

EDIT2: You can obtain the exact expression, or use the following simple bound $$ E(\hat{p}) = p + \sum_{k=2}^{\infty} \frac{1}{k} p (1-p)^{k-1} > p $$ for $p \in (0,1)$ since the sum above is strictly positive.

passerby51
  • 4,140
  • and then $E[\hat{p}]=p\cdot\sum\frac{1}{k}(1-p)^{k-1}< p\cdot\sum(1-p)^{k-1}=\frac{p}{1-(1-p)}=p$ proves the estimation is unbiased? –  Aug 22 '13 at 00:51
  • Sorry. I didn't check your bound. Your bound does not seem correct. In general bounding is easier, but in this case, the bound you have is $p \sum (1-p)^{k-1} = 1$ which is not enough. – passerby51 Aug 22 '13 at 01:15
  • fine, another try: $E[\hat{p}]=\frac p {(1-p)} \sum\frac{(1-p)^k}{k}=-\frac{p\cdot log(p)}{1-p}$ which is less than p (since the log for small values of p tend to $-\infty$. Is this fine? –  Aug 22 '13 at 01:21
  • The expression $- p \log p / (1-p)$ seems correct to me. It is enough to say that this is not equal $p$ for all $p \in (0,1)$, which is obvious. However, it seems that in fact $-p \log(p) /(1-p)$ is $> p$ over $(0,1)$. Here is a link to a plot of $E[\hat{p}] / p$ suggesting this: http://wolfr.am/14kDWU9 – passerby51 Aug 22 '13 at 01:26
  • 1
    For clarification, the MLE of p is then a biased estimator of p? – kathystehl Apr 28 '15 at 18:00
  • 1
    @kathystehl, yes it is. – passerby51 May 01 '15 at 16:49