1

I kinda doing some review questions for my finals and I kinda got stuck on this question.

a busy cat

I'm able to do part a by finding the maximum likelihood estimator but for some reason.

To find the variance I used $Var(\theta)= E(\theta^2)-[E(\theta)]^2$

However given how the estimator of theta is written in b) I'm not sure how to find E$(\theta)$.

F.A.
  • 1,001

1 Answers1

1

Hint: Find the information $I(\theta_0)$ for each estimator $\theta_0$. Then the asymptotic variance is defined as $$\frac{1}{nI(\theta_0 \mid n=1)}\,$$ for large enough $n$ (i.e., becomes more accurate as $n \to \infty$). Recall the definition of the Fisher information of an estimator $\theta$ given a density (probability law) $f$ for a random observation $X$: $$I(\theta):=\mathbb{E}\left(\frac{\partial}{\partial \theta}\text{log}f(X \mid \theta)\right)^2 \,.$$ Under certain regularity conditions (which apply here), the information can be shown to be given by (this is much easier to compute than the above) $$I(\theta)=-\mathbb{E}\left(\frac{\partial^2}{\partial \theta^2}\text{log}f(X \mid \theta)\right) \,.$$ Let us begin the computation for part (b). If you have computed the maximum likelihood estimator from part (a), then you should know that the log-likelihood function (for a single observation) is given by $$\ell(\theta)=\log f(X \mid \theta) = \log\left(\frac{1}{\theta}\exp\left(-\frac{X}{\theta}\right)\right)=-\log(\theta)-\frac{X}{\theta} \,.$$ Now differentiating twice with respect to $\theta$, we obtain $$\ell \,'(\theta)=-\frac{1}{\theta}+\frac{X}{\theta^{\,2}} \implies \ell \,''(\theta)=\frac{1}{\theta^{\,2}}-2\frac{X}{\theta^{\,3}}\,.$$ Now, as an aside, note that with $n=1$, we can write $$\tilde{\theta} = \sqrt{\frac{X^2}{2}} = \frac{X}{\sqrt{2}}$$ where the second equality comes from the fact that $X \geq 0$. Substituting $\theta=\tilde{\theta}$ (given $n=1$) and assuming $X \neq 0$, we have $$\begin{align} \ell \,''(\tilde{\theta})&=\frac{2}{X^2}-2\frac{X}{\left(\frac{X}{\sqrt{2}}\right)^3} \\&= \frac{2}{X^2}-\frac{4\sqrt{2}}{X^2} \\&=2\frac{1-2\sqrt{2}}{X^2}\,. \end{align}$$ Thus, the information is given by $$I(\tilde{\theta} \mid n=1) = -\mathbb{E}\left(2\frac{1-2\sqrt{2}}{X^2}\right)=2(2\sqrt{2} - 1)\cdot \mathbb{E}\left(X^{-2}\right) \,.$$ It remains to compute the expectation of $X^{-2}$ (allowing you to obtain the information and subsequently determine the asymptotic variance), which I will leave to you.

Michael Hardy
  • 1
afedder
  • 2,102
  • Yeah I I kinda know about the fisher information for a MLE. Part a isn't really my problem but could I use it for part b? –  May 29 '14 at 04:21
  • You need the Fisher information for both the maximum likelihood estimator $\hat{\theta}$ and the estimator given in part (b) $\tilde{\theta}$ to compute the asymptotic variance in both cases. @user131516 – afedder May 29 '14 at 04:26
  • ah okay, then I should be able to solve it I think. Is there a condition which it is not possible to use the fisher information to find the asymp variance? And what are the regularity conditions? –  May 29 '14 at 04:43
  • No, this is the definition of the asymptotic variance (especially in all but very few instances in earlier courses in probability). You should assume this is what is meant by asymptotic variance unless it is explicitly defined in some other way. – afedder May 29 '14 at 04:45
  • In terms of the regularity conditions, they are complicated - like I said, pretty much all cases you would come across reduce to the second formula for the information. @user131516 – afedder May 29 '14 at 04:48
  • Yeah, after 3 years of doing econometrics this is the first time I've seen the fisher information –  May 29 '14 at 04:51
  • One obvious regularity condition is that the density is continuous and has continuous first and second order partial derivatives @user131516 ...not a problem, but the Fisher information is very important in variances of estimators (it is also used for other topics in statistics), you might also want to read up on the Cramér–Rao lower bound for another application of the information with these estimators. – afedder May 29 '14 at 04:57
  • I'm familiar with Cramer-Rao, but i think i might be having some problems with part b) again. –  May 29 '14 at 05:06
  • But in part b isn't it the Sum of X^2? I think that is my main issue especially with the square root of Sum (X^2) with n observations. –  May 29 '14 at 05:26
  • Yes, but to compute the information, assume that $n=1$... – afedder May 29 '14 at 05:27
  • The information is given as the expectation of the second partial derivative of $\text{log}f(X ,|, \theta)$, not that of $\text{log}f(X_1,X_2,...,X_n ,|, \theta)$. – afedder May 29 '14 at 05:29
  • Why do we assume n=1 to compute the information? Sorry if I'm asking too many question –  May 29 '14 at 05:31
  • You can compute it either way, but then there is no need for the factor $n$ in the denominator of the asymptotic variance if you use $n$ observations in computing the Fisher information – afedder May 29 '14 at 05:31
  • Usually, the Fisher information is for a single observation, unless explicitly asked otherwise. However, it really depends on the convention used in your context. – afedder May 29 '14 at 05:32
  • If you want to compute it with $n$ observations, then substitute $$\theta = \tilde{\theta} = \sqrt{\frac{\sum_{i=1}^{n} X_i^2}{2n}}$$ into the likelihood $f(X_1,X_2,...,X_n ,|, \theta)$. – afedder May 29 '14 at 05:37
  • I deleted the other suggested substitution because it seems from this context that we should calculate the information for $n$ observations...please use the above substitution – afedder May 29 '14 at 05:42
  • However, you can do as I said before if it is easier by substituting $$\theta = \tilde{\theta}_{n=1} = \sqrt{\frac{X^2}{2}} = \frac{|X|}{\sqrt{2}} = \frac{X}{\sqrt{2}},,$$ where the last equality assumes $X$ is nonnegative. – afedder May 29 '14 at 05:43
  • I think this substitution will be easier for me to handle. So...either substitution is fine? –  May 29 '14 at 11:46
  • Either way I'm still stuck. I differentiate log f(x|theta) twice and then substitute in the estimator. This is where i got stuck. –  May 29 '14 at 12:14
  • I added part of the solution for part (b), let me know if this helped! @user131516 – afedder May 30 '14 at 17:22
  • It really helped! but I'm stuck trying to compute E(X^-2). I 'm sorry if this is taking really long but what I did was integrate X^-2*f(x) from x=0 to infinity but I can't get it to converge. –  May 31 '14 at 06:14
  • @user131516 try integrating by parts – afedder Jun 01 '14 at 04:35