5

Can $$I(\theta) = -E\left[\frac{\partial^2}{\partial\theta^2} \ln f(x,\theta)\right] = 0$$ I've heard that Fisher's Information is strictly greater than zero but I'm unsure why. A proof would be much appreciated!

rookie
  • 409
  • 5
  • 16

3 Answers3

5

The Fisher information is the variance of the score, given as $$ I(\theta) = \mathbb E\left[\left(\frac{\partial}{\partial\theta} \ln f(x\mid\theta)\right)^2\right], $$ which is nonnegative. For twice differentiable likelihoods, integration by parts yields the alternative formula given above, i.e., minus the expectation of the Hessian. For likelihoods that do not have two derivatives the alternative formula is not valid and can yield negative values. En example is the triangular distribution (continuous piecewise linear density $f$ on $[0,1]$, with $f(0)=f(1)=0$ and knot at $\theta\in(0,1)$) (see Berger, Bernardo, Sun, 2009).

Wicher
  • 51
2

Show that the definition you've given is the same as $$ I(\theta) = \mathbb E\left[\left(\frac{\partial}{\partial\theta} \ln f(x\mid\theta)\right)^2\right] $$ without the "$\ln$".

Michael Hardy
  • 1
0

From the Cramer-Rao bound, we know that variance of some unbiased estimator is lower bounded by reciprocal the fisher information. That is, $$Var(\hat{\theta})\geq \frac{1}{I(\theta)}.$$

Then we can get $I(\theta)\geq \frac{1}{Var(\hat{\theta})}$. Since the variance of some estimator should be $0<Var(\hat{\theta})<\infty$. We clearly have $0< I(\theta)< \infty$. Note that if $Var(\hat{\theta})\rightarrow\infty$, then we have $I(\theta)\rightarrow 0$.

aaronyxt
  • 95
  • 1
    By going from $\operatorname{Var}(\hat{\theta})\geq \frac{1}{I(\theta)}$ to $I(\theta) \geq \frac{1}{\operatorname{Var}(\hat{\theta})}$, you're already assuming that $I(\theta)>0$, so I don't think that this proof is valid. – Dylan Nov 26 '15 at 06:58
  • Well. I think you should review the definition of $I(\theta)$. That is, $I(\theta) = E[(\frac{\partial \log f(x|\theta)}{\partial \theta})^2|\theta]$. This definition always has $I(\theta)\geq 0$. – aaronyxt Nov 26 '15 at 13:28