1

In the Fisher information, do we consider joint probability distribution. I have referred Thomas M. Cover's Elements of Information theory. The authors mention that -

$$FI(\theta) = E_{\theta}({\frac{\partial }{\partial \theta} {\ln f(X; \theta)})^2}$$

Does $f(X, \theta)$ refers to the joint probability distribution or the conditional probability distribution?

A rural reader
  • 2,507
nivedita
  • 11
  • Please clarify your specific problem or provide additional details to highlight exactly what you need. As it's currently written, it's hard to tell exactly what you're asking. – Community May 05 '23 at 14:01
  • $f(X; \theta)$ means the parametrised density of the random vector $X$ when the parameter is set equal to $\theta$ and the density is evaluated at $X$ (i.e., at the observed random quantity). For example, if you had a "normal model", you'd have densities parametrised with $(\mu, \sigma^2)$ and if a normal observation gave 1.2345, then $f(1.2345; (0,1))$ simply means the density of the normal distribution evaluated at 1.2345 when the mean is $0$ and variance is $1.$ – William M. May 05 '23 at 15:24
  • Ok, you mean to say it is not at all related to the conditional or the joint probability. The Wikipedia article (https://en.wikipedia.org/wiki/Fisher_information) got me further confused. Can you please share any reference which would clarify the basics in detail? – nivedita May 07 '23 at 03:48

0 Answers0