Questions tagged [fisher-information]

For question about fisher information that appears in mathematical statistics.

The Fisher information is a way of measuring the amount of information that an observable random variable $X$ carries about an unknown parameter θ upon which the probability of $X$ depends. Let $f(X; \theta)$ be the probability density function (or probability mass function) for $X$ conditional on the value of $\theta$. This is also the likelihood function for $\theta$. It describes the probability that we observe a given sample $X$, given a known value of $\theta$. If $f$ is sharply peaked with respect to changes in $\theta$, it is easy to indicate the “correct” value of $\theta$ from the data, or equivalently, that the data $X$ provides a lot of information about the parameter $\theta$. If the likelihood $f$ is flat and spread-out, then it would take many, many samples like $X$ to estimate the actual “true” value of $\theta$ that would be obtained using the entire population being sampled. This suggests studying some kind of variance with respect to $\theta$.

$$I(\theta) = E\left[\left(\frac{\partial}{\partial \theta} \log f(X; \theta \right)^2|\theta\right]$$

191 questions
1
vote
0 answers

Fisher Information

In the Fisher information, do we consider joint probability distribution. I have referred Thomas M. Cover's Elements of Information theory. The authors mention that - $$FI(\theta) = E_{\theta}({\frac{\partial }{\partial \theta} {\ln f(X;…
1
vote
1 answer

Expectation of score function (partial derivative of the log-likelihood function)

according to the Wikipedia: https://en.wikipedia.org/wiki/Score_(statistics), expected value of a score function should equals to zero and the proof is following: \begin{equation} \begin{aligned} \mathbb{E}\left\{ \frac{ \partial }{ \partial \beta }…
0
votes
0 answers

Riccati transformation about fisher-information

my question is about, how to find the Substitute term f(x)=sqrt(g(x)/g(0)), with g(0)=1 I(μ) =∫ (g′(x))^2/g(x)) dx. (Fisher-Information) How did he finde f(x) ? thanks everyone
0
votes
1 answer

Fisher information of normal distribution where the mean is separated into two components

Say we have a normally distributed variable $X \sim \mathcal{N}(\mu, \sigma^2)$. The Fisher information for $\mu$ is $\mathcal{I}(\mu) = \frac{1}{\sigma^2}$. But if the variable is distributed as $X \sim \mathcal{N}(\alpha - \beta, \sigma^2)$, (and…
J. Doe
  • 249
0
votes
0 answers

Minimizing Fisher information with absolute moment constraints

Hello dear community, I apologize for my english, but would like to ask how to show something like this? the fisher-information is defined the convexity is shown a hyperplane is created here we assume to have a density that minimizes the fisher…
0
votes
1 answer

Fisher information matrix for normal distribution

The below is captured from my lecture note, for the third column of first and second row and for the third row of the first and second column, is it because the summation of $x_i - \alpha - Bz_i$ equal zero so that these four entries equal to…
Thomas
  • 173
0
votes
0 answers

How does one usually evaluate the expected value of observed Fisher information?

How does one usually evaluate the expected value of observed Fisher information? That is what does $$\mathcal{I}(\theta)=-E\left[\frac{\partial^2}{\partial\theta^2}l(X,\theta)\mid\theta\right]$$ evaluate to? Particularly, how is $E$ treated? How…
mavavilj
  • 7,270
0
votes
1 answer

Fisher information of sample

Definition of single random variable Fisher Information is $$I(\theta)=E_\theta((\frac{\partial}{\partial \theta} ln f(x,\theta))^2)$$ and we get that $$E_\theta((\frac{\partial}{\partial \theta} ln…