I'm trying to prove Rao-Cramer lower bound from scratch. Here's my attemption:
It suffice to consider single sample case: $X$ sampled from distribution with p.d.f $p(x;\theta)$. And $f$ is an estimator with
$$\int f(x) p(x;\theta) dx=\theta$$
holds for every $\theta$.
And w.l.o.g again we consider the case $\theta=0$, namely we need to minimize
$$\int f^2(x) p(x;0) dx$$.
this looks like a harmless quadratic programming problem with linear constrains. So we have lagrangian $$L=\int f^2(x) p(x;0) dx-\int\lambda(\theta)(\int f(x) p(x;\theta) dx-\theta)d\theta$$.
Differentiating against $f$ and $\lambda$ we have:
$$2f(x)p(x;0)=\int\lambda(\theta)p(x;\theta)d\theta$$ for every $x$ and $$\int f(x) p(x;\theta) dx=\theta$$ for every $\theta$, then I got stuck here, is it possible to get Rao-Cramer bound following this strategy? And do we always have Rao-Cramer bound euqal to the optimum for the proposed optimization problem?
