How do you "guess" a candidate for an UMVUE?

Question

The following distribution is given $X \sim \operatorname{BIN}(1, p)$, then $\mathbb{E}(X)=p$ and $\mathbb{V} \operatorname{ar}(X)=p(1-p)$. The CRLB is computed with:

$$ \begin{aligned} f(x ; p) &=p^x(1-p)^{1-x} \\ \ln f(x ; p) &=x \ln p+(1-x) \ln (1-p) \\ \frac{\partial}{\partial p} \ln f(x ; p) &=\frac{x}{p}-\frac{1-x}{1-p}=\frac{x-p}{p(1-p)} \\ \mathbb{E}\left(\frac{\partial}{\partial p} \ln f(X ; p)\right)^2 &=\mathbb{E}\left(\frac{X-p}{p(1-p)}\right)^2=\frac{\mathbb{E}(X-p)^2}{p^2(1-p)^2}=\frac{\operatorname{Var}(X)}{p^2(1-p)^2}=\frac{1}{p(1-p)} \end{aligned} $$ The CRLB is now obtained as $$ \frac{\left[\tau^{\prime}(p)\right]^2}{n \mathbb{E}\left(\frac{\partial}{\partial p} \ln f(X ; p)\right)^2}=\frac{1}{\frac{n}{p(1-p)}}=\frac{p(1-p)}{n} $$

The subsequent task is to find the UMVUE. Algebraically I know how to check wheter an estimator is an UMVUE, but I find it hard to do this the "other way around", so to come up with an UMVUE yourself. The following is a snippet from the solution manual.

Looking at your answer for part (a) you should recognize that the CRLB coincides with $\operatorname{Var}(X) / n$. As an educated guess we therefore try $\hat{p}=\bar{X}$. First, from $\mathbb{E}(X)=p$

The "educated guess" part makes me uncomfortable, because there are so many possebilities to choose from. How do you know that you pick the right one? Feedback on this would be very much appreciated.

Questions:

• Why is $p$ an educated guess?
• How to make an educated guess yourself?
• Do you have general advice to come up with an UMVUE yourself?

Answer 1

1. The educated guess comes from the fact that in general, for i.i.d. random variables $X_1,\dots,X_n$, the variance of the sample mean $\bar{X}$ is always $\text{Var}(X_1)/n$. So you have shown that in this case, when $X_1 \sim \text{Bernoulli}(p)$, the CRLB is $p(1-p)/n = \text{Var}(X_1)/n$.

2. see above.

3. The Lehmann-Scheffe theorem tells us that if $T$ is a complete sufficient statistic for a parameter $\theta$, and $W$ is any unbiased estimator of $\tau(\theta)$, the $\hat{\tau}(T) = E[W|T]$ is the UMVUE of $\tau(\theta)$.

In other words, if you have (any) unbiased estimator of $\tau(\theta)$, and you know a statistic that is complete and sufficient for $\theta$, then $E[W|T]$ is a UMVUE.

Here is a nice example of this result. Let $X_1,\dots,X_n$ be i.i.d. Bernoulli with parameter $\theta$. Find the UMVUE of $\tau(\theta) = \theta^2$.

First, show that $T = \sum_i X_i$ is complete and sufficient for $\theta$. Then, we need any unbiased estimator for $\theta^2$. It is straight forward to take $W = X_1X_2$. Obviously $E(W) = \theta^2$. Finally, let $\hat{\tau}=E[X_1 X_2|T=t]=\frac{t(t-1)}{n(n-1)}$