4

enter image description here

The above image is from wikipedia.

I'm having troubles with the third line to the 4th:

question 1):

How did they get from $2E[(\hat{\theta} - E(\hat{\theta})(E(\hat{\theta}) - \theta)]$ to $2(E(\hat{\theta}) - \theta)E(\hat{\theta} - E(\hat{\theta}))$? I thought $E(XY) = E(X)E(Y)$ if and only if $X,Y$ are independent random variables?

qeustion 2) Furthermore, why does $E(\hat{\theta} - E(\hat{\theta})) = E(\hat{\theta}) - E(\hat{\theta})$?

question 3) from the 4th to the 5th line they have for the last term:

$E[(E(\hat{\theta}) - \theta)^2]$ but that is not equal to $\text{Bias}(\hat{\theta},\theta)^2$. I have $\text{Bias}(\hat{\theta},\theta)^2 = (E(\hat{\theta}) - \theta)^2$?

Michael Hardy
  • 1
statsguy23
  • 43

1 Answers1

5

$\newcommand{\E}{\operatorname{E}}$The equality $\E(XY)=\E(X)\E(Y)$ holds if $X$ and $Y$ are independent and their expected values exist, but "only if" is wrong: generally this equality holds if $X$ and $Y$ are uncorrelated even if they are not independent. For example, let $X$ be $-1$, $0$, or $1$ each with equal probability and let $Y=X^2$; then that equality holds although $X$ and $Y$ are far from independent.

However, there is no need for that here. What is claimed is this $$ \E\Big( (\hat\theta - \E\hat\theta)\underbrace{(\E(\hat\theta)-\theta)}_{\text{a constant}} \Big) = \Big(\underbrace{\E(\hat\theta) - \theta}_{\begin{smallmatrix} \text{the same} \\ \text{constant} \end{smallmatrix}} \Big)\E\Big(\hat\theta-\E(\hat\theta)\Big). $$

Note well: the expression over the $\underbrace{\text{underbrace}}$ is a constant and may be pulled out of the expression $\E\Big(\cdots\cdots\Big)$. That was done.

I've edited the Wikipedia article somewhat to clarify this.

As for $\E(\hat{\theta} - \E(\hat{\theta})) = \E(\hat{\theta}) - \E(\hat{\theta})$, that is like $\E(\hat\theta-5) = \E(\hat\theta)-\E(5)=\E(\hat\theta)-5$. What is done with $5$ in this last sequence of equalities is done with another constant in the sequence of equalities that you ask about.

Finally, $\Big( \E(\hat\theta) -\theta \Big)^2$ is the square of the bias, and $\E\left( \Big( \E(\hat\theta) -\theta \Big)^2 \right)$, being the expected value of a constant, is the same thing.

Michael Hardy
  • 1