2

How to show the

$$ E(\Sigma_{MLE})=E\left(\dfrac{1}{n}\sum_{i=1}^{n}({x}_{i}-{\mu}_{MLE})({x}_{i}-\mathbf{{\mathbf{\mu}}}_{MLE})'\right)=\dfrac{n-1}{n}\Sigma $$

Where ${\mu}_{MLE}=\dfrac{1}{n}\sum_{i=1}^{n}{x}_{i}$ and ${x}_{i}\sim N(\mu,\Sigma)$ and $i=1,\ldots,n$

Here is what I've tried:

$$ E(\Sigma_{MLE})=E\left(\dfrac{1}{n}\sum_{i=1}^{n}({x}_{i}-{\mu}_{MLE})({x}_{i}-\mathbf{{\mathbf{\mu}}}_{MLE})'\right)=\dfrac{1}{n}E\left\{\sum_{i=1}^{n}(x_{i}x_{i}'-x_{i}\mu_{MLE}'-\mu_{MLE}x_{i}'-\mu_{MLE}\mu_{MLE}')\right\} $$

$$ =\dfrac{1}{n}\sum_{i=1}^{n}E(x_{i}x_{i}'-2x_{i}'\mu_{MLE}-\mu_{MLE}\mu_{MLE}') $$

$$ =\dfrac{1}{n}\sum_{i=1}^n \{E(x_{i}x_{i}')-2E(x_{i}'\mu_{MLE})-E(\mu_{MLE}\mu_{MLE}')\} $$

I think $E(x_{i}x_{i}')=\mu\mu'$ but don't know what is the $E(x_{i}'\mu_{MLE})$ and $E(\mu_{MLE}\mu_{MLE}')$

Ada
  • 179
  • 2
    Please, consider updating your question to include what you have tried and where you are getting stuck. That way, people on this site will know exactly what help you need. – Did Sep 23 '13 at 19:56
  • Updated the question to include what I've tried in response to comment – Ada Sep 23 '13 at 20:04
  • Notice that you need to bring in the True Mean $\mu$ somehow before you get to the final proof. Hint for this: Add and subtract $\mu$ before you expand the initial $(x_{n}-\mu_{ML})^{2}$ into separate terms. Then also here, $x=x_{n}^{\prime}$ since the Random Variable $X$ takes on only real values. – Sudarsan Sep 23 '13 at 21:55
  • Hint 2: Also you'd have to use the following fact somewhere down the line: $Var(\mu_{ML})=\frac{\Sigma^{2}}{n}$. – Sudarsan Sep 23 '13 at 22:01

1 Answers1

3

\begin{align} & \phantom{={}} \underbrace{\sum_{n=1}^N (x_n-\mu)(x_n-\mu)'}_{(0)} = \sum_{n=1}^N ((x_n-\mu_{ML})+(\mu_{ML}-\mu))((x_n-\mu_{ML})+(\mu_{ML}-\mu))' \\[12pt] & = \underbrace{\sum_{n=1}^N (x_n-\mu_{ML})(x_n-\mu_{ML})'}_{(1)} + \underbrace{\sum_{n=1}^N (x_n-\mu_{ML})(\mu_{ML}-\mu)}_{(2)} + \underbrace{\sum_{n=1}^N (\mu_{ML}-\mu)(x_n-\mu_{ML})'}_{(3)} \\[10pt] & {}\qquad{}+ \underbrace{\sum_{n=1}^N (\mu_{ML}-\mu)(\mu_{ML}-\mu)'}_{(4)}. \end{align} The expression $(0)$ has expected value $N\Sigma$ since each term has expected value $\Sigma$. The expression $(1)$ is something whose expectation you need to find. Each of $(2)$ and $(3)$ is $0$, since $\mu-\mu_{ML}$ is a factor that does not change as $n$ goes from $1$ to $N$ and the sum of the terms $x_n-\mu_{LM}$ is $0$.

So now think about the expected value of $(4)$. Notice that $(4)$ is a sum of $N$ terms that are all the same, so just find the expected value of one of them and multiply by $N$.

Michael Hardy
  • 1