2

The main purpose of this question is to check my understanding. As in I have an answer that I think is correct, but I'm not sure, since Stats is not my forte.

So given a random variable $X$, we take some samples from it and denote by $X_i$ the random variable "the $i$th observation from each sample" and denote by $\overline{X}$ the random variable the sample mean.

We're trying to show that: $$E\left[\frac{1}{n-1}\sum(X_i-\overline{X})^2\right] = \sigma^2 \tag 1$$ where $\operatorname{Var}(X)=\sigma^2$.

Let $Y_i = X_i-\overline{X}$ and since $E[X_i]=E[\overline{X}]$ we have that $E(Y_i)=0$.

This means that the LHS of (1) becomes:

$$ \begin{align} E\left[\frac{1}{n-1}\sum(Y_i^2)\right] & = \frac{1}{n-1}\sum E[Y_i^2] \\[8pt] & = \frac{1}{n-1}\sum (\operatorname{Var}(Y_i)-(E[Y_i])^2) \\[8pt] & = \frac{1}{n-1}\sum \operatorname{Var}(Y_i) \\[8pt] & =\frac{1}{n-1}\sum \operatorname{Var}(X_i-\overline{X}) \end{align} $$

Here is where I made the mistake of saying that $$\operatorname{Var}(X_i-\overline{X}) = \operatorname{Var}(X_i)+(-1)^2\operatorname{Var}(\overline{X}) = \sigma^2+\frac{\sigma^2}{n} = \sigma^2\frac{n+1}{n}$$

However, the proof I was following gave a longer calculation ending with $Var(Y_i)=\sigma^2\frac{n-1}{n}$ which then lead immediately to the desired result. So I wondered where I had gone wrong.

My educated guess is that it is not true that $\operatorname{Var}(X_i-\overline{X})=\operatorname{Var}(X_i)+(-1)^2\operatorname{Var}(\overline{X})$ because $\operatorname{Var}(X+Y) = \operatorname{Var}(x)+\operatorname{Var}(Y)$ only if the random variables are independent and $X_i$ and $\overline{X}$ are not.

Have I explained my error correctly or is there another mistake in my calculations?

Thank you, Diana

Michael Hardy
  • 1
Diana
  • 23

1 Answers1

2

$$ \operatorname{var}(X_i-\bar X) = \operatorname{var}(X_i) + \operatorname{var}(\bar X) - 2\operatorname{cov}(X_i,\bar X) = \sigma^2 + \frac{\sigma^2} n - 2\frac{\sigma^2} n = \frac{n-1}n\sigma^2. $$

The statement that $\operatorname{var}(X+Y) = \operatorname{var}(X)+\operatorname{var}(Y)$ ONLY if $X$ and $Y$ are independent is incorrect. It holds if they are independent, but not only if they are independent. In fact it holds if, and only if, they are uncorrelated. An example in which it holds when they are nowhere near independent is this: suppose $X$ is uniformly distributed on the set $\{-1,0,1\}$ and $Y=X^2$. Then $\operatorname{var}(X)=2/3$ and $\operatorname{var}(Y)=2/9$ and $\operatorname{var}(X+Y) = 2/3 + 2/9$, but clearly they are not independent.

Michael Hardy
  • 1