2

I know question on central limit theorem has been asked numerous times and I have searched around the website but still couldn't get what I wanted to find out (probably most people who asked the question is stuck at different parts of the theory).

Attached in my question is a lecture slide for the search for a transformation of x̅ that has a limiting distribution. I do not understand why x̅ and µ is multiply by √n instead of n. Also, why when the statistic √n(x̅ - µ) is used, µ becomes zero?

Thank you in advance!

enter image description here

Ken
  • 23
  • 3

1 Answers1

1

A previous slide defines $$ \bar{X} = \frac{X_1+\cdots+X_n}{\sqrt{n}}, $$ where the random variables $X_1,\ldots,X_n$ are independent copies of the same distribution, which has mean $\mathbb{E}[X_i] = \mu$ and variance $\mathbb{V}[X_i] = \sigma^2$. One calculates that the mean of $\bar{X}$ is also $\mu$, and its variance is $\sigma^2/n$.

We want to normalize $\bar{X}$ so that it has mean $0$ and variance $1$. If we subtract $\mu$ from $\bar{X}$ we obtain a random variable $\bar{X} - \mu$ with mean $0$ and the same variance. If we multiply the result by $\sqrt{n}/\sigma$ then the mean remains $0$ and the variance becomes $1$. The new random variable is $$ \tilde{X} = \frac{\sqrt{n}(\bar{X} - \mu)}{\sigma}. $$ It so happens that as $n\to\infty$, $\tilde{X}$ tends to a normal distribution $N(0,1)$. This is the central limit theorem.

In order for a theorem of this sort to hold, the mean and variance of $\tilde{X}$ have to be fixed. This is why we normalized it to have mean $0$ and variance $1$. We could have chosen other values and would have obtained an equivalent central limit theorem, but the choice of zero mean and unit variance is the conventional standard.

The slide tries to explain why other normalizations won't work:

  1. $\bar{X}$ has fixed mean but the variance tends to $0$. Therefore $\bar{X}$ tends to the constant $\mu$ distribution. This is the law of large numbers. The law tells us what $\bar{X}$ tends to, but not how the deviation from the mean behaves.

  2. $\sqrt{n} \bar{X}$ has fixed variance $\sigma^2$ but the mean tends to infinity (or minus infinity). So $\sqrt{n} \bar{X}$ cannot converge to any fixed distribution, unless $\mu = 0$. This is because if $\sqrt{n} \bar{X}$ tends to a distribution with mean $m$, then the mean of $\sqrt{n} \bar{X}$ must tend to $m$.

  3. $\sqrt{n}(\bar{X} - \mu)$ has mean $0$ and variance $\sigma^2$, and we can hope for a limit theorem. Indeed, the central limit theorem shows that $\sqrt{n}(\bar{X} - \mu)$ tends to $N(0,\sigma^2)$, a normal distribution with mean $0$ and variance $\sigma^2$. This is equivalent to the central limit theorem stated above.

Yuval Filmus
  • 57,157
  • Thank you. Your explanation is very thorough! :) I still have some questions:

    in 2) we can basically choose any combination value of µ and σ^2 but it's meaningless, hence we choose the standard normal values of mean o and variance 1 so that we can use the standard normal statistical table, right?

    also, I do not understand why x̅ = (X1+⋯+Xn) / √n and consequently, why σ^2/n can just become σ^2 when everything else is multiplied by √n? does my question make sense? why √n is used instead of just n?

     – Ken Apr 03 '15 at 05:29
  • You can choose whichever mean and variance you want, and it's meaningful. While it's true that standard "tables" (nowadays called library functions) usually work only for standard normals, it's not difficult to use them to analyze arbitrary normals. 2) When you add $n$ i.i.d. random variables, the order of magnitude of the error is the standard deviation, which is of order $\sqrt{n}$. This is a very basic fact that follows from the central limit theorem. 3) A calculation shows that the variance of $(X_1+\cdots+X_n)/\sqrt{n}$ is $\sigma^2$. Make sure that you know how to perform it.
    • – Yuval Filmus Apr 03 '15 at 06:15
  • We divide by $\sqrt{n}$ rather than by $n$ since we want the variance to be constant. If we divided by $n$, the variance would tend to zero and we wouldn't be able to see the errors. You can try it – add 100 numbers, each chosen uniformly at random among $1,-1$. If you divide by 100 and do a histogram (i.e., repeat the experiment many times and plot how many times each value was obtained) then the histogram will be very close to 0. Replace 100 by 1000, and it becomes even closer to 0. – Yuval Filmus Apr 03 '15 at 06:17
  • In contrast, if you divide by 10 (that is, $\sqrt{100}$) then you should see a Gaussian. You should see a very similar Gaussian if you replace 100 with 1000 and 10 with $\sqrt{1000}$. I encourage you to do this experiment and see for yourself. – Yuval Filmus Apr 03 '15 at 06:18
  • I got it!! thanks!!! – Ken Apr 03 '15 at 06:58