6

According to the central limit theorem, if one takes random samples of size $n$ from a population of mean $\mu$ and standard deviation $\sigma$, then as $X$ gets large, $X$ approaches the normal distribution with mean $\mu$ and standard deviation $\frac{\sigma}{\sqrt{n}}$.

$\frac{\sigma}{\sqrt{n}}$ doesn't make sense to me. Lets look at the extreme case. Say my sample consists of the entire population. Then, shouldn't my standard deviation be just $ \sigma$ instead of $\sigma/\text{(population size)}$?

David Faux
  • 3,425
  • 1
    If we are sampling independently (so with finite populations, with replacement) then the variance of the sample mean is $\sigma^2/n$. Nothing to do with near normality. In the case of finite population, and sampling without replacement, the variance is $\le \sigma^2/n$. – André Nicolas Oct 04 '13 at 22:08
  • Yeah, but you must understand that $X=\frac{X_1+X_2+...+X_n}{n}$, such that every $X_i$ is just a random sample. And when we talk about the population we refer to the distribution of every single $X_i$ with mean $u$ and standard deviation $\sigma$. – maenju Apr 21 '22 at 19:50

2 Answers2

4

If $X$ represents here the sample mean $\bar X_n$, then the Central Limit Theorem says that the quantity

$$Z = \sqrt n(\bar X_n-\mu)$$ tends in distribution to $N(0,\sigma^2)$ as $n$ tends to infinity, and then by abusing notation and asymptotics, we write

$$ \bar X_n = \frac{1}{\sqrt n}Z + \mu$$ which gives us that $\bar X_n \approx N(\mu,( \frac{\sigma}{\sqrt n})^2) $.

...which in a sense holds for some "intermediate range" of $n$ - because if $n$ truly passes over to infinity, then the distribution collapses to a single point, since the variance goes to zero (which is as it should).

Alecos Papadopoulos
  • 10,388
  • I appreciate that you write, "and then by abusing notation and asymptotics, we write..." Your answer would be improved if you added a section on how to get from $a$ to $b$ without abusing notation or asymptotics. – Michael Levy Feb 12 '24 at 20:12
2

Your $X$ is more commonly denoted ${\bar X}$, because it is the average of all observations. And intuitively, it seems likely that the average of many observations is closer to the real average $\mu$ of the population than the average of few observations. That is (part of) what the central limit theorem says: that you're more sure to be near the real average when you take more samples.

Also note that if your sample consists of the entire population, then the central limit theorem doesn't apply (because you didn't pick your samples independently). But still, if you were to take so many independent samples that you'd have as many samples as your population is large, then you'd very likely be very close to the real population average.

Magdiragdag
  • 15,049