5

How do we calculate $$ \begin{align}E[S]&=E\left[\sqrt{\frac{1}{n-1}\sum_i(x_i-\bar{x})^2}\right]\\&=E\left[\sqrt{\frac{1}{n-1}} \sqrt{\sum_ix_i^2-n\bar{x}^2}\right]. \end{align}$$

I am asking this question out of curiosity mostly because it is easy to show that $E[S^2]=\sigma^2$ but not so easy to calculate this.

Michael Hardy
  • 1
matt
  • 876
  • what are you asking? Do you want to know why $\sum (x_i-\bar{x})^2=\sum x^2_i-n\bar{x}^2$? – MAN-MADE Sep 01 '17 at 05:24
  • No I am asking how to calculate or simplify $E[S]$ – matt Sep 01 '17 at 05:26
  • I want to know that too! But what I know $(S^2-\sigma^2)\Rightarrow N(0, V)$, and $\sqrt{x}$ continuous increasing for $x>0$, then $(\sqrt{S^2}-\sqrt{\sigma^2})\Rightarrow N(0, V_1)$. "$\Rightarrow$" mean weakly convergence. (I forgot this statement, ha ha. I might be wrong!) – MAN-MADE Sep 01 '17 at 05:41
  • @MANMAID : Limits as $n\to\infty$ are not needed here. One can compute an integral. If no on answers this by $12$ hours from now, remind me and then maybe I'll post something. – Michael Hardy Sep 01 '17 at 05:51
  • ok, The short answer: $$ \frac{(n-1)S^2}{\sigma^2} \sim \chi^2_{n-1}. $$ So find $$ \int_0^\infty \sqrt x f(x), dx $$ where $f$ is the chi-square density. This comes down to the sort of integral that defines the Gamma function. Then use standard Gamma-function identities. – Michael Hardy Sep 01 '17 at 05:53
  • @MichaelHardy is this also true for $x_i$ are not sample from normal distribution? (I am curious!) – MAN-MADE Sep 01 '17 at 06:23
  • @MANMAID : I don't think so. Maybe the simplest example to try is a Bernoulli distribution. If it doesn't work there, then you can't drop the assumption of normality. To what extent you can weaken the assumption I don't know. – Michael Hardy Sep 01 '17 at 17:46

2 Answers2

6

When sampling from a normal population, the sample SD calculated from $n$ independent observations has expected value $$E(S)=\sigma \cdot \sqrt{ \frac{2}{n-1} } \cdot \frac{ \Gamma(n/2) }{ \Gamma( \frac{n-1}{2} ) },$$ where $\sigma$ is the population standard deviation. For non-normal populations it's hard to find an explicit formula for $E(S)$. But you can prove that for any population, the sample SD tends to underestimate the population SD.

See this article on stats.stackexchange.com for details.

grand_chat
  • 38,951
  • nice! But can you please clarify what Michael Hardy said (The short answer). Did he implied only for $x_i$'s are rs from normal distribution or it is true in general? (assuming $E(X^2)<\infty$ I guess) – MAN-MADE Sep 01 '17 at 06:48
  • 1
    @MANMAID The approach that Michael Hardy outlined is valid only for the normal distribution. It's the same approach as detailed in the stats.stackexchange article. – grand_chat Sep 01 '17 at 06:50
  • You haven't included the standard way to simplify that quotient of Gamma functions. – Michael Hardy Sep 01 '17 at 17:44
  • @MichaelHardy I'm not familiar with that simplification. Feel free to edit! – grand_chat Sep 01 '17 at 19:07
4

$\newcommand{\e}{\operatorname{E}}$I've up-voted the answer by "grand_chat", but it omits some things, including (in the answer's present form) how to simplify the quotient of Gamma functions.

With any distribution with finite variance, you have $\e(S^2) = \sigma^2.$ But if I'm not mistaken, $\e(S)$ depends on the shape of the distribution. Maybe I'll say more about that later. I might start by looking at what it is for a Bernoulli distribution with $n=1,2,3.$

But assuming an i.i.d. sample from a normal distribution, and that it's already established that $$ (n-1) \frac{S^2}{\sigma^2} \sim \chi^2_{n-1}, $$ let us first recall the chi-square density (with $n-1$ degrees of freedom): $$ f(x)\,dx = \frac 1 {\Gamma(\frac{n-1} 2)} \left(\frac x 2\right)^{(n-3)/2} e^{-x/2} \, \left(\frac{dx} 2\right) \text{ for } x\ge0. $$ Therefore \begin{align} \e\left( \sqrt{n-1} \, \frac S \sigma \right)& = \int_0^\infty \sqrt x f(x) \,dx = \frac 1 {\Gamma\left( \frac{n-1} 2 \right)} \sqrt 2 \int_0^\infty \sqrt{\frac x 2 } \left( \frac x 2 \right)^{(n-3)/2} e^{-x/2} \, \left( \frac {dx} 2 \right) \\[10pt] & = \frac {\sqrt 2} {\Gamma\left( \frac{n-1} 2 \right)} \int_0^\infty \left( \frac x 2 \right)^{(n/2)-1} e^{-x/2} \, \left( \frac {dx} 2\right) = \frac{\sqrt 2}{\Gamma\left( \frac{n-1} 2 \right)} \cdot \Gamma \left( \frac n 2 \right) \\[15pt] \text{Therefore } \e S & = \sigma \sqrt{\frac 2 {n-1}} \cdot \frac{\Gamma\left( \frac n 2\right)}{\Gamma\left( \frac{n-1} 2 \right)}. \end{align}

I dislike piecewise definitions, but here I don't know off-hand any way (but maybe somebody does?) to avoid looking at two cases:

First assume $n$ is odd:

Then $$ \Gamma\left( \frac n 2 \right) = \frac{n-2}2\cdot\frac{n-4}2\cdots\cdots \frac 1 2 \Gamma\left( \frac 1 2 \right) = \frac{(n-1)!}{2^{n-1} \left( \frac{n-1} 2 \right)!} \Gamma\left( \frac 1 2 \right) = \frac{(n-1)!}{2^{n-1} \left( \frac{n-1} 2 \right)!} \sqrt \pi. $$ Then in the quotient of two Gamma functions above you will have $\displaystyle \frac{(n-1)!}{\left(\left( \frac{n-1} 2 \right)!\right)^2} = \binom { n-1 }{ \frac{n-1} 2 }.$

Do a similar thing if $n$ is even, working first on $\displaystyle\Gamma\left( \frac {n-1} 2\right).$

Michael Hardy
  • 1