4

Say I have a mean and standard deviation for a dataset of 5 elements.

I now add a sixth element. Is there a way to calculate the new mean and standard deviation using the information we had prior (i.e. not just recalculating the whole thing from scratch)?

For the mean, I see that I can just multiply the old one by $5$, add the new element, and divide by $6$.

I'm not sure if there's something I can do with the standard deviation, however.

$$\sigma_{old} = \sqrt{\sum_i (X_i - \mu_{old})^2}$$

$$\sigma_{new} = \sqrt{\sum_i (X_i - \mu_{new})^2 + (X_{new} - \mu_{new})^2}$$

$$\mu_{new} = \frac{\mu_{old}*N + X_{new}}{N+1}$$

$$\sigma^2_{new} = \sigma^2_{old} + \sum_i \left( (X_i - \mu_{new})^2 - (X_i - \mu_{old})^2 \right) + (X_{new} - \mu_{new})^2$$

After putting it in terms of the old stats, this becomes (I think)

$$\sigma^2_{new} = \sigma^2_{old} + \sum_i \left(2 X_i + \frac{(2N+1) \mu_{old} + X_{new}}{N+1} \right) \left(\frac{X_{new} - \mu_{old}}{N+1}\right) + (X_{new} - \frac{\mu_{old}*N + X_{new}}{N+1})^2$$

Is there anything better than this monstrosity?

MT_
  • 19,603
  • 9
  • 40
  • 81

2 Answers2

2

Let's say you started with n points and have added an $(n+1)^{st}$. To handle the variance write $\mu_{new} = \mu_{old} + \delta$ . We see that we need to compute $$\sum_{i=1}^{n} (x_i - \mu_{new})^2$$ Where the sum is just taken over the old $x_i$'s (the contribution from the $(n+1)^{st}$ sample being easily incorporated. But $$(x_i - \mu_{new})^2 = (x_i - \mu_{old} - \delta)^2$$ So our sum becomes $$\sum_{i=1}^{n} (x_i - \mu_{new})^2 = \sum_{i}^{n} (x_i - \mu_{old})^2 - 2 \delta \sum_{1}^{n} (x_i - \mu_{old}) + n \delta^2 = \sum_{i}^{n} (x_i - \mu_{old})^2 + n \delta^2$$ Where the middle sum vanishes as the old x's sum to the old mean.

Combining all this (and trusting that no algebraic error has been made!) we see that $$Var_{new} = \frac{(x_{n+1}-\mu_{new})^2}{n+1}+ \frac{n}{n+1}Var_{old} + \frac{n}{n+1}\delta^2$$

Not too terrible!

lulu
  • 70,402
1

If $\mu_n = \frac1n \sum_{i=1}^n x_i$, $$\mu_{n+1} = \frac{1}{n+1}\sum_{i=1}^{n+1}x_i = \frac{1}{n+1}\big[x_{n+1} + \sum_{i=1}^nx_i\big] = \frac{x_{n+1}}{n+1} + \frac{n}{n+1}\mu_n$$

As you stated, the running sample standard deviation is much trickier. Check this link out: Incremental Mean and Standard Deviation Calculation

They provide derivations for the following incremental variance formula: $$\sigma_{n+1}^2 = \frac{1}{n+1}\big[n\sigma_n^2 + n(n-1)(\mu_{n+1} - \mu_n)^2\big]$$

jameselmore
  • 5,207