0

Say I have the following list of numbers, each corresponding to a single day:

list = [1,   6,    7,    12,   5]

I can calculate the daily, rolling standard deviations between each as:

stds = [NaN, 3.54, 0.71, 3.54, 4.95]

Is it now possible to calculate the standard deviation across all values of list, using only the values of stds?

KOB
  • 293
  • Not sure I understand the values in stds. The standard deviation of the values $1,2$ is not $3.54$, whether you divide by $n$ or by $n-1$. What exactly is the $3.54$? Also...regardless of what your numbers mean, why isn't the last term in stds the desired value for the entire list? – lulu Jan 16 '19 at 13:09
  • @lulu Sorry, the second number was supposed to be 6, not 2. stds contains the standard deviations between each successive pair of numbers. The first value is blank, since there is no number before 1. The second value is std(1, 6), the third value is std(6, 7), etc. – KOB Jan 16 '19 at 13:17
  • Ok, but then the answer is "obviously not". The list $[1,6,7,12,19]$ has the same pairwise standard deviations but not the same total standard deviation. – lulu Jan 16 '19 at 13:22
  • Note that you can calculate standard deviation incrementally using just three stored values: the number of data observed, the sum of the data, and the sum of their squares. This seems to me a much more useful trick, since storing the list of "daily rolling standard deviations" takes about the same resources as storing the numbers themselves. – David K Jan 17 '19 at 06:11
  • @DavidK unfortunately the only data I receive is the pairwise standard deviations, and this cannot be changed as of now. – KOB Jan 17 '19 at 07:26

1 Answers1

1

The rolling pairwise standard deviations don't give you enough information to get the total standard deviation. The two lists $$[0,1,0,1]\quad \&\quad [0,1,2,3]$$ both have the pairwise standard deviations $$\Big[\frac 1{\sqrt 2},\,\frac 1{\sqrt 2},\,\frac 1{\sqrt 2}\Big]$$

But the total standard deviation of the first is $.577$ and the standard deviation of the second is $1.29$.

Note: in all cases we are using the sample standard deviation (i.e. we divide by $n-1$ not $n$).

lulu
  • 70,402