1

I am trying to track abnormal values in a dataset over a period of time. Currently, I am using z-scores and the 68-95-99.7 rule for all datasets that are normally distributed. Unfortunately, a small number of these datasets are not normally distributed, and I need a different strategy.

After reading this reddit post, it seems like working with z-scores is still viable as long as I remember that "since [the] distribution is skewed right, the distribution of z-scores will be skewed right". So, it looks like I need to do two things:

  1. measure the skewness of the distribution
  2. adjust the z-score to normalize for this skewness

Anyways, I was wondering if there was a better approach for using z-scores with abnormal distributions. If not, what measures of skewness would be best used for this formula, and in what way could the z-score be normalized with this in mind?

Thanks for your help,

foxygen
  • 111
  • 1
    What are you using $z$ scores for? If you have the data, could you not just look at the $0.68$, $0.975$ and $0.99$ quantiles of your data without having to standardise? – Henry May 13 '14 at 12:20
  • z scores allow me to test for abnormal values (based on the quantiles above) without saving the entire dataset (I just save the previous sample size, sum of data, and sum of squares). I'm not sure how (or if it's possible) to calculate what values fall within the quantiles on a rolling basis in the same way I am doing with the z scores. If someone could show me how to do this (calculate rolling percentiles), that would solve my problem – foxygen May 13 '14 at 13:31

0 Answers0