1

http://www.real-statistics.com/statistics-tables/shapiro-wilk-table/

I am trying to build a function that performs a Shapiro-Wilk test on a sample population, but I cannot figure out how the a values in the Shapiro-Wilk weight table are calculated. I have been Googling and researching for almost 2 hours and cannot figure it out. Could someone please help me?

Thanks

myworkaccount
  • 93
  • I don't know about these things. In statistics they sometimes use weir language. But it seems to me that in the Wikipedia page they tell you the definition of all the elements of the test. – OR. Jan 02 '14 at 23:29
  • How to compute the expected values and the co-variance matrix of the order statistics of normally distributed variables can probably be found here: http://stats.stackexchange.com/questions/9001/approximate-order-statistics-for-normal-random-variables – OR. Jan 02 '14 at 23:38
  • It says "Table 2 contains the probability that the data corresponds to a normal distribution." I will be surprised if that's literally what Table 2 contains! – Michael Hardy Jan 03 '14 at 00:50
  • @ABC : The Wikipedia article does not say anything about how to find the probability distribution of the test statistic. – Michael Hardy Jan 03 '14 at 01:07
  • @ABC : The expected values of the order statistics of an i.i.d. sample from the normal distribution are needed to compute the test statistic, BUT that does not say anything about the probability distribution of the test statistic given that the null hyphothesis is true. That's what the table gives information about. So the answers are not found in the places you're suggesting. – Michael Hardy Jan 03 '14 at 01:09
  • @myworkaccount: I would try two things: (1) Look at the original paper of Shapiro and Wilk, which appeared in 1965; and (2) post this to statistics.stackexchange.com. – Michael Hardy Jan 03 '14 at 01:13

1 Answers1

2

It has taken me a few days to figure how to calculate the weight table, based on a Matlab function developed by A. Trujillo-Oriz et al. The results appear to agree with the table linked to some extent.

Suppose we have $N$ samples, and let us denote the weights as $w(n)$ for $n\in \{1,2,...,N\}$.

If $N=3$ we have three weights with value

$w(1)=\sqrt{0.5}, w(2)=0,w(3)=-\sqrt{0.5}$

If $N > 3$, we calculate N quantiles of the zero mean and unity variance Normal distribution, where $F^{-1}(\cdot)$ below denotes the quantile (or inverse CDF) function e.g. the NormInv() function as found in Excel:

$\large m(n)=F^{-1}\left(\frac{n-\frac{3}{8}}{N+0.25}\right)$ for $n\in {1,2,..,N}$

Next we calculate the following two coefficients:-

$\large c_N=\frac{m(N)}{\sqrt{\sum_{n=1}^Nm(n)^2}}$

$\large c_{N-1}=\frac{m(N-1)}{\sqrt{\sum_{n=1}^Nm(n)^2}}$

We create two vectors of polynomial coefficients (index starting at $0$, increasing from left to right)

$p1 = [-2.706056,4.434685,-2.071190,-0.147981,0.221157,c_N];$

$p2 = [-3.582633,5.682633,-1.752461,-0.293762,0.042981,c_{N-1}];$

Let $u=\frac{1}{\sqrt{N}}$

We set the last and first weights as

$w(N)=\sum_{k=0}^5p1(k)u^{5-k}$

$w(1)=-w(N)$

If $N=4,5$, we set

$\large \phi=\frac{(\sum_{n=1}^Nm(n)^2)-2m(N)^2}{1-2w(N)^2}$

and for $k\in \{2,...,N-1\}$

$\large w(k)=\frac{m(k)}{\sqrt{\phi}}$

If $N \geq 6$, we have

$w(N-1)=\sum_{k=0}^5p2(k)u^{5-k}$

$w(2)=-w(N-1)$

We set the following:-

$\large \phi=\frac{(\sum_{n=1}^Nm(n)^2)-2m(N)^2-2m(N-1)^2}{1-2w(N)^2-2w(N-1)^2}$

and for $k\in \{3,4,...,N-2\}$

$\large w(k)=\frac{m(k)}{\sqrt{\phi}}$

For more details, have a look at

http://scistatcalc.blogspot.co.uk/2013/10/shapiro-wilk-test-testing-for-normality.html

Alijah Ahmed
  • 11,609
  • Great description. Helped me a lot! Good to point out for newbies like myself is, that F^-1 is the NormInv() function found for example in Excel, and it is applied to the stuff in the brackets after it. It's not a constant. – Jens Jan 05 '16 at 12:54
  • I am glad you found the answer helpful- I have taken on board your point about $F^{-1}$ and have edited my answer accordingly. – Alijah Ahmed Jan 05 '16 at 21:38