Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Error in shapiro.test : sample size must be between

I have a vector, in R, with 1521298 points, which have to be tested for normality. I chose the Shapiro-Wilk test, but the R function shapiro.test() says:

Error in shapiro.test(z_scores) : sample size must be between 3 and 5000

Do you know any other function to test it or how to circumvent this issue?

like image 873
Peter Pfand Avatar asked Jan 29 '15 14:01

Peter Pfand


People also ask

What should be the sample size for Shapiro test?

The Shapiro–Wilk test is more appropriate method for small sample sizes (<50 samples) although it can also be handling on larger sample size while Kolmogorov–Smirnov test is used for n ≥50. For both of the above tests, null hypothesis states that data are taken from normal distributed population.

Is Shapiro-Wilk test good for large sample size?

The Shapiro-Wilk Test is more appropriate for small sample sizes (< 50 samples), but can also handle sample sizes as large as 2000. The normality tests are sensitive to sample sizes.

When p-value is less than 0.05 in Shapiro-Wilk test?

The null hypothesis for a Shapiro Wilk test is that there is no difference between your distribution and a normal distribution. The alternative hypothesis is that there is a difference. If your p value is less than 0.05, which it is, then you reject the null hypothesis and conclude that your data is nonormal.

What should be the value of Shapiro-Wilk test?

Shapiro-Wilks Normality Test. The Shapiro-Wilks test for normality is one of three general normality tests designed to detect all departures from normality. It is comparable in power to the other two tests. The test rejects the hypothesis of normality when the p-value is less than or equal to 0.05.


2 Answers

Shapiro test cannot done using more than 5.000 records.

You can try to do the shapiro test using only the first 5.000 samples. IF it can help you, use the code like this:

shapiro.test(beaver2$temp[0:5000])

But pay attention, the test will use only the first 5.000 samples of your data.

In the other hand, if you need to use all the records of your sample, use another similar test, like Anderson-Darling normality test. You also can execute both and compare, like this script below:

# clean workspace
rm(list=ls())

# Install required packages:
install.packages('nortest')

#Model data tho use
ModelData = beaver2$temp

#Do shapiro test with only the first 5000 records
shapiro.test(ModelData[0:5000])$p.value

#Anderson-Darling normality test
library(nortest)
ad.test(ModelData)$p.value
like image 184
Wagner Cipriano Avatar answered Sep 25 '22 19:09

Wagner Cipriano


You can try, Anderson-Darling normality test, which works for larger sample sizes.

library(nortest)
ad.test(data$variable)
like image 28
VishnuVardhanA Avatar answered Sep 26 '22 19:09

VishnuVardhanA