Menu
I have a vector, in R, with 1521298 points, which have to be tested for normality. I chose the Shapiro-Wilk test, but the R function shapiro.test() says:
Error in shapiro.test(z_scores) : sample size must be between 3 and 5000
Do you know any other function to test it or how to circumvent this issue?
873
The Shapiro–Wilk test is more appropriate method for small sample sizes (<50 samples) although it can also be handling on larger sample size while Kolmogorov–Smirnov test is used for n ≥50. For both of the above tests, null hypothesis states that data are taken from normal distributed population.
The Shapiro-Wilk Test is more appropriate for small sample sizes (< 50 samples), but can also handle sample sizes as large as 2000. The normality tests are sensitive to sample sizes.
The null hypothesis for a Shapiro Wilk test is that there is no difference between your distribution and a normal distribution. The alternative hypothesis is that there is a difference. If your p value is less than 0.05, which it is, then you reject the null hypothesis and conclude that your data is nonormal.
Shapiro-Wilks Normality Test. The Shapiro-Wilks test for normality is one of three general normality tests designed to detect all departures from normality. It is comparable in power to the other two tests. The test rejects the hypothesis of normality when the p-value is less than or equal to 0.05.
Shapiro test cannot done using more than 5.000 records.
You can try to do the shapiro test using only the first 5.000 samples. IF it can help you, use the code like this:
shapiro.test(beaver2$temp[0:5000])
But pay attention, the test will use only the first 5.000 samples of your data.
In the other hand, if you need to use all the records of your sample, use another similar test, like Anderson-Darling normality test. You also can execute both and compare, like this script below:
# clean workspace
rm(list=ls())
# Install required packages:
install.packages('nortest')
#Model data tho use
ModelData = beaver2$temp
#Do shapiro test with only the first 5000 records
shapiro.test(ModelData[0:5000])$p.value
#Anderson-Darling normality test
library(nortest)
ad.test(ModelData)$p.value
You can try, Anderson-Darling normality test, which works for larger sample sizes.
library(nortest)
ad.test(data$variable)
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
