Testing for missing values in R

Question

I have a time series data set which has some missing values in it. I wish to impute the missing values but I am unsure as to which method is most appropriate e.g linear, spline or stine from the imputeTS package.

For the sake of completeness I wish to test whether my data is MCAR, MAR, NMAR. I've a fair idea it's MCAR but I'm interested to do the test.

str(wideRawDF)
'data.frame':   1343 obs. of  13 variables:
 $ Period.Start.Time: POSIXct, format: "2017-01-20 16:30:00" "2017-01-20 16:45:00" "2017-01-20 17:00:00" "2017-01-20 17:15:00" ...
 $ DO0182U09A3      : num  -102 -101 -101 -101 -101 ...
 $ DO0182U09B3      : num  -103.4 -102.8 -103.3 -95.9 -103 ...
 $ DO0182U09C3      : num  -103.9 -104.2 -103.9 -99.2 -104.1 ...
 $ DO0182U21A1      : num  -105 -105 -105 -104 -102 ...
 $ DO0182U21A2      : num  -105 -104 -105 -105 -105 ...
 $ DO0182U21A3      : num  -105 -105 -105 -105 -105 ...
 $ DO0182U21B1      : num  -102 -103 -104 -104 -104 ...
 $ DO0182U21B2      : num  -99.4 -102 -104 -101.4 -104.1 ...
 $ DO0182U21B3      : num  -104 -104 -104 -104 -104 ...
 $ DO0182U21C1      : num  -105 -105 -105 -104 -105 ...
 $ DO0182U21C2      : num  -104 -105 -105 -103 -105 ...
 $ DO0182U21C3      : num  -105 -105 -105 -105 -105 ...

md.pattern(wideRawDF)
     Period.Start.Time DO0182U21C1 DO0182U21C2 DO0182U21C3 DO0182U21B1 DO0182U21B2 DO0182U21B3 DO0182U09A3 DO0182U09B3 DO0182U09C3 DO0182U21A1 DO0182U21A2
1327                 1           1           1           1           1           1           1           1           1           1           1           1
   3                 1           1           1           1           1           1           1           0           1           1           1           1
   1                 1           1           1           1           1           1           1           1           0           1           1           1
   2                 1           1           1           1           1           1           1           1           1           0           1           1
   1                 1           1           1           1           1           1           1           0           1           1           0           0
   1                 1           1           1           1           1           1           1           0           0           1           0           0
   3                 1           1           1           1           1           1           1           1           0           0           0           0
   2                 1           1           1           1           1           1           1           0           0           0           0           0
   3                 1           1           1           1           0           0           0           1           0           0           0           0
                     0           0           0           0           3           3           3           7          10          10          10          10
     DO0182U21A3   
1327           1  0
   3           1  1
   1           1  1
   2           1  1
   1           0  4
   1           0  5
   3           0  5
   2           0  6
   3           0  8
              10 66

As you can see, some of the columns in my DF do not have NA values. I wish to pass only the columns which have NA to the TestMCARNormality function in the MissMech package.

I have tried the following but I keep getting the same error:

> TestMCARNormality(wideRawDF[,3:4])
Warning: 8 Cases with all variables missing have been removed 

          from the data.
Warning: More than one missing data pattern should be present. 

Using colnames I get the index of the columns which i reference to the above output of md.pattern to be certain that I am using columns with NA values.

> colnames(wideRawDF)
 [1] "Period.Start.Time" "DO0182U09A3"       "DO0182U09B3"       "DO0182U09C3"       "DO0182U21A1"       "DO0182U21A2"       "DO0182U21A3"       "DO0182U21B1"      
 [9] "DO0182U21B2"       "DO0182U21B3"       "DO0182U21C1"       "DO0182U21C2"       "DO0182U21C3"

What is the smart way to test for missing values and pass only the columns with NAs to the TestMCARNormality function?

Answer 1

As per comment, you can use the following:

has_na <- sapply(wideRawDF, function(x) any(is.na(x)))
TestMCARNormality(wideRawDF[has_na])

has_na is a boolean vector corresponding to each column of wideRawDF. It will be TRUE for any column that has at least one missing value in it.

Therefore, wideRawDF[has_na] is your data frame wideRawDF, but only the columns that have a missing value.