Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Conditional row removal based on number of NA's within the row

Tags:

list

r

na

I am looking to remove rows from my dataset based on two conditions as follows:

  1. Remove row if 3 consecutive cells are NA or
  2. If four or more cells are NA

My sample data:

data <- rbind(c(1,1,2,3,4,2,3,2),
              c(NA,1, NA, 4,1,1,NA,2), 
              c(1,4,6,7,3,1,2,2), 
              c(NA,3, NA, 1,NA,2,NA,NA), 
              c(1,4, NA, NA,NA,4,3,2))

I have researched within the existing questions and found that na.omit or complete.cases can remove rows with NA but as I have conditions, doing further research I have found the following code within the existing questions:

data[! rowSums(is.na(data)) >4  , ]   
data[! rowSums(is.na(data)) ==3  , ]

The first line full fill my second condition. the second line does remove rows with three NA's but not looking for consecutive and removing any rows with total 3 NA's. for example:

> data
     [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8]
[1,]    1    1    2    3    4    2    3    2
[2,]   NA    1   NA    4    1    1   NA    2
[3,]    1    4    6    7    3    1    2    2
[4,]   NA    3   NA    1   NA    2   NA   NA
[5,]    1    4   NA   NA   NA    4    3    2

> data[! rowSums(is.na(data)) ==3  , ]
     [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8]
[1,]    1    1    2    3    4    2    3    2
[2,]    1    4    6    7    3    1    2    2
[3,]   NA    3   NA    1   NA    2   NA   NA

What I actually want is the 5th row to be removed only as this has three consecutive NA's and not the 2nd row.

Could anyone please advice me how can I overcome this?

like image 313
Achak Avatar asked Dec 20 '22 09:12

Achak


1 Answers

Both conditions at once:

data[!apply(is.na(data), 1, function(x) 
  {v <- cumsum(x); any(diff(v, 3) == 3) | 4 %in% v}), ]
#      [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8]
# [1,]    1    1    2    3    4    2    3    2
# [2,]   NA    1   NA    4    1    1   NA    2
# [3,]    1    4    6    7    3    1    2    2

any(diff(v, 3) == 3) is TRUE if there were NA three times in a row (so the difference somewhere is 3) and 4 %in% v corresponds to the second condition.

like image 152
Julius Vainora Avatar answered Jan 30 '23 21:01

Julius Vainora