R data.table replace NA with mean for numeric columns and most frequent value for nominal values

Question

I've the following data.table

x = structure(list(id1 = c("a", "a", "a", "b", "b", NA), id2 = c(2, 3, NA,3, 4, 5)), .Names = c("id1", "id2"), row.names = c(NA, -6L), class = c("data.table", "data.frame"), .internal.selfref = <pointer: 0x1fe4a78>)

I'm trying to replace the NA in each column with separate strategies. For numeric columns I want to replace it with the mean and for factor or character columns I want to replace it with the most frequent value. I tried the following but it just does nothing.

for (j in 1:ncol(x)){
  if(is.numeric(unlist(x[,j,with=FALSE]))){
     m = mean(unlist(x[,j,with=FALSE]))
     set(x,which(is.na(x[[j]])),j,m)
   }else{
     m = sort(table(x),decreasing=TRUE)[[1]]
     set(x,which(is.na(x[[j]])),j,m)
}

Answer 1

Using base approaches, you can write a function like the following:

myFun <- function(x) {
  if (is.numeric(x)) {
    x[is.na(x)] <- mean(x, na.rm = TRUE)
    x
  } else {
    x[is.na(x)] <- names(which.max(table(x)))
    x
  }
}

... and apply it with:

x[, lapply(.SD, myFun)]
#    id1 id2
# 1:   a 2.0
# 2:   a 3.0
# 3:   a 3.4
# 4:   b 3.0
# 5:   b 4.0
# 6:   a 5.0

Note that which.max will take the first largest value in case there are ties.

---

I guess it could alternatively be written something like:

myFun <- function(inDT) {
  for (i in 1:ncol(inDT)) {
    temp <- unlist(inDT[, i, with = FALSE], use.names = FALSE)
    set(inDT, which(is.na(temp)), i, 
        if (is.numeric(temp)) {
          mean(temp, na.rm = TRUE) 
        } else {
          names(which.max(table(temp)))
        } )
  }
  inDT
}

y <- copy(x)

myFun(y)
#    id1 id2
# 1:   a 2.0
# 2:   a 3.0
# 3:   a 3.4
# 4:   b 3.0
# 5:   b 4.0
# 6:   a 5.0