R parallel rbindlist does not work

Question

I have a function that uses rbindlist, and I would like to call that function using parLapply. Simplified version below:

func <- function(x){
  df1 <- data.frame(a = c(x,2), b = c(3,4))
  df2 <- data.frame(a = c(x,2), b = c(4,4))
  rbindlist(list(df1,df2))
}

cl <- makeCluster(getOption('cl.cores', detectCores()))
tmp <- parLapply(cl,c(1,2),func)
stopCluster(cl)

However, I get an error that says:

Error in checkForRemoteErrors(val) : 2 nodes produced errors; first error: could not find function "rbindlist"

Answer 1

When using other libraries with parlapply, make sure you properly load them on each node. you can do

clusterEvalQ(cl, library(data.table))

Before running your commands, or include

require(data.table)

in your function.