mc.cores > 1 is not support on windows

Question

I am new in R programming and I have code like below and I know that the windows does not support multicore but I don't know how to change this part of the code. Can someone suggest me an equivalent code without using the mc.cores feature?

rpl <- unlist( lapply( waydf$geometry$coordinates , nrow ) ) # row per line
 waydf <- waydf[ rpl > 1 , ]
ll <- parallel::mclapply( waydf$geometry$coordinates , st_linestring,
                         mc.cores =parallel::detectCores() - 1  )
outdf <- sf::st_sf(
line_geometry = sf::st_sfc( ll , crs = epsg ) ,
osm_id = waydf$id
)

Answer 1

If all you want to do is make it so this code doesn't run in parallel, you just need to tell it to use 1 core, then it will use lapply under the hood.

ll <- parallel::mclapply(waydf$geometry$coordinates, st_linestring, mc.cores = 1)

Or just swap mclapply out for lapply.

ll <- lapply(waydf$geometry$coordinates, st_linestring)

Answer 2

You'll have to clarify what st_linestring is or does because you're trying to pass the contents of waydf$geometry$coordinates to it, but haven't specified any arguments, such as st_linestring(waydf$geometry$coordinates[i])

In Windows, you would use parLapply instead of mclapply.

# original
ll <- parallel::mclapply( waydf$geometry$coordinates , st_linestring,
                         mc.cores =parallel::detectCores() - 1  )

# replace the above with all of the below
library(parallel)
cl <- makeCluster(detectCores())
cl <- clusterEvalQ(cl, { library(sf) })  # you need to export packages as well
# cl <- clusterExport(cl, "st_linestring")  # each worker is a new environment, you will need to export variables/functions to
ll <- parallel::parLapply(cl, waydf$geometry$coordinates, function(i) st_linestring)    # if st_linestring takes arguments then st_linestring(x)
stopCluster(cl)

Edit since st_linestring is a function from the package sf, it is sufficient to export sf

2nd edit

rpl <- unlist( lapply( waydf$geometry$coordinates , nrow ) ) # row per line
 waydf <- waydf[ rpl > 1 , ]

library(parallel)
cl <- makeCluster(detectCores())
cl <- clusterEvalQ(cl, { library(sf) })  # you need to export packages as well
# cl <- clusterExport(cl, "st_linestring")  # each worker is a new environment, you will need to export variables/functions to
ll <- parallel::parLapply(cl, waydf$geometry$coordinates, function(i) st_linestring)    # if st_linestring takes arguments then st_linestring(x)
stopCluster(cl)

outdf <- sf::st_sf(
line_geometry = sf::st_sfc( ll , crs = epsg ) ,
osm_id = waydf$id
)

Answer 3

The problem is that you do cl <- ... in all those rows; you keep redefining that variable to be something else. You should only assign cl once and then reuse it.

library("parallel")
cl <- makeCluster(detectCores())
clusterEvalQ(cl, { library("sf") })
clusterExport(cl, "st_linestring")
res <- parallel::parLapply(cl, X = waydf$geometry$coordinates, 
                         fun = function(i) st_linestring)
stopCluster(cl)

The message Error in checkCluster(cl): not a valid cluster that you get with your code is because after you do cl <- clusterEvalQ(cl, { library("sf") }) it is no longer a cluster object.