I am building a package for R which I want to be able to be cross-platform. I am developing under Linux, and the function mclapply
will be used from the parallel
package. This package is not supported for Windows (which uses doParallel
). I really like the parallel
package though for it's simplicity and speed, and I do not know if this should be a reason to have 2 different versions available of the package for CRAN, for the separate OS (seems like extra work to maintain), not to mention if it is even allowed.
Thoughts?
Also, for now I am regarding parallel
's
mclapply(ldata, function(x), mc.cores=cores)
to be equivalent of doParallel
's
cl <- makeCluster(cores)
parLapply(cl, ldata, function(x))
Is that correct?
First, both mclapply
and parLapply
are in the parallel
package, although mclapply
doesn't actually run in parallel on Windows. parLapply
runs in parallel on all supported platforms, but isn't always as efficient as mclapply
. The doParallel
package is used with the foreach
package, and acts as an adapter to the parallel
package.
To write a package that works on both Windows and non-Windows, you have a variety of reasonable options:
parLapply
since it works everywhereparLapply
on Windows and mclapply
elsewheredoParallel
with foreach
The doParallel
package is convenient because it makes use of mclapply
on non-Windows platforms. For example:
library(doParallel)
registerDoParallel()
foreach(i=1:10, .options.snow=list(preschedule=TRUE)) %dopar% {
Sys.sleep(2)
}
This uses mclapply
on Linux and Mac OS X, but will automatically create a PSOCK cluster object behind the scenes on Windows. The use of preschedule=TRUE
(added in doParallel
1.0.3) will cause doParallel
to preschedule the tasks using clusterApply
internally, much like parLapply
.
Note that if you explicitly create and register a cluster object, then mclapply
will not be used, regardless of the platform. It will work fine, but may not be as efficient. To use mclapply
, you must call registerDoParallel
with a numeric argument, or no argument at all.
You can look at the source code for the boot
package for an example of how to use either mclapply
or parLapply
depending on your platform.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With