Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to use parallel make when compiling R packages?

When using the make tool directly, one can use the -j option to build in parallel.

How can I use parallel build when installing an R package using install.packages()? make is invoked by R, not by me, so I can't pass the -j option to it. Setting export MAKE_FLAGS=-j4 before starting R did not work. I am looking to set up parallel build permanently for my R installation.

like image 407
Szabolcs Avatar asked Apr 27 '26 16:04

Szabolcs


1 Answers

The options(Ncpus=8) route is one way. In install.packages() you have Ncpus = getOption("Ncpus") and that option is described as

Ncpus: the number of parallel processes to use for a parallel install of more than one source package. Values greater than one are supported if the ‘make’ command specified by ‘Sys.getenv("MAKE", "make")’ accepts argument ‘-k -j Ncpus’.

I don't see it listed in update.packages() but it functions that same way on my Linux machines so builds generally happen in parallel.

In short, this uses parallel builds of multiple packages, as opposed to make -j ... when just building one package. I tried that route too but found the gains less compelling.

like image 51
Dirk Eddelbuettel Avatar answered Apr 29 '26 05:04

Dirk Eddelbuettel