Install.packages installs source in Rstudio console but binary when using Rscript

Question

I am working on a package that supports interaction with an alternatively shaped library structure which supports installing multiple versions of a package in parallel (RVClibrary, soon on CRAN). Recently, I met some strange behaviour and I hope someone can explain.

When installing a package, it can sometimes have a source version which is later than the binary version. The source (uncompiled C code) is in that case not yet compiled (by ...?) but contains a later version than the already compiled binary version. The following is shown in my case:

  There is a binary version available but the source version is later:
      binary source needs_compilation
rlang  0.2.0  0.3.4              TRUE

A choice is given to the user whether A or B should be relatively installed or compiled. In other words, if you want version 0.2.0 or 0.3.4 (for which you need Rtools).

This choice will not be given when the install.packages.compile.from.source option is set.

> options(install.packages.compile.from.source = "always")
> install.packages('rlang', lib = "C:\\test", quiet = TRUE, repos = "http://cran.us.r-project.org")

  There is a binary version available but the source version is later:
      binary source needs_compilation
rlang  0.2.0  0.3.4              TRUE

installing the source package ‘rlang’

The source is installed.

But when I create a simple script (I name it install_test.R):

packageName <- commandArgs(trailingOnly = TRUE)[1]

cat(packageName, '\n')

options(install.packages.compile.from.source = "always")
install.packages(packageName, lib = "C:\\test", quiet = TRUE, repos = "http://cran.us.r-project.org")

The following behaves the same with system in R and directly in CMD.
(Make sure you have the path to Rscript.exe in your environment variable PATH)

> system("Rscript.exe C:\\test\\install_test.R rlang")
rlang 

  There is a binary version available but the source version is later:
      binary source needs_compilation
rlang  0.2.0  0.3.4              TRUE

  Binaries will be installed
package 'rlang' successfully unpacked and MD5 sums checked

The Binaries are installed!

I need to be able to always install the Source version so I do not run into dependency issues. ggplot2 for example depends on >= 0.2.1 already...

Summary

• I have Rtools 3.5 installed (which devtools can find using both routes and thinks it is not compatible..., but installing from source works fine in Rstudio, and (https://cran.r-project.org/bin/windows/Rtools/) CRAN says it's compatible to)
• R version:

platform       x86_64-w64-mingw32
arch           x86_64
os             mingw32
system         x86_64, mingw32
year           2016
month          06
day            21
svn rev        70800
language       R
version.string R version 3.3.1 (2016-06-21)
nickname       Bug in Your Hair

• running Windows 10
• The difference between environment variables can be seen here (Rstudio to the right, Rscript left): https://pasteboard.co/IasIVMe.png

---

(my edit here is removed to keep it clean. See my answer (or history) to read more)

Answer 1

I tried different environment variable configurations to see if installing/compiling from source can be performed by an Rscript.exe instance running install.packages.

(It works differently (less restrictive) when working from Rstudio console. It seems that it recognizes the registry entry. It works without specifying an environment var at all).

1. Set MAKE variable (Sys.setenv(MAKE = 'C:\\Rtools\\bin\\make.exe')) (partially SUCCEEDS)
2. Set PATH variable (appending to the beginning)
   • Include all Rtools bin dirs (C:\\Rtools\\bin;C:\\Rtools\\mingw_32;C:\\Rtools\\mingw_64) (SUCCEEDS)
   • Include Rtools C:\\Rtools\\bin only (SUCCEEDS)
   • Include Rtools C:\\Rtools\\bin only (to the end of PATH variable) (SUCCEEDS)
   • Include Rtools C:\\Rtools\\mingw_32 only (FAILS)
   • Include Rtools C:\\Rtools\\mingw_64 only (FAILS)
3. Set only the PATH and not the MAKE variable.

Under the condition that the following property is set: options(install.packages.compile.from.source = "always")

Solution 1:
The install.packages function 'checks' if it should try building from source by looking for the 'MAKE' environment variable nzchar(Sys.which(Sys.getenv("MAKE", "make"))). Unfortunately, if you only specify the MAKE environment variable, which you must provide a complete path to make it (the Sys.which) work, it will still not find all required source build tools (Rtools/bin) that appear to be necessary and crash without any clarrification:

> Sys.setenv(MAKE = 'C:\\Rtools\\bin\\make.exe')
> system("Rscript.exe C:\\R_projecten\\test\\install_test.R rlang")

     There is a binary version available but the source version is later:
         binary source needs_compilation
   rlang  0.2.0  0.3.4              TRUE

   installing the *source* package 'rlang'

   Warning messages:
   1: running command '"C:/PROGRA~1/R/R-33~1.1/bin/x64/R" CMD INSTALL -l "C:\R_projecten\test\test4" C:\Users\SB947~1.FRO\AppData\Local\Temp\RtmpwbEcl6/downloaded_packages/rlang_0.3.4.tar.gz' had status 1 
   2: In install.packages(packageName, lib = "C:\\R_projecten\\test\\test4",  :
     installation of package 'rlang' had non-zero exit status

So solution 1 is not sufficient, and can even be replaced by solution 3:

   PATH : C:\Program Files\R\R-3.3.1\bin\x64; ....  ;C:\Rtools\bin; 
   MAKE : 

     There is a binary version available but the source version is later:
         binary source needs_compilation
   rlang  0.2.0  0.3.4              TRUE

   installing the source package 'rlang'

As shown, only providing C:/Rtools/bin in PATH, and no MAKE variable is sufficient.

The check decribed earlier checks if the MAKE variable exists, and use that value, or it will look for make, the program name (provided as default to Sys.getenv). When specifying the PATH only, the make(.exe) file can be found and no MAKE env var is required. With the added benefit of already pointing to the Rtools/bin directory from which apparently more files are needed.

Which files I am still wondering though...

NOTE: The check nzchar(Sys.which(Sys.getenv("MAKE", "make"))) is NOT sufficient to prove that your environment is ready for building from source.