installing Rcartogram packages - error message

Question

I am trying to build a cartogram like here

Installation from the link does not work:

`install.packages('Rcartogram', repos = 'http://www.omegahat.org/R', type = 'source')`
Installing package into ‘C:/Users/Milena/Documents/R/win-library/3.2’
(as `lib` is unspecified)

Warning in install.packages :
  package ‘Rcartogram’ is not available (for R version 3.2.0)

Neither from the zip file:

install.packages("C:/Users/Milena/Downloads/Rcartogram_0.2-2.tar.gz", repos = NULL, type = "source")

Installing package into ‘C:/Users/Milena/Documents/R/win-library/3.2’ (as lib is unspecified) * installing source package Rcartogram ... **********************************************
WARNING: this package has a configure script It probably needs manual configuration **********************************************

** libs

*** arch - i386 Warning: running command 'make -f "Makevars" -f "C:/PROGRA~1/R/R-3.2.0/etc/i386/Makeconf" -f "C:/PROGRA~1/R/R-3.2.0/share/make/winshlib.mk" SHLIB="Rcartogram.dll" OBJECTS="Rcart.o cart.o"' had status 127 ERROR: compilation failed for package 'Rcartogram' * removing 'C:/Users/Milena/Documents/R/win-library/3.2/Rcartogram' Warning in install.packages : running command '"C:/PROGRA~1/R/R-3.2.0/bin/x64/R" CMD INSTALL -l "C:\Users\Milena\Documents\R\win-library\3.2" "C:/Users/Milena/Downloads/Rcartogram_0.2-2.tar.gz"' had status 1 Warning in install.packages : installation of package ‘C:/Users/Milena/Downloads/Rcartogram_0.2-2.tar.gz’ had non-zero exit status

How can I solve this problem?

I am working on Windows machine.

Thank you to everyone who took the time to look up this question.

Here is my sessionInfo:

R version 3.2.0 (2015-04-16)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 8 x64 (build 9200)

locale:
[1] LC_COLLATE=English_United Kingdom.1252  LC_CTYPE=English_United Kingdom.1252    LC_MONETARY=English_United Kingdom.1252 LC_NUMERIC=C                           
[5] LC_TIME=English_United Kingdom.1252    

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] fftw_1.0-3

loaded via a namespace (and not attached):
[1] tools_3.2.0

Answer 1

Installing Rcartogram on Windows

Rcartogram is an R package (by Duncan Temple Lang) whose main purpose is to provide a R wrapper for a some C code (written by Mark Newman) which actually does the work of constructing a cartogram (aka morphing a map). The C code written by Mark Newman makes use of the FFTW (fastest Fourier Transform in the West) compiled library.

The link in your question by Truc Viet Le describes how to install Rcartogram on a Unix system. There are a few extra tricks and traps involved in getting Rcartogram onto a Windows system, even though at its heart, it is pretty much the same process.

To install Rcartogram on a Windows system you need first to install all the pre-requisites, namely:

• the Rtools suite of programs (which you need to be able to install any R package which involves C code onto a Windows machine), and
• The FFTW library (which the Rcartogram code uses).

You then need to tell R where to find the FFTW library when you are first installing Rcartogram, and you will almost certainly need to let R know where to find the FFTW library whenever you load Rcartogram, eg via library(Rcartogram) in an R session.

I found I also needed to make a few very small changes to the Rcartogram R code (mostly to bring it into line with changes to R syntax since it was written) in order to get it to install happily, and run correctly on Windows.

So the full answer involves several steps.

Step 1 : Install the Rtools suite

I suspect you need to install Rtools in order to get past the status 127 error. The official instructions on how to do that are here http://cran.r-project.org/doc/manuals/R-admin.html#The-Windows-toolset. There are user friendly explanations of how to install Rtools into a Windows system elsewhere on the web --- see for example https://github.com/stan-dev/rstan/wiki/Install-Rtools-for-Windows . (The official instructions tell you how to install lots of other stuff as well, that you need if you want to build R itself from source on Windows, or produce package documentation using LaTeX, but you don't need all that stuff to get Rcartogram working).

A bit longer answer: I can now replicate your status 127 error ---by removing the references to directories where Rtools lives from my PATH. When I do that the Windows cmd shell (where you might type R CMD INSTALL … can’t find the Rtools executables and that results in the 127 error message. Likewise trying to run install.packages() from within R also fails the same way, since under the hood install.packages calls the Windows cmd shell.

Why do you need Rtools? Well Rcartogram is a package which includes C code, as well as pure R code. In fact it is mostly C code – from Mark Newman. Installing from source for a package which includes C code requires a C compiler. In fact it is best (almost essential) that it is the same C compiler that R itself was built from. That is what Rtools mostly is – an installable on Windows version of a C compiler. Running a C compiler in Windows needs a few extra shell commands (aka small programs) as well and that is what the rest of Rtools is. Most of the (open source) C community seem to work in a Unix (or variant thereof) world and those extra commands (and indeed the C compiler itself) are part of the “standard” system in Unix. It’s only those of us who work in Windows who need to install Rtools, which is a port of the necessary tools from Unix to Windows.

Step 2 : Install the FFTW library

Initially I got the FFTW library from here http://www.fftw.org/ .

There are two versions, a 32 bit version and a 64 bit version. On a Windows 64 bit machine you need both versions. (Aside, well there may be a way you can get away with only one, by setting flags when you install Rcartogram, but I haven't tested that route myself yet). Unzip the 32 bit version into a sub-directory /i386, and the 64 bit version into a subdirectory /x64. In my case (see below), I made these as subdirectories of "C:/msys/fftwtest". (Aside these subdirectories are conventions that R uses - you could theoretically put them elsewhere, but why make extra complications!).

One trap that stumped me for quite a while was that these libraries are dynamic libraries (ie .dll s) and so - later on - I needed to make sure that when I installed them on my pc I put them in locations that were on my PATH (or alternatively I altered my PATH by adding in the locations - aka directories - where they were installed) since otherwise I got very unhelpful error messages in the final checks that R does before it finishes installing a package. Both the 32 bit and 64 bit (sub) directories should be included in your PATH.

Step 3 : Tell R where to find the FFTW library

The trick to telling R (on a Windows machine) where to find the FFTW libraries when it is trying to install Rcartogram is to add a src/Makevars.win file into the src subdirectory of the Rcartogram package. That means you will have to unzip and untar the Rcartogram tar.gz file before you can make this change. (Aside : I use 7zip to uncompress these types of files on my machine).

My src/Makevars.win file (it is a text file) has 2 lines,

PKG_CPPFLAGS=-I"C:\msys\fftwtest\x64\include"   -DNOPROGRESS 
PKG_LIBS=-L"C:\msys\fftwtest\x64\lib" -L"C:\msys\fftwtest\i386\lib" -lfftw3 -lm

The file names in quotes are where I put my versions of the FFTW library. (These aren't exactly the ones I downloaded, along the way I learnt how to compile FFTW from source, and made my own copies, but explaining how to do that is a looong story so I won't attempt it here).

The directory mentioned in PKG_CPPFLAGS line is the one containing a header file called fftw3.h that the C pre-processor needs. It doesn't matter whether you point at the 32 bit (\i386 subdirectory) or the 64 bit (\x64 subdirectory) - the fftw3.h file is a C source file and is the same no matter what architecture R is installing for.

The 2 directories mentioned in PKG_LIBS line are the ones where files called libfftw3.something can be found, and that the linker needs when it is putting everything together at the end of the compilation step. something might be ".dll" (in which case the subdirectory might be \bin instead of \lib), or it might be ".a" or ".la" (which is what R looks for when it uses the static FFTW libraries which I created once I had learnt how to compile FFTW from source).

2 directories are needed because R by default tries to install both 32 bit and 64 bit versions of Rcartogram on Windows machines. If you supply FFTW library files in .dll format, then these are the exact same libraries that must be on your PATH (because when you try to do library(Rcartogram) R needs to find the FFTW dll libraries again while it is loading the installed Rcartogram package) (Aside, that's why in the end I compiled my own static FFTW libraries, so I didn't have to mess with my PATH variable in the Windows environment).

If you are using the downloaded binaries from the link above, the fftw3.h and the libfftw3.dll files are all in the same (sub) directory, and the libfftw3.dll file is actually called libfftw3-3.dll, so in this case your src/Makevars.win file would need to be :

PKG_CPPFLAGS=-I"main libfftw directory\x64"   -DNOPROGRESS 
PKG_LIBS=-L"main libfftw directory\x64" -L"main libfftw directory\i386"  -lfftw3-3 -lm

The key differences from my src/Makevars.win are :

• inserting the name of your main libfftw directory - ie the parent directory under which you created the /i386 and /x64 subdirectories when you unzipped the downloaded FFTW binaries
• the deletion of the \include and \lib sub-sub-directories, and
• changing -libfftw3 to -libfftw3-3 (note also that there must be a space in front of each - (minus) sign at the start of the -L and -l flags).

What is the Makevars.win file doing? It is telling the R install process the flags that it will need when it tries to preprocess, compile and link the C code in Rcartogram's src subdirectory. The value of PKG_CPPFLAGS is a set of flags for the C pre-processor, and the value of PKG_LIBS is a set of flags for the link step.

• The -I is a flag that says 'try looking in the following directory when the C pre-processor is looking for include files', so in the example above it says to look in "main libfftw directory\x64". The include file it seeks is fftw3.h (that filename is buried in the C code inside Rcartogram)
• The -L flag says 'try looking in the following directory when the linker is looking for files from any library that you expect to use', so -L"main libfftw directory\x64" says try looking in the "main libfftw directory\x64" directory. You can (and need to) have more than 1 directory on that search path - the linker just keeps looking till it finds what it is looking for (or runs out of places to look and gives an error message), and
• The -l flags gives the name of the library file that the linker should look for, but not verbatim --- instead the name is constructed from what you enter following a (slightly crazy to me) convention from the unix world. Because the file name of the library always begins with “lib”, the first part of the convention is that you leave "lib" out of the name you put in the flag. The file name of the library can have several different extensions ( eg “.dll” or “.a”) so the second part of the convention is that you leave you leave the file extension out of the -l flag value as well, and let the linker sort out what it wants. So –lfftw3 says look for a file called either libfftw3.dll or one called libfftw3.a (there may be other possible extensions as well, I'm not sure).

The downloaded dlls are actually called libfftw3-3.dll, (unlike the ones I compiled myself, which are called libfftw3.a) hence the need to change the –l flag to –lfftw3-3

NB If you are using the downloaded FFTW library which uses .dlls make sure you have put them on your PATH (see the last para of step 2) as well.

Step 4 : Small fixes to the Rcartogram C code

There were two other small changes I found I had to make to the Rcartogram code itself get things running.

First in the file R/cart.R there are two lines, both of which use the .Call( ) function. I needed to add one more argument (namely PACKAGE = "Rcartogram") to the .Call function, so for example

tmp = .Call("R_makecartogram", popEls, x, y, dim, as.numeric(blur))

became

tmp = .Call("R_makecartogram", popEls, x, y, dim, as.numeric(blur), PACKAGE = "Rcartogram")

Likewise, further down cart.R the altered .Call became

.Call("R_predict", object, as.numeric(x),  as.numeric(y), ans, dim(object$x), PACKAGE = "Rcartogram")

Second, again in R/cart.R, I had to change

tmp = rep(as.numeric(NA), length(x))
ans = list(x = tmp, y = tmp)

to

# Avoid problems with the same vector (tmp) being sent to C twice due to R's
# copy-on-modify rules
tmp_x = rep(as.numeric(NA), length(x))
tmp_y = rep(as.numeric(NA), length(y))
ans = list(x = tmp_x, y = tmp_y)

This one took me a lot of work to find, but without it, the demo for Rcartogram gave the wrong results (even though it ran OK).

Step 5 : Actually install Rcartogram

You should now be able to install Rcartogram. Either

• by opening a cmd window, changing directory (cd to) the location where the unzipped and modified Rcartogram package source code lives, and typing R CMD INSTALL --preclean . or

• by starting an R session, setting the working directory to wherever the Rcartogram source is and typing install.packages(".", repos = NULL, type = 'source', INSTALL_opts = "--preclean")

The . works because you have cded to directory where the Rcartogram source code lives. The --preclean flag tells R to tidy up any leftover intermediate files from earlier (failed) attempts to compile the C code in Rcartogram before it begins. If you get this far and are still having trouble, there is also a --debug flag that can be added as well. It gives more detail about why the install is failing.

Step 6 : Enjoy morphing maps

I am just getting started actually using Rcartogram myself (it took me a while to get this far!), but you may want to check out the getcartr --- R package. That package uses Rcartogram, and it seems pretty neat! And the installation instructions given on the github website worked first time for me (I do already have devtools installed and working though).

Hope this helps (and congratulations to anyone who has read this far)

Update May 2017

I haven't worked on this for a couple of years now (so no guarantees it will still work), but after I wrote the post above, I created a forked copy of RCartogram at https://github.com/Geoff99/Rcartogram/tree/WindowsInstall.

See the WindowsInstall branch which includes

• a heavily commented src/Makevars.win meant to make it easier to install RCartogram on Windows, and
• an even more comprehensive tutorial than the above post in vignettes/README.WindowsInstall.Tutorial.Rmd . See the following link https://github.com/Geoff99/Rcartogram/blob/WindowsInstall/vignettes/README.WindowsInstall.Tutorial.Rmd

(To use the tutorial, you need to use the WindowsInstall branch of the forked repository!)