How to install XGBoost on OSX with multi-threading

Question

I'm trying to install xgboost on my mac (osx 10.12.1) following the guide here but I'm running into some issues.

---

Step1

Obtain gcc-6.x.x with openmp support by brew install gcc --without-multilib

Terminal

Ben$ brew install gcc --without-multilib
Error: gcc-5.3.0 already installed
To install this version, first `brew unlink gcc`

Ben$ brew unlink gcc
Unlinking /usr/local/Cellar/gcc/5.3.0... 1288 symlinks removed

Ben$ brew install gcc --without-multilib
[26 minutes later]
==> Summary
🍺  /usr/local/Cellar/gcc/6.2.0: 1,358 files, 238.3M, built in 26 minutes 20 seconds

---

Step2

Clone the repository git clone --recursive https://github.com/dmlc/xgboost

Terminal

Ben$ git clone --recursive https://github.com/dmlc/xgboost
Cloning into 'xgboost'...
remote: Counting objects: 18754, done.
remote: Compressing objects: 100% (21/21), done.
remote: Total 18754 (delta 1), reused 0 (delta 0), pack-reused 18733
Receiving objects: 100% (18754/18754), 6.23 MiB | 3.74 MiB/s, done.
Resolving deltas: 100% (11347/11347), done.
Checking connectivity... done.
Submodule 'dmlc-core' (https://github.com/dmlc/dmlc-core) registered for path 'dmlc-core'
Submodule 'rabit' (https://github.com/dmlc/rabit) registered for path 'rabit'
Cloning into '/Users/Ben/xgboost/dmlc-core'...
Cloning into '/Users/Ben/xgboost/rabit'...
Submodule path 'dmlc-core': checked out 'f35f14f30835af238257b979cc1fac3e41ff3291'
Submodule path 'rabit': checked out 'a9a2a69dc1144180a43f7d2d1097264482be7817'

---

Step3

build using the following commands cd xgboost; cp make/config.mk ./config.mk; make -j4

NOTE: If you use OSX El Capitan, brew installs gcc the latest version gcc-6. So you may need to modify Makefile#L46 and change gcc-5 to gcc-6. After that change gcc-5/g++-5 to gcc-6/g++-6 in make/config.mk then build using the following commands

Hmmm. Not entirely sure what to do here. I see a file called "Makefile" inside xgboost/. Inside it I see a lot of stuff I don't understand, but these lines look relevant

Makefile (snippet)

# on Mac OS X, force brew gcc-6, since the Xcode c++ fails anyway
# it is useful for pip install compiling-on-the-fly
OS := $(shell uname)
ifeq ($(OS), Darwin)
export CC = $(if $(shell which gcc-6),gcc-6,$(if $(shell which gcc-mp-6), gcc-mp-6, clang))
export CXX = $(if $(shell which g++-6),g++-6,$(if $(shell which g++-mp-6),g++-mp-6, clang++))
endif

Going to ignore this and move on...

Terminal

Ben$ cd xgboost; cp make/config.mk ./config.mk; make -j4
[Tons of log output. Appears to finish succesfully]
...
g++-6 -std=c++0x -Wall -O3 -msse2  -Wno-unknown-pragmas -funroll-loops -Iinclude   -Idmlc-core/include -Irabit/include -fPIC -fopenmp -shared -o lib/libxgboost.so build/learner.o build/logging.o build/c_api/c_api.o build/c_api/c_api_error.o build/common/common.o build/data/data.o build/data/simple_csr_source.o build/data/simple_dmatrix.o build/data/sparse_page_dmatrix.o build/data/sparse_page_raw_format.o build/data/sparse_page_source.o build/data/sparse_page_writer.o build/gbm/gblinear.o build/gbm/gbm.o build/gbm/gbtree.o build/metric/elementwise_metric.o build/metric/metric.o build/metric/multiclass_metric.o build/metric/rank_metric.o build/objective/multiclass_obj.o build/objective/objective.o build/objective/rank_obj.o build/objective/regression_obj.o build/tree/tree_model.o build/tree/tree_updater.o build/tree/updater_colmaker.o build/tree/updater_histmaker.o build/tree/updater_prune.o build/tree/updater_refresh.o build/tree/updater_skmaker.o build/tree/updater_sync.o dmlc-core/libdmlc.a rabit/lib/librabit.a -pthread -lm  -fopenmp 

---

Step4 (instructions here?)

If you would like to use the latest xgboost version and already compiled xgboost, use library(devtools); install('xgboost/R-package') to install manually xgboost package (change the path accordingly to where you compiled xgboost).

New R session

library(devtools)
install("/Users/Ben/xgboost/R-package")

Installing xgboost
'/Library/Frameworks/R.framework/Resources/bin/R' --no-site-file --no-environ --no-save --no-restore --quiet CMD INSTALL  \
  '/Users/Ben/xgboost/R-package' --library='/Library/Frameworks/R.framework/Versions/3.3/Resources/library' --install-tests 

* installing *source* package ‘xgboost’ ...
** libs
g++-5 -std=c++11 -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG -I../..//include -I../..//dmlc-core/include -I../..//rabit/include -I../../ -DXGBOOST_STRICT_R_MODE=1 -DDMLC_LOG_BEFORE_THROW=0 -DDMLC_ENABLE_STD_THREAD=1 -DDMLC_DISABLE_STDIN=1 -DDMLC_LOG_CUSTOMIZE=1 -DXGBOOST_CUSTOMIZE_LOGGER=1 -DRABIT_CUSTOMIZE_MSG_ -DRABIT_STRICT_CXX98_ -I/usr/local/opt/llvm/include   -fopenmp  -fPIC  -Wall -mtune=core2 -g -O2 -c xgboost_R.cc -o xgboost_R.o
make: g++-5: No such file or directory
make: *** [xgboost_R.o] Error 1
ERROR: compilation failed for package ‘xgboost’
* removing ‘/Library/Frameworks/R.framework/Versions/3.3/Resources/library/xgboost’
* restoring previous ‘/Library/Frameworks/R.framework/Versions/3.3/Resources/library/xgboost’
Error: Command failed (1)

Fail :( Where did I go wrong?

Answer 1

Updated Solution

As of March 2019, R version 3.5.3, xgboost version 0.82.0.1, things have changed since my old answer. The following steps are based on the installation guide here, but differ slightly.

1. brew install cmake or brew upgrade cmake (cmake --version shows 3.14.0 for me)
2. brew install gcc or brew upgrade gcc (Note the version of gcc that gets installed. It should be located in /usr/local/bin. I get gcc 8.3.0 (gcc-8 --version))
3. cd some/unimportant/directory
4. git clone --recursive https://github.com/dmlc/xgboost
5. cd xgboost/
6. mkdir build
7. cd build
8. CC=gcc-8 CXX=g++-8 cmake .. -DR_LIB=ON
9. make -j4
10. sudo make install

Now restart/refresh RStudio and it should be installed

Test in R

set.seed(222)
N <- 2*10^5
p <- 350
x <- matrix(rnorm(N  * p), ncol = p)
y <- rnorm(N)

system.time(mymodel <- xgboost(
  # nthread = 4,
  data = x,
  label = y, 
  nrounds = 5, 
  objective = "reg:linear", 
  tree_method = "exact",
  max_depth = 10,
  min_child_weight = 1, 
  eta = 1, 
  subsample = 0.66, 
  colsample_bytree = 0.33
))

# Tested on 2018 MPB, xgboost version 0.82.0.1, multi-threaded version
# nthread = default:   7.4 seconds (elapsed)
# nthread = 1:        24.0 seconds (elapsed)
# nthread = 2:        13.7 seconds (elapsed)
# nthread = 4:         7.5 seconds (elapsed)

---

Old Answer

It's amazing how writing your question on StackOverflow often leads you directly to the answer. After spending hours on this, I figured out that I needed to change these three lines in ~/.R/Makevars

CC=gcc-5
CXX=g++-5
CXX1X = g++-5

to

CC=gcc-6
CXX=g++-6
CXX1X = g++-6

Also, I ended up installing xgboost from the "drat" repo

install.packages("drat", repos="https://cran.rstudio.com")
drat:::addRepo("dmlc")
install.packages("xgboost", repos="http://dmlc.ml/drat/", type = "source")

Answer 2

My approach is after your first(update gcc) and second(clone) steps, no need to build anything(they are for other languages like python)

3) update ~/.R/Makevars

https://github.com/dmlc/xgboost/issues/1136

How to make R package xgboost parallel in OS X with OpenMP compilation?

4) from the clone xgboost/R-package/configure change ac_pkg_openmp to yes https://github.com/dmlc/xgboost/issues/2503

5) in xgboost/R-package type R CMD INSTALL . http://xgboost.readthedocs.io/en/latest/build.html#r-package-installation