Spark MLlib - Training collaborative filtering with implicit feedback - strange warnings

Question

I am trying to build Collaborative filtering model on the user orders and getting some useful results with ALS.train() but I would like to try ALS.trianImplicit() but trianImplicit() is predicting just zeros on same dataset which ASL.train() I was getting decent predictions.

When using ALS.trianImplicit() to train the model I was getting following warnings:

15/09/01 15:39:29 WARN BLAS: Failed to load implementation from: com.github.fommil.netlib.NativeSystemBLAS
15/09/01 15:39:29 WARN BLAS: Failed to load implementation from: com.github.fommil.netlib.NativeRefBLAS

Does this indicate that it was an error and not warnings and model just could not train anything because of missing libraries?

Answer 1

As @eliasah has pointed out this warnings an not-critical, but may slow down performance. Using native BLAS can result in a huge performance improvement. There are some instructions on https://github.com/amplab/ml-matrix/blob/master/EC2.md on how to setup Spark + BLAS on EC2.

If your cluster is running Ubuntu you can install the following packages:

libblas3gf
libblas-doc
libblas-dev

liblapack3gf
liblapack-doc
liblapack-dev

However, some people have reported better performance from using http://www.openblas.net/ so you can try installing that on your workers. You will also need to include com.github.fommil.netlib with your application (Spark currently uses version 1.1.2).