How to use BLAS library in Spark?

Question

I'm new to scala and I'm writing a Spark application in Scala and I need to use the axpy function from org.apache.spark.mllib.linalg.BLAS. However it looks to be not accessible to users. Instead I try to import the com.github.fomil.netlib and directly access it. But I could either. I need to multiply to DenseVector.

Answer 1

Right now, the BLAS class within mllib is marked private[spark] in the spark source code. What this means is that it is not accessible external to spark itself as you seem to have figured out. In short, you can't use it in your code.

If you want to use netlib-java classes directly, you need to add the following dependency to your project

libraryDependencies += "com.github.fommil.netlib" % "all" % "1.1.2" pomOnly()

That should allow you to import the BLAS class. Note, I haven't really tried to use it, but I am able to execute BLAS.getInstance() without a problem. There might be some complexities in the installation on some Linux platforms as described here - https://github.com/fommil/netlib-java.