What performance tradeoffs exist between various clojure matrix libraries?

Question

There are a number of matrix libraries out there for clojure:

• vectorz-clj
• clatrix
• parallel colt
• cerebro

What are the performance trade-offs between these libraries? I've heard that with some of the underlying implementations, there are trade-offs between (for-example) matrix instantiation and operation performance, but I haven't been able to find a comprehensive resource detailing these considerations.

Thanks

Answer 1

If you want to make use of core.matrix, there are only two implementations at present that are reasonably mature and performant:

• Clatrix - uses calls to native BLAS
• vectorz-clj - a flexible and fast pure-JVM implementation

It really comes down to your use cases. If you mostly care about big linear algebra operations and don't mind the native dependencies, then Clatrix is your best bet at present - simply because BLAS implementations are so fast. This is particularly useful for:

• Large matrix multiplication
• Linear algebra (matrix decompositions etc.)

If you want to do general array-programming work, then vectorz-clj has the advantage of being pure JVM code and much more flexible in terms of array/matrix formats. Examples of things that vectorz-clj supports well that you can't do in Clatrix:

• N-dimensional arrays
• Various specialised types of sparse arrays (diagonal matrices, different sparse storage formats etc.)
• Arrays with arbitrary strided access (like Numpy)
• Lightweight "views" into larger arrays

Overall, vectorz-clj won't be as fast for things like big matrix multiplication, but is probably faster than Clatrix for many other operations and small/medium sized vector work. I'd normally choose vectorz-clj unless I thought that linear algebra performance would be the main bottleneck.

The other core.matrix implementations are less mature, but may still be useful for specific use cases. A nice feature of core.matrix is the ability to mix and match implementations while using the same common API, so it's not an "all or nothing" choice.

Disclaimer: I have created or contributed to many of the above projects. I hope I've given a fairly unbiased and objective evaluation.

If you don't need core.matrix support, then you have many more options - you can use any of the Java matrix libraries via Clojure's Java interop. In theory, these could become core.matrix implementations as well - the only constraint is that someone needs to do the work to extend the core.matrix protocols to support the new matrix types.