access numpy array from a functional language

Question

My primary language is Python. Often when I need to do some cpu heavy task on a numpy array I use scipy.weave.inline to hook up c++ with great results.

I suspect many of the algorithms (machine learning stuff) can however be written simpler in a functional language (scheme, haskell...).

I was thinking. Is it possible to access numpy array data (read and write) from a functional language instead of having to use c++?

Answer 1

You might have a look at using a shared-memory array of some sort. This implementation would probably be a good place to start: https://bitbucket.org/cleemesser/numpy-sharedmem/src

This implementation is intended to be shared between python processes, but it's using named shared memory to do it, so you should be able to access the relevant chunk of memory from any other process.

I'm not familiar enough with haskell to give you any advice on that side, but I assume you can use a pointer to a shared memory buffer as an array of some sort in haskell...

Answer 2

There's no single standard way to call Haskell from Python at the moment. There are certainly ways to call haskell from C, which means there's no obstacle in principle to calling Haskell -- the work simply hasn't been done to make this particularly easy.

On the other hand, if your data structures aren't themselves enormous, serializing them to a Haskell program (either via the command line, or using, a client-server model with e.g. thrift) is very straightforward, and if the computation cost is what sufficiently dominates, the cost may be minimal.

Finally, it is very easy to call Python from Haskell! The classic package for this is missingpy: http://hackage.haskell.org/package/MissingPy

There's also a newer package called cpython which attempts to be more comprehensive: http://hackage.haskell.org/package/cpython

Conceptually, it shouldn't be very hard, I imagine, to host your Python app in Haskell rather than the other way around.