What are examples of efficient, type-inferred languages appropriate for working with multi-dimensional arrays

Question

I don't care much about garbage collection, if present it should be optional. The language D fits the bill but I am exploring other options. Surprising to me, this seems to be a sparsely populated spot among languages. I want something with which I can run things at 80% of the speed of C, if possible.

I also would like the language to have good support for multicores. Not necessarily via threads, but anything that does not involve lots copying. For example GNU's parallel mode for libstdc++ is a reasonably good abstraction for me, but a little weak at giving pre-baked array primitives (this is not a complaint, it is not its job to give array primitives).

I suspect what I am driving at is a OCaMl like language with:

1. good support for multidimensional arrays,
2. no(or optional) garbage collection,
3. parallel programming primitives for array intensive code,
4. convenient C FFI,
5. and with a reasonable chance of running at 80% of C speed.

I am not sure what tags to use so suggestions are welcome. I also want to make this a wiki, but not sure how to do it. I have heard of Felix but do not know if that would be appropriate here.

Answer 1

You're pretty much describing D, especially if you have a long time horizon and are willing to wait for some stuff that's in the works to fully pan out.

1. Good support for multidimensional arrays: I'm mentoring a Google Summer of Code project focused on just that. It's not polished and ready for prime time yet, but it's already usable if you live on the bleeding edge. It also includes bindings to BLAS and LAPACK and expression templates and evaluators to provide a nice wrapper over these libraries.

2. D's default approach to memory management is garbage collection, but enough low-level facilities exist that you don't have to use it where performance or space efficiency is a high priority. You have full access to the C standard library and can use C's malloc and free directly. The ability to work with untyped blocks of memory also allows custom allocators to be implemented. Both I and the GSoC student I'm mentoring have been using a region allocator, which will be reviewed soon for inclusion in Phobos (the D standard library).

3. Parallel programming: See the std.parallelism module of the standard library. It's not geared specifically to array intensive code but it's definitely usable for that purpose.

4. D fully supports the C ABI. All you have to do is translate the header file and link in a C object file. There's even a tool called htod that can automate the simple cases.

5. D is currently somewhat slower than C because it hasn't had years of work put into optimization, as the focus has been on features and bug fixes. However, it's not far behind and almost all of the performance difference can be explained by a somewhat naive garbage collector and lack of aggressive inlining. If you avoid using the garbage collector in the most performance-critical parts of your code and manually inline a few functions here and there, it should be as fast as C.

Answer 2

Although I'm a huge fan of both OCaml and D, in regards to existing libraries I have found c++ to be the out and out winner here. Expression template work cut it's teeth in c++'s manipulation of multidimensional arrays and has been used to create extremely efficient libraries that vastly outperform bare c. Temporaries may be removed from calculations that you just cannot remove from c code that is generic enough to compose operations. Loops can be unrolled automatically during translationtime. And you can use autoparallelisation features out of the box that further drive your use of multicore boxes to new heights. With c++11's type inference features, you have all of your requested behavior in a mature implementation that will outperform nearly any other language out there.

Now, I'm not saying you can't do the same in D. You just won't have the maturity of libraries like uBLAS, Eigen, or Blitz++. And you won't have the compiler support that you find with Intel's Cilk Plus and other Parallel Building Blocks. Obviously, down the road I have no doubt that this kind of support will be available from the community, but c++ is the only language I have used that offers it today.

I am saying that you can't get that with OCaml, at least the standard compiler and library, simply because of the lack of true symmetric multiprocessing. Something like JoCaml, OC4MC, etc. may provide the necessary parallelisation, but you still lack a deep use of temporary reduction in commonly available matrix libraries.

This is just my experience. It is much harder yet to get these kinds of optimisations from C#, F#, and that family due to lack of deterministic destruction of temporaries - making expression template techniques much more unwieldy. The lack of compiletime type inference of templates makes many techniques unreachable. Of all the languages I have worked with over the years, building tensor and spinor libraries, graph libraries, abstract algebras (represented elements in matrices), etc., c++ still holds the best support for efficiency, now in a much better type deduced environment.

Answer 3

I would suggest F#.

• Familiar syntax of Ocaml
• 2D array support in std library for F#
• TPL, PLINQ and Async support for parallel programming
• CLR based, hence garbage collection (Runs on Microsoft .NET as well as open source Mono)
• PInvoke for call native C libraries
• Good performance