clojure pmap/preduce vs fork-join

Question

Looks like clojure will have a fork-join implementation which looks like a functional wrapper over java's fork join framework.

I am wondering what the difference between these and pmap/preduce could be ?

Answer 1

Fork-join is more general than the sequence-based pmap/preduce, and should allow for more fine-grained control over parallelism. The exact APIs for doing this are still up in the air.

Answer 2

From looking at that code, their functionality will be mostly the same - the only difference is that pmap uses Futures running on the Agent threadpool as it's underlying primitive, while pvmap uses fork-join.

I'm not in a position to say for sure, but I'd expect that whichever one performs better in the general case would become the standard implementation for pmap, unless there are significant enough tradeoffs to make having both worthwhile.

It also looks like (for now at least) the fork-join framework only supports vectors, so it's not semi-lazy like pmap.

Answer 3

These slides contain some charts showing comparisons between the two approaches: http://data-sorcery.org/2010/10/23/clojureconj/