how to efficiently apply a medium-weight function in parallel

Question

I'm looking to map a modestly-expensive function onto a large lazy seq in parallel. pmap is great but i'm loosing to much to context switching. I think I need to increase the size of the chunk of work thats passed to each thread.

I wrote on a function to break the seq into chunks and pmap the function onto each chunk and recombine them. this 'works' but the results have not been spectacular. The origional code essentially looks like this:

(pmap eval-polynomial (range x) coificients)

How can I really squeez this while keeping it lazy?

Answer 1

How about using the partition function to break up your range sequence? There was an interesting post on a similar problem at http://www.fatvat.co.uk/2009/05/jvisualvm-and-clojure.html