Are there functional programming languages that run on the GPU?

Question

Using the traditional, sequential reduction approach, the following graph is reduced as:

(+ (+ 1 2) (+ 3 4)) ->
(+ 3 (+ 3 4)) ->
(+ 3 7) ->
10

Graph reductions are, though, inherently parallel. One could, instead, reduce it as:

(+ (+ 1 2) (+ 3 4)) ->
(+ 3 7) ->
10

As far as I know, every functional programming language uses the first approach. I believe this is mostly because, on the CPU, scheduling threads overcompensate the benefits of doing parallel reductions. Recently, though, we've been starting to use the GPU more than the CPU for parallel applications. If a language ran entirely on the GPU, those communication costs would vanish.

Are there functional languages making use of that idea?

Answer 1

What makes you think on GPU scheduling would not overcomponsate the benefits?

In fact, the kind of parallelism used in GPUs is far harder to schedule: it's SIMD parallelism, i.e. a whole batch of stream processors do all essentially the same thing at a time, except each one crushes a different bunch of numbers. So, not only would you need to schedule the subtasks, you would also need to keep them synchronised. Doing that automatically for general computations is virtually impossible.

Doing it for specific tasks works out quite well and has been embedded into functional languages; check out the Accelerate project.