pmap slow for toy example

Question

I'm testing out parallelism in Julia to see if there's a speedup on my machine (I'm picking a language to implement new algorithms with). I didn't want to dedicate a ton of time writing a huge example so I did the following test on release version Julia 0.4.5 (Mac OS X and dual core):

$ julia -p2

julia> @everywhere f(x) = x^2 + 10
julia> @time map(f, 1:10000000)
julia> @time pmap(f, 1:10000000)

pmap is significantly slower than map (>20x) and allocates well over 10x the memory. What am I doing wrong?

Thanks.

Answer 1

That's because pmap is intended to do heavy computations per core, not many simple ones. If you use for something simple as your function, the overhead of moving along the information along proccesors is bigger that the benefits. Instead, test this code (I run it with 4 cores in a i7):

function fast(x::Float64)
    return x^2+1.0
end

function slow(x::Float64)
    a = 1.0
    for i in 1:1000
        for j in 1:5000
            a+=asinh(i+j)
        end
    end
    return a
end

info("Precompilation")
map(fast,linspace(1,1000,1000)) 
pmap(fast,linspace(1,1000,1000))
map(slow,linspace(1,1000,10)) 
pmap(slow,linspace(1,1000,10))

info("Testing slow function")
@time map(slow,linspace(1,1000,10)) #3.69 s
@time pmap(slow,linspace(1,1000,10)) #0.003 s
info("Testing fast function")
@time map(fast,linspace(1,1000,1000)) #52 μs
@time pmap(fast,linspace(1,1000,1000)) #775 s

For parallelization of a lot of very small iterations, you can use @parallel, search for it in the documentation.