Unpacking return of map in Julia

Question

I have a function that returns an array. I'd like to map the function to a vector of inputs, and the the output to be a simple concatenation of all the arrays. The function is:

function log_it(r, bzero = 0.25, N = 400)
    main = rand(Float16, (N+150));
    main[1] = bzero;
    for i in 2:N+150
        main[i] = *(r, main[i-1], (1-main[i-1]))
    end;
    y = unique(main[(N+1):(N+150)]);
    r_vec = repeat([r], size(y)[1]);
    hcat(r_vec, y)
end;

and I can map it fine:

map(log_it, 2.4:0.001:2.405)

but the result is gross:

 [2.4 0.58349609375]
 [2.401 0.58349609375]
 [2.402 0.583984375; 2.402 0.58349609375]
 [2.403 0.583984375]
 [2.404 0.583984375]
 [2.405 0.58447265625; 2.405 0.583984375]

NB, the length of the nested arrays is unbounded - I'm looking for a solution that doesn't depend on knowing the the length of nested arrays in advance.

What I want is something like this:

 2.4    0.583496
 2.401  0.583496
 2.402  0.583984
 2.402  0.583496
 2.403  0.583984
 2.404  0.583984
 2.405  0.584473
 2.405  0.583984

Which I made using a for loop:

results = Array{Float64, 2}(undef, 0, 2)
    for i in 2.4:0.001:2.405
        results = cat(results, log_it(i), dims = 1)
    end
    results

The code works fine, but the for loop takes about four times as long. I also feel like map is the right way to do it and I'm just missing something - either in executing map in such a way that it returns a nice vector of arrays, or in some mutation of the array that will "unnest". I've tried looking through functions like flatten and collect but can't find anything.

Many thanks in advance!

Answer 1

Are you sure you're benchmarking this correctly? Especially with very fast operations benchmarking can sometimes be tricky. As a starting point, I would recommend to ensure you always wrap any code you want to benchmark into a function, and use the BenchmarkTools package to get reliable timings.

There generally shouldn't be a performance penalty for writing loops in Julia, so a 3x increase in runtime for a loop compared to map sounds suspicious.

Here's what I get:

julia> using BenchmarkTools

julia> @btime map(log_it, 2.4:0.001:2.405)
121.426 μs (73 allocations: 14.50 KiB)

julia> function with_loop()
           results = Array{Float64, 2}(undef, 0, 2)
           for i in 2.4:0.001:2.405
               results = cat(results, log_it(i), dims = 1)
           end
          results
       end

julia> @btime with_loop()
173.492 μs (295 allocations: 23.67 KiB)

So the loop is about 50% slower, but that's because you're allocating more.

When you're using map there's usually a more Julia way of expressing what you're doing using broadcasting. This works for any user defined function:

julia> @btime log_it.(2.4:0.001:2.405)
121.434 μs (73 allocations: 14.50 KiB)

Is equivalent to your map expression. What you're looking for I think is just a way to stack all the resulting vectors - you can use vcat and splatting for that:

julia> @btime  vcat(log_it.(2.4:0.001:2.405)...)
122.837 μs (77 allocations: 14.84 KiB)

and just to confirm:

julia> vcat(log_it.(2.4:0.001:2.405)...) == with_loop()
true

So using broadcasting and concatenating gives the same result as your loop at the speed and memory cost of your map solution.