Memory allocation in Julia function

Question

Here is a simple function in Julia 0.5.

function foo{T<:AbstractFloat}(x::T)
  a = zero(T)
  b = zero(T)
  return x
end

I started with julia --track-allocation=user. then include("test.jl"). test.jl only has this function. Run foo(5.). Then Profile.clear_malloc_data(). foo(5.) again in the REPL. Quit julia. Look at the file test.jl.mem.

        - function foo{T<:AbstractFloat}(x::T)
        -   a = zero(T)
   194973   b = zero(T)
        0   return x
        - end
        - 

Why is there 194973 bytes of memory allocated here? This is also not the first line of the function. Although after Profile.clear_malloc_data(), this shouldn't matter.

Answer 1

Let's clarify some parts of the relevant documentation, which can be a little misleading:

In interpreting the results, there are a few important details. Under the user setting, the first line of any function directly called from the REPL will exhibit allocation due to events that happen in the REPL code itself.

Indeed, the line with allocation is not the first line. However, it is still the first tracked line, since Julia 0.5 has some issues with tracking allocation on the actual first statement (this has been fixed on v0.6). Note that it may also (contrary to what the documentation says) propagate into functions, even if they are annotated with @noinline. The only real solution is to ensure the first statement of what's being called is something you don't want to measure.

More significantly, JIT-compilation also adds to allocation counts, because much of Julia’s compiler is written in Julia (and compilation usually requires memory allocation). The recommended procedure is to force compilation by executing all the commands you want to analyze, then call Profile.clear_malloc_data() to reset all allocation counters. Finally, execute the desired commands and quit Julia to trigger the generation of the .mem files.

You're right that Profile.clear_malloc_data() prevents the allocation for JIT compilation being counted. However, this paragraph is separate from the first paragraph; clear_malloc_data does not do anything about allocation due to "events that happen in the REPL code itself".

Indeed, as I'm sure you suspected, there is no allocation in this function:

julia> function foo{T<:AbstractFloat}(x::T)
         a = zero(T)
         b = zero(T)
         return x
       end
foo (generic function with 1 method)

julia> @allocated foo(5.)
0

The numbers you see are due to events in the REPL itself. To avoid this issue, wrap the code to measure in a function. That is to say, we can use this as our test harness, perhaps after disabling inlining on foo with @noinline. For instance, here's a revised test.jl:

@noinline function foo{T<:AbstractFloat}(x::T)
    a = zero(T)
    b = zero(T)
    return x
end

function do_measurements()
    x = 0.  # dummy statement
    x += foo(5.)
    x       # prevent foo call being optimized out
            # (it won't, but this is good practice)
end

Then a REPL session:

julia> include("test.jl")
do_measurements (generic function with 1 method)

julia> do_measurements()
5.0

julia> Profile.clear_malloc_data()

julia> do_measurements()
5.0

Which produces the expected result:

        - @noinline function foo{T<:AbstractFloat}(x::T)
        0     a = zero(T)
        0     b = zero(T)
        0     return x
        - end
        -
        - function do_measurements()
   155351     x = 0.  # dummy statement
        0     x += foo(5.)
        0     x       # prevent foo call being optimized out
        -             # (it won't, but this is good practice)
        - end
        -