Why julia takes long time to import a package?

Question

I am quite concern about performance. So, I am creating this as a question regarding the latency while calling or importing a package at first time. It might be a silly question.

When the first time I add package for ex, Plots, it consumes some amount of time to build the package. Again when i import the package first time ever on my notebook that also took some time (~1 min) says Precompiling message After importing the package, when I hit plot() this also consumes some time (30s - 60s) and finally returns a plot.

Once I used plot function, whenever I use next time it's not taking much time to produce result.

This latency happens whenever I restart a notebook.

I guess it's compiling functions before execution. Because unlike python, julia is not a scripting language. So, it supposed to undergo compilation. But, Why the latency occurs every time when I restart notebook?

Is there anyway I can suppress this latency? Is there anyway I can precompile everything once so that next time on wards I don't see any latency without worrying about kernal restart in notebook or in Julia Terminal. Why does the latency happen? Is it fully because of compilation time or it depends on my machine?

Answer 1

You can do two things to reduce the latency:

1. Use Julia 1.5.0 (beta as of today) - it dynamically allows to use different optimization levels for packages and Plots.jl is making use of that reducing the time-to-first-plot by more than half. This is indeed a low hanging fruit - update the Julia. Here are the measurements on my laptop:

julia> @time using Plots
 16.816181 seconds (14.46 M allocations: 854.353 MiB, 2.31% gc time)

julia> @time Plots.plot(sin.(1:0.25:7))
  4.292128 seconds (4.70 M allocations: 243.866 MiB, 2.01% gc time)
  # this waits another 7s before the plot actually appears on screen

Those times are not excellent but acceptable.

2. Build Plots into your system image (for details see https://julialang.github.io/PackageCompiler.jl/dev/examples/plots/)

using PackageCompiler
create_sysimage(:Plots, sysimage_path="sys_plots.so", precompile_execution_file="precompile_plots.jl")

This reduces your time-to-first-plot below half second. Has also a disadvantage that the Julia interpreter (REPL) takes 400ms more time to start and you need to use the flag --sysimage sys_plots.so (or --sysimage sys_plots.dll on Windows) when starting Julia. Precompiling packages can sometimes bring other caveats too (e.g. package updating each time requires recompiling etc.).

Answer 2

It is fully because of the compilation and optimization julia does. You can change the optimization level. This could bring up your first plot a bit faster. But perhaps also not. Smaller levels are less optimized. To do so start julia with the -O, --optimize={0,1,2,3} option, default is 2:

$ julia -O=1

Time to first plot is a issue touching many people. By the nature of the topic, it has to be addressed by many volunteers.