Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Speeding up package load in Julia

I wrote a program to solve a linear program in Julia using GLPKMathProgInterface and JuMP. The Julia code is being called by python program which runs multiple instances of the Juila code through multiple command line calls. While I'm extremely happy with the performance of the actual solver the initialization is extremely slow. I was wondering if there were approaches to speed this up.

For example if I just save the following to a file

@time using DataFrames, CSV, GLPKMathProgInterface, JuMP, ArgParse

and run it

mylabtop:~ me$ julia test.jl 
12.270137 seconds (6.54 M allocations: 364.537 MiB, 3.05% gc time)

This seems extremely slow, is there some good way to speed up using modules like a precompile step I could do once?

like image 591
Ryan F Avatar asked Apr 03 '18 16:04

Ryan F


People also ask

What makes Julia fast?

Many people believe Julia is fast because it is Just-In-Time (JIT) compiled (i.e. every statement is run using compiled functions which are either compiled right before they are used, or cached compilations from before).

How do you Precompile Julia?

To create an incremental precompiled module file, add __precompile__() at the top of your module file (before the module starts). This will cause it to be automatically compiled the first time it is imported. Alternatively, you can manually call Base. compilecache(modulename) .


1 Answers

Since you haven't gotten any answers yet, let me give you the general first order answers - although I hope someone more qualified will answer your question in more detail (and correct me if I'm wrong).

1) Loading packages in Julia is sometimes rather slow up to the time of this writing. It has been discussed many times and you can expect improvements in the future. AFAIK this will happen in early 1.x releases after 1.0 is out. Have a look at this thread.

2) Since you typically only have to pay the loading time cost once per Julia session one approach is to keep the session running for as long as possible. You can execute your script with include("test.jl") from within the session. Let me also mention the amazing Revise.jl - it's hardly possible to overemphasize this package!

3) (I have no experience with this more difficult approach.) There is PackageCompiler.jl which allows you to compile a package into your system image. Read this blog post by Simon.

4) (Not recommended) There has also been the highly experimental static-julia which statically compiles your script into an shared library and executable.

Hope that helps.

like image 191
carstenbauer Avatar answered Sep 30 '22 10:09

carstenbauer