Am I using Julia right?

Question

I started using Julia a couple of months ago, deciding to give it a try after weeks of hearing people praise various features of it. The more I learned about it, the more I liked its style, merging ease of expressing concepts in high-level language with a focus on speed and usability. I implemented a model that I'd also written in C++ and R in Julia, and found the Julia version ran much faster than the R version, but still slightly slower than C++. Even so the code was more legible in Julia than it was in both other languages. This is worth a lot, and in particular as I've generalized the model, the amount of work to do to broaden the scope of the Julia code was far less than the comparable amount of worked threatened to be in the other languages.

Lately, I have been focused on getting my Julia code to run faster, because I need to run this model trillions of times. In doing so I've been guided by @code_warntype, @time, @profile and ProfileView and the track-allocation flag. Great. This toolkit isn't as nice as some other languages' profiling tooling, but it still pointed out a lot of bottlenecks.

What I find is that I have in my code precisely the high-level expressivity that I like in Julia, and when I rewrite that expressivity to avoid unnecessary allocations, I lose that very expressivity. As a trivial example, I recently changed a line of code that read

sum([x*y for x in some_list, y in similar_list])

to a loop that iterates over the lists and adds to a state variable. Not rocket science, and I understand why it's a little faster not to have to allocate the array. Actually it's a lot faster though. So I've done similar things with avoiding using Dicts or the composite types that "feel" right for the problem, when I can instead just manually keep track of indices in temporary parallel arrays, a style of coding I abhor but which evidently runs a lot faster when I'm repeating the particular operation of creating and briefly using a small data structure lots and lots of times.

In general, this is largely fine because I've taken to heart the instructions to write short methods, so the higher-level methods that compose behavior of my own shorter methods don't need to "worry" about how the shorter methods work; the shorter ones can be clunky to read without making the core of my program clunky to read.

But this has me wondering if I'm "missing something." If the whole point of the language (for me as a not-theory-concerned end user) is in part to combine speed with ease of development/thought/reading/maintenance etc., i.e. to be a writable and usable technical computing language, then doesn't that mean that I shouldn't ever be spending time thinking about the fastest way to add up a bunch of numbers, shouldn't be rewriting "easy to read" or elegant code that takes advantage of mappings, filters, and high-level function concepts into "clunky" code that reinvents the wheel to express those things in terms of low-level keeping track of array indices everywhere? (Some part of me expects a language with so much intelligence behind its design to be smart enough to "figure out" that it doesn't actually need to allocate a new array when I write sum([x*y]), and that I'm just too obtuse to figure out the right words to tell the language that except by literally "telling it" the whole loop business manually. At one point I even thought about writing @macros to "convert" some quickly expressed code into the long but faster loopy expressions, but I figured that I must be thinking about the problem wrong if I'm essentially trying to extend the compiler's functions just to solve fairly straightforward problems, which is what led me to write this question.)

Perhaps the answer is "if you want really performant code, you pay the price no matter what." Or put differently, fast code with annoyingly unpleasant to read loops, arrays, keeping track of indices, etc. is on the efficient frontier of the speed-legibility tradeoff space. If so, that's perfectly valid and I would not say therefore that I think any less of Julia. I'm just wondering if this sort of programming style really is on the frontier, or if my experience is what it is because I'm just not programming "well" in the language. (By analogy, see the question What is your most productive shortcut with Vim? , where the accepted and excellent answer is essentially that the OP just didn't "get" it.) I'm suspicious that even though I've succeeded in getting the language to do much of what I want, that I just don't "get" something, but I don't have a good sense of what to ask for as the thing I fear I don't get is an unknown unknown to me.

TL;DR: Is it expected for best-practice in Julia that I spend a lot of time "decomposing" my high-level function calls into their primitive implementations in terms of loops and arrays in order to get faster performance, or is that indicative of me not thinking about programming in / using the language correctly?

Answer 1

I think this topic is closely in a line with a discussion that was already going on the Julia-users group Does Julia really solve the two-language problem? and I would like to cite here one paragraph from that discussion:

@Stefan Karpinski:

There are different styles and levels of writing code in every language. There's highly abstract C++ and there's low-level pointer-chasing C++ that's basically C. Even in C there's the void*-style of programming which is effectively dynamically typed without the safety. Given this fact, I'm not sure what solving the two language problem would look like from the perspective this post is posing. Enforcing only one style of programming sounds like a problem to me, not a solution. On the contrary, I think one of Julia's greatest strengths is its ability to accommodate a very broad range of programming styles and levels.

My own experience in Julia programming, shows that it can fill the blank box of a modern programming language that could bring high level features like parallel processing, socket server and ... in the hand of scientists, engineers and all computation gurus and pragmatic programmers that want to do their work in an efficient, maintainable and readable manner, using an all-in-one programming language.
In my opinion you are using Julia in the right way, Julia like other languages represents different styles of programming for different situations, you can optimize bottlenecks for speed, and keep the other parts more readable. also you could use tools like Devectorize.jl to avoid rewriting issue.

Answer 2

This is a difficult question to answer satisfactorily. Therefore I'll try to make a short contribution in the hope that a "pot" of different opinions might be good enough. So here are 3 opinions going through my mind currently:

1. A programming language may be likened to an object with momentum. The user-base being its mass and their styles exerting force on it. The initial users/developers can pull a language in a certain direction because it still has low mass. A language can still evolve even if extremely popular (e.g. C/C++), but it's harder. The future of Julia is still unwritten, but its promise is in the initial direction imbued by its creators and early users.

2. Optimization is best delayed until correctness is well tested. "Premature optimization is the root of all evil" (D. Knuth). It never hurts to remember this, again. So, keeping the code readable and correct is better until the optimization stage which may obfuscate bounded areas of the code.

3. The expression sum([x*y ...]) may require the compiler to be too clever, and it might be better to simply define a sumprod(x,y). This would allow sumprod to harness the generic function multiple dispatch framework of Julia and stay optimized for x and y and possibly later even more optimized for specifically typed x and y.

That's it, for now. Opinions are many. So let the discussion continue.