Computationally efficient C++ - general reading [closed]

Question

My job is mainly in high-performance 'scientific' computing. I've been doing that for ~15 years now, but only recently realized that my software wastes computational time. In short words: my ways of writing efficient C++ code no longer work.

From time to time I see a piece of code, written by some kid, that does basically the same calculations as mine (same algorithm, similar approach), but - magically! - performs far faster. In most cases I'm even unable of tracking down the origins of the difference!

My question is: how can I learn the art of modern C++ code optimization? Perhaps something on SSE, caching/mem alignment issues? Any suggestion of book, PDF, article, exercise or website is welcome!

PS. I'm well aware of tricks that are either:

• Too general (e.g. 'Use profiler', 'Use good algorithms', 'Go multithreaded')
• Trivial (e.g. 'Avoid virtual functions', 'Do ++i instead of i++', 'Enable -O3')
• Questionable (e.g. 'Reuse memory with reinterpret_cast<>', 'Tabularize sine and cosine', 'Write inline assembly')
• Ridiculous (e.g. 'Do template metaprogramming')

These are not what I'm asking about.

Answer 1

I too work in scientific computation though for rather longer than OP and mainly in Fortran. here's a little advice from my experience;

1) Keep up to date with what compiler(s) can do. On the one hand don't try to beat the compiler at optimisation tricks that the compiler knows about, on the other, know what compilers still aren't good at. For example, right now I think I can do a better job than my compiler at loop tiling. Learn too how to make it easy for the compiler to optimise code.

OP will be tempted to pass this point off as an example of advice which is too general to be of use. I see that the Intel C++ compiler manual has about 800 pages of documentation of the compiler options, and a further 400 on optimizing applications. Has OP read all this (or similar quantity of documentation for preferred compiler) ?

2) Keep up to date with computer architecture, in particular with the design of the memory hierarchy and of the fpus. If nothing else, this helps to understand what the limits of performance one can reasonable expect might be. But it also provides input to decisions on program design and implementation, and indications of how those decisions ought to change when programs are moved to the next generation of hardware.

3) Use libraries. Write code as a last resort.

4) Don't pooh-pooh ideas such as template metaprogramming which have a very good reputation for helping the programmer to create fast code. Study Boost and Blitz.

5) Program performance is an empirical discipline. Believe only data, not argument. Not even argument made by me.

Finally, even in large-scale high-performance computing (my largest jobs run for days on 10K CPUs and more so I have a little knowledge of this), sometimes the activity to optimise is development time, not execution time.

PS Did you ask the kid for instruction ?

Answer 2

Processors are much faster than they were 15 years ago. Memory has not incrased in speed at the same rate. This combined with larger data sets, particularly in big scientific simulations, means you have to think a bit more carefully about how the data is accessed. That is maybe one of the differences.

I found these articles interesting:

http://overbyte.com.au/2011/10/21/optimisationmasterclass1/

http://overbyte.com.au/2011/11/10/optimisation-lesson-2/

They are written by a guy I know who wrote games engines and who now optimizes PS3 games. You might find them useful.