How does a compiler decide whether it's worth making my functions inline or not?

Question

I am wondering if anyone knows the logic which C++ compilers generally use to decide whether or not to inline a function at compilation (assuming inline has been requested).

Is this type of thing public knowledge?

Answer 1

I gave a more thorough answer in this other question.

Basically compilers have heurstics based on cost-analysis, quoting myself (does this mean I am going senile ?)

If you think about inlining, and its consequences, you'll realise it:

• you avoid a function call (with all the register saving/frame adjustment)
• you expose more context to the optimizer (dead stores, dead code, common sub-expression elimintation...)
• at the cost of duplicating code (bloating the instruction cache and the executable size, among other things)

And of course, there is also partial inlining in which only part of the function is inline, typically a leading if guard like foo(T* t) { if (!t) { return; } <many many things> }.

Answer 2

Yes, this kind of information is public knowledge. Especially since there are tons of open source compilers out there.

I believe the book to read is the Dragon Book. They go through all of this there, non?

But basically: A function call has a cost - setting up the registers, jumping, collecting the results. This can be counted in cycles. The function body also has a cost measured in cycles. Compare the two. Extra points for taking cache locality into account.

Answer 3

I think compilers use some kind of heuristic to decide on whether or not to inline a function.

If the function is called from one place only, inlining it may shave off a few instructions and cycles, improving the size and speed.

If it's a tiny function, inlining can do the same.

If the function is called in a tight loop many times, inlining can improve performance.

At the same time inlining can result in bigger code overall and may also have some negative effects on performance due to caching of more code.

Compilers try to make optimal choices, but they also have to take into account things like configurable optimization options (the programmer may be able to say that he prefers speed over size or the opposite) and how much time it would take them to make these decisions and generate the code (you don't want the compiler to heavily optimize everything and take many extra hours or days to compile your code). There are tradeoffs.

Also, compilers aren't almighty. They just can't put certain twos and twos together, not because that's expensive, but because it's almost impossible. This is why we do performance and load testing, have profilers and have humans write code because humans are still generally more intelligent than their programs.

Answer 4

The heuristics will depend on the options you pass the compiler, and will depend on the compiler as well. There's no general answer. Most compilers today have combinations of options where they will inline nothing. Most also have a combination of options where what they inline will depend on the profiler output of earlier runs of the program. Otherwise: from a quality of implementation point of view, it is reasonable to expect the compiler to inline functions declared inline more aggressively than those not declared inline. And it will usually be only when the most aggressive optimization has been activated that they will inline a function whose definition is not present in the translation unit. Finally, certain things in the function may inhibit inlining: some compilers might not inline recursive functions, for example (although g++ does, at least in certain cases).