Why is there overhead when calling functions?

Question

Often, people speak of the calling of functions producing a certain amount of overhead, or an inescapable set of additional concerns and circumstances, in a program. Can this be better explained and compared to a similar program without the function call?

Answer 1

It depends on your compiler settings and the way it optimizes code. Some functions are inlined. Others are not. It usually depends on whether you're optimizing for size or for speed.

Generally, calling function causes delay for two reasons:

• The program needs to hook to some random location in memory where your function code starts. To do this, it needs to save the current cursor position into a stack so it knows where to return. This process consumes more than one CPU cycle.

• Depending on your CPU architecture, there may be a pipeline, which fetches the next few instruction from memory into the CPU cache in parallel with your current instruction execution. This is to speed up execution speed. When you call a function, the cursor hooks to a completely different address and all the cached instructions are flushed from the pipeline. This causes further delays.

Answer 2

Also see here and here for a discussion on when inlining would make sense.

• Inlining

  In general, you can only suggest to the compiler to inline a function, but the compiler might decide otherwise. Visual Studio offers its own forceinline keyword though. Some functions can not be inlined, e.g. when they are recursive or when the target function can not be determined at compile time (calls through function tables, virtual function calls in C++).

  I suggest you trust the compiler whether or not a function should be inlined. If you truly want to inline your code, consider using a macro instead.

• Overhead

  Memory overhead is at a minimum when you use functions because you do not duplicate code; inlined code is duplicated into the call site. Performance overhead these days is negligible because modern architectures are really good predicting and calling with about 1-2 cycle overhead only.