When is sizeof(myPOD) too big for pass by value on x64?

Question

I expect no difference when it comes to structures that are up to 8 bytes in size, but what about bigger POD types? Does pass by value become more expensive the moment the type's size exceeds machine word size or is there something else (like cache line size) that can affect the performance?

I'm mainly interested in x64, but feel free to include some numbers for x86 as well.

Clarifications:

• I'm probably thinking too narrowly because I'm not aware of everything that plays a role in this (registers, calling conventions, compiler optimizations). I'm mainly interested in Microsoft's C++ compiler and it only uses __fastcall.
• I'm interested if there is any kind of general recommendation when it comes to parameter passing knowing the architecture, type size, cache size, etc. Something like: "Prefer passing the type by value when it's smaller than N bytes." where N is something that can be derived from the things we know.

Answer 1

You're confusing two separate issues. You can pass any object by value (as long as it is copyable).

Whether or not it will be passed in a register or on the stack depends on the implementation and specifically, the calling convention used.

Under some calling conventions, parameters larger than 8 bytes (the general-purpose register size) will be passed on the stack. Under other calling conventions, they may simply be split across several registers.

Under some, it is possible that objects are never passed in registers, regardless of their size.

Similarly, SIMD values (SSE/AVX) may be passed in registers in some calling conventions, but will always be put on the stack in others. And the same may be true for scalar floating-point values.

But what you're asking can't really be meaningfully answered. The speed of copying an object is affected by the object's size, yes. If the object is a POD type, and it fits in a register, then it can be copied with a simple mov instruction. Whether or not the compiler will do that is up to the compiler.

And obviously, the large the object is, the more cache space it takes up, which means you'll get more cache misses.

But this is all so vague that it is next to useless. We don't know what your object looks like, and we don't know what your code does with it. If you have a specific type in mind, then write a benchmark to see how it is handled by the compiler.

In response to your edit

I'm interested if there is any kind of general recommendation when it comes to parameter passing knowing the architecture, type size, cache size, etc. Something like: "Prefer passing the type by value when it's smaller than N bytes.

First, trust your compiler. It will aggressively optimize copies away in many situations, so even if you do pass a large object by value, it's unlikely to be a measurable problem.

Second, you're looking at a microoptimization which is unlikely to make a noticeable difference either way. For small objects, passing by value avoids a pointer indirection, so it's probably slightly faster. At some point, this becomes overwhelmed by the cost of copying (assuming the object is copied, see above). For very large objects (for the sake of argument, let's say 500 bytes or above, so large that objects normally don't reach it), you should definitely pass by reference.

But for objects of 8, 16, 24, 40 bytes? Who knows? Who cares? It's unlikely to make a measurable difference in real code.

Which leads me to the two rules of thumb:

1. do what seems natural: if passing by copy makes your code simpler or cleaner, do that.
2. if performance matters, then (1) make sure that what you're looking at actually has any noticeable impact on your performance at all. Measure it. If it affects performance, then it can be measured. If it can't be measured, then the difference in performance, by definition, cannot be noticeable.

So, in short:

• for primitive types, pass by value.
• for very large types, pass by reference.
• for everything else, stop worrying and spend your time on something productive.