Memory layout optimization in C#

Question

Disclaimer : there might be some misconceptions in the phrasing below, please correct me if i misinterpreted the wy my code is handled in C# between the moment i write it to the point it looks like zeroes and ones

The questions are the following (those are linked) :

Is there any way in C# where my Data structures and/or my data manipulation implementation, will have a performance impact whether I use optimization techniques or not?

What does the compiler do when outputting IL, is it reliable? Meaning : if i make my data SOA will it be SOA in IL? Always?

What happens to my data structure when the JIT reads the IL? is it changed? is it optimized automatically to fit my processor?

cf: that talk about C/C++

I know that this talk is targeted at native code, and talks about the specifics of the processor layout vs your data layout in native code.

I also know the C# compiler and the JIT compiler will optimize stuff for me in regards of those issues.

Basically I wonder if those kind of optimization will have an impact on my perfs :

• SOA instead of AOS
• Vectors access patterns (to be accessed contiguously in memory)
• etc... you name it...

I work in game development and performance is critical, we manipulate large amounts of data and we need to do that minimum 24 times per second, I cannot have the GC do stuff for 300ms or memory to be accessed/allocated all over the place when I'm trying to detect collisions between 3000 different objects

For reference about stuff i read but did not really answer the question :

• Excellent Eric Lippert article about structs and values types in C# (please read it if you think values types are always on the stack in C# you're in for a treat)

• Excellent video about PerfView to track your GC behaviour and it's impact on your perfs

• That SO question about Best practices to optimize memory in C# (and more importantly it's answer)

But those do not answer the performance cost relative to processor and data layout implementation.

---

To go further after what Hans answered :

When you say : "You can pursue SOA but that doesn't help. Yes, your program will slow down because of all that structure copying and does so in a deterministic way. But it doesn't stop the rain. You get the worst of both, a slow program and the exact same pauses."

It does not mean that my program benefits nothing from SOA it WILL be faster (potentially) because it will help processing my data. Just that it will have no impact on the GC in itself.

Other thing is that If i do not do SOA or other improvements in my data layout, the compiler won't improve that for me right? i cannot rely on the compiler to deal with that kind of things?

Answer 1

Worrying about the GC is like worrying about whether it is going to rain today. It is going to rain sooner or later, nothing you can do to stop it. And it is required, you can't keep that lawn look pretty green if it doesn't. What you never want to do is intentionally stop it from raining. Because if you do it will come down in a deluge, spilling away that pretty lawn. A steady drizzle is what you want. And preferably at night when you're not looking.

The .NET GC strongly supports this. Only the small gen #0 and #1 collections will pause your program. The expensive gen#2 collection happens in the background while your code continues executing. Worst-case pause hovers somewhere near a hundred microseconds. Which is pretty indistinguishable from other reasons your program will pause on a modern OS. Like your game loop getting suspended temporarily because another higher priority kernel thread needs to run. Just a drizzle, unobservable to the human eye.

You can pursue SOA but that doesn't help. Yes, your program will slow down because of all that structure copying and does so in a deterministic way. But it doesn't stop the rain. You get the worst of both, a slow program and the exact same pauses.

Don't worry about the rain, just make sure it comes down at the right time. To take advantage of background GCs you want to structure your data so it is either very short lived, so it disappears easily with a gen #0/1 collection. Or lives for very long so it finds a comfortable home in gen #2 and stays there for a while. Which in general is a very common pattern in programs, especially so in games. Pretty unlikely you need to do anything at all.