Why not allocate and deallocate memory frequently in C++ games?

Question

I'm a recent convert to C++ for game programming - I have a lot of experience dealing with memory management and garbage collection woes in C#, but not as much with C++.

I've heard some vague advice in the past to avoid allocations and deallocations during gameplay (i.e. new and delete) and to pre-allocate everything you might need up front. But that's a lot more tedious and architecturally complex than just allocating and freeing game objects as needed while the game's running (enemies, particles, etc.).

I think the advice I read was referring to resource-constrained platforms - I'm aiming to develop mostly for PC, and I imagine the game state data that would be changing frequently would be on the order of a few megabytes at most. The rest are textures, sound assets, etc. that I'll be preloading.

So my question is: in a world of PCs with gigabytes of memory, is it worth the headache to set up elaborate memory pooling, pre-allocation, and so forth for my game state data? Or is this just some unquestioned "best practice" tradition that evolved when maxing out limited platforms, that is now repeated as gospel?

If my 2 MB of game data gets fragmented and is now spread over 4MB, I can't imagine that mattering in the slightest on a PC made after 1990 - but curious to know if I'm missing something :).

Answer 1

The main reasons to avoid calling new unless necessary in a game environment are that

1. Dynamic allocation of memory really is surprisingly expensive.
2. Cache misses are detrimental to performance.

Dynamic Allocation

At my work, we develop a game-like product (virtual surgery) and most of our memory is pre-allocated and handled via factories or memory pools. This is done because, dynamically allocating memory takes a very long time. The system has to deal with memory requests of many different sizes at any and all times. This means there's a lot of work going into processes such as minimizing fragmentation. If you ask the system for memory, you'll have to wait for it to do these things. If you pre-allocate memory, you can use factories or other block-size-specific memory managers to alleviate these concerns.

I can tell you from experience that a simple mistake like allocating a reasonably large std::vector from scratch every frame, instead of reusing pre-allocated memory, can drag the frame rate down into the gutter.

Cache Misses

Another related issue is cache coherence. Cache misses, which force the OS to bring a new page into the cache, are also very expensive. If this happens often, you'll have an unplayable game. If, however, you pre-allocate large chunks of memory, this can go a long way towards improving cache locality, which makes cache misses few and far between.

Moral of the story

So, in short: If you don't manage your own pre-allocated memory, you can expect a lot of your computational time to be lost to waiting for the system to allocate memory or handle cache misses.