What's the best way to run an expensive initialization?

Question

I've got a singleton that is expensive to initialize:

struct X {...};

const X&
get_X()
{
    static const X x = init_X();
    return x;
}

The first time get_X() is called, it can take hundreds of milliseconds to initialize the function-local static. But after that is done, the things I need to do with the X are relatively fast:

get_X().find_something_for_me();  // expensive if this is the first call
get_X().find_something_for_me();  // now fast

How can I minimize having a large delay the first time I call get_X()? I have plenty of cores...

Answer 1

As soon as your application starts, and (hopefully) before you actually need to call get_X(), gratuitously call it. Furthermore, so that your initialization stage is quicker, feed your expensive initializations off to different threads. For example:

#include <thread>

int
main()
{
    std::thread(get_X).detach();
    // continue with other initialization...
}

When some task is as expensive as several hundred milliseconds (or more), the overhead of spinning off a thread to deal with it is in the noise level. And if you are on multi-core hardware (what isn't these days?), then this is a clear performance win if your application doesn't actually need anything from this singleton until the initial call to get_X completes.

Notes/Questions:

• Why detach the thread? Why not join?

  If you decide to join this thread, that means you just have to wait for it to finish, why not do other things instead. When it finishes, detach has it clean up after itself. You don't even need to retain a handle to the thread. The temporary std::thread destructs, but the OS thread lives on, running get_X to completion.

  When thread was being standardized, there were viewpoints that detached threads were not only useless, but dangerous. But here is a perfectly safe, and quite motivating use case for detached threads.

• What if my application calls get_X() before the detached thread finishes the first call to get_X()?

  There is a performance hit, but not a correctness hit. Your application will block at this line in get_X():

  static const X x = init_X();

  until the detached thread is finished executing it. Thus there is no data-race.

• What if my application ends before the detached thread is complete?

  If your application ends during the initialization stage, something has evidently gone catastrophically wrong. If get_X touches something that is already destructed by the at_exit chain (which executes after main), bad things will happen. However, you are already in a state of panic shutdown ... one more emergency isn't likely to make your panic shutdown worse. You're already dead. Otoh, if your initialization is something that takes minutes to hours, you probably do need better communication about when your initialization is complete, and more graceful shutdown procedures. In that case you need to implement cooperative cancelation in your threads, detached or not (something the std committee declined to provide you with).

• What if the first call to get_X() throws an exception?

  In that case, the second call to get_X() has its chance to initialize the function local static, assuming you don't leave the exception uncaught and allow it to terminate your program. It may too throw, or it may succeed in initialization (that is up to your code). In any case, calls to get_X() will continue to try to initialize, waiting if initialization is in progress, until somebody manages to do so without throwing an exception. And this is all true whether or not the calls are coming in from different threads or not.

In summary

std::thread(get_X).detach();

is a good way to harness the power of your multiple cores to get independent expensive initializations out of the way as quickly as possible, without compromising thread-safety correctness.

The only downside is that you initialize the data within get_X() whether you need it or not. So be sure you will need it before using this technique.

---

[Footnote] For those using Visual Studio, this is good motivation to move to VS-2015. Prior to this version VS does not implement thread-safe function-local statics.