Set CPU affinity when create a thread

Question

I want to create a C++11 thread which I want it to run on my first core. I find that pthread_setaffinity_np and sched_setaffinity can change the CPU affinity of a thread and migrate it to the specified CPU. However this affinity specification changes after the thread has run.

How can I create a C++11 thread with specific CPU affinity (a cpu_set_t object)?

If it is impossible to specify the affinity when initializing a C++11 thread, how can I do it with pthread_t in C?

My environment is G++ on Ubuntu. A piece of code is appreciated.

Answer 1

I am sorry to be the "myth buster" here, but setting thread affinity has great importance, and it grows in importance over time as the systems we all use become more and more NUMA (Non-Uniform Memory Architecture) by nature. Even a trivial dual socket server these days has RAM connected separately to each socket, and the difference in access to memory from a socket to its own RAM to that of the neighboring processor socket (remote RAM) is substantial. In the near future, processors are hitting the market in which the internal set of cores is NUMA in itself (separate memory controllers for separate groups of cores, etc). There is no need for me to repeat the work of others here, just look for "NUMA and thread affinity" online - and you can learn from years of experience of other engineers.

Not setting thread affinity is effectively equal to "hoping" that the OS scheduler will handle thread affinity correctly. Let me explain: You have a system with some NUMA nodes (processing and memory domains). You start a thread, and the thread does some stuff with memory, e.g. malloc some memory and then process etc. Modern OS (at least Linux, others probably too) do a good job thus far, the memory is, by default, allocated (if available) from the same domain of the CPU where the thread is running. Come time, the time-sharing OS (all modern OS) will put the thread to sleep. When the thread is put back into running state, it may be made runnable on any of the cores in the system (as you did not set an affinity mask to it), and the larger your system is, the higher the chance it will be "woken up" on a CPU which is remote from the memory it previously allocated or used. Now, all your memory accesses would be remote (not sure what this means to your application performance? read more about remote memory access on NUMA systems online)

So, to summarize, affinity setting interfaces are VERY important when running code on systems that have more-than-trivial architecture -- which is rapidly becoming "any system" these days. Some thread runtime environments/libs allow for control of this at runtime without any specific programming (see OpenMP, for example in Intel's implementation of KMP_AFFINITY environment variable) - and it would be the right thing for C++11 implementers to include similar mechanisms in their runtime libs and language options (and until then, if your code is aimed for use on servers, I strongly recommend that you implement affinity control in your code)