Cannot use more than half of total available thread in CPU

Question

I am testing a program on a server, which is running 64bit Windows Server 2008 R2 Enterprise, and it has 4 Intel E7-4870 processors, total 40 cores and 80 available threads (I can see the 80 CPU usage graphs in the Windows Task Manager).

The program code is like:

numlist is List contains hundreds of numbers, each one is a parameter to be used in some calculation

Parallel.ForEach(numlist, num =>
                 {
                    // do some calculation using parameter = num             
                 });

The issue is that when I run this progrm on the server, only half of the available threads are shown to be used in windows task manager (of course CPU usage is shown 50%), and the rest of 40 are all totally unused and idling.

I also tested the same program on another server which only has 2 processors and total 24 available threads, and all 24 threads will be fully used and CPU usage is shown as 100%.

Is there any way I can make the 40 core - CPU server to run this program and fully utilized all its avaialble 80 threads (or close to 80 threads)? The performance is not good enough when only 50% of CPU resource is used.

---

Here is the full program code I am testing:

namespace Test
{
    internal class Program
    {
        private static void Main(string[] args)
        {
            Console.WriteLine("Press any key to start");
            Console.ReadLine();
            List<int> numlist = new List<int>();
            for (int i = 0; i < 100; i++)
            {
                numlist.Add(i);
            }

            Parallel.ForEach(numlist, num =>
                                 {
                                 while (true)
                                 {
                                     num++;
                                 }
                             });

        }
    }
}

when it is run on a server with 2 Intel X5690 processors (total 24 threads available), all of the 24 threads are used and CPU usage is shown as 100%;

But when I run it on the 4 processors server with 80 threads available, only 40 of the threads are used and CPU usage is only 50%. Is there any compiler settings related to this?

Answer 1

Depending on the type of work, hyper threading does not always help. With many types of pure math operations, each core can only process one work item effectively, not 2 as suggested by the processor "thread count".

Hyper threads aren't actually separate cores, so the instructions that run on them do not always lead to gains. This is discussed here:

Depending on the cluster configuration and, most importantly, the nature of the application running on the cluster, performance gains can vary or even be negative. The next step is to use performance tools to understand what areas contribute to performance gains and what areas contribute to performance degradation.

Hyper threading tends to lead to around a 30% increase in overall performance in the best case. For this to work, you typically need different CPU instructions pushing through each thread on the core, so the core can perform the work properly. When running the same calculation in parallel across many hyper threaded "CPU threads", you often will see no advantage vs. one process running per core.

This may also be due to the fact that you're using managed code, which is going to be limited to processor group 0, as the CLR doesn't use the new NUMA instructions in Windows 2008 R2. As such, if your system is setup so processor group 0 is 40 processors, and the other 40 are split into processor group 1, you may be saturating the entire first processor group with this process. For details, see How to Get Started with Multi-Core: Parallel Processing You Can Use.