Multithreading slower than Singlethreading

Question

I have the following Code (complete content of 'Program.cs' of console application). The single threaded execution of 'countUp' till 'countUp4' takes 13 sec., the multi threaded execution 21 sec..

I have a Intel Core i5-2400 @ 3.10 GHz, 8 GB Ram, Windows 7 64 Bit. So why is the multi threaded execution slower than the single threaded one?

Is multithreading just useful for not blocking the main routine of simple c# applications? When does multithreading give me an advantage in execution speed?

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading;

namespace ConsoleApplication1
{
    class Program
    {
        static int counter = 0;
        static int counter2 = 0;
        static int counter3 = 0;
        static int counter4 = 0;

        static void Main(string[] args)
        {
            Console.WriteLine("Without multithreading:");
            Console.WriteLine("Start:" + DateTime.Now.ToString());

            countUp();
            countUp2();
            countUp3();
            countUp4();

            Console.WriteLine("");
            Console.WriteLine("With multithreading:");
            Console.WriteLine("Start:" + DateTime.Now.ToString());

            Thread thread1 = new Thread(new ThreadStart(countUp));
            thread1.Start();
            Thread thread2 = new Thread(new ThreadStart(countUp2));
            thread2.Start();
            Thread thread3 = new Thread(new ThreadStart(countUp3));
            thread3.Start();
            Thread thread4 = new Thread(new ThreadStart(countUp4));
            thread4.Start();

            Console.Read();
        }

        static void countUp()
        {
            for (double i = 0; i < 1000000000; i++)
            {
                counter++;
            }

            Console.WriteLine(counter.ToString());
            Console.WriteLine(DateTime.Now.ToString());
        }

        static void countUp2()
        {
            for (double i = 0; i < 1000000000; i++)
            {
                counter2++;
            }

            Console.WriteLine(counter2.ToString());
            Console.WriteLine(DateTime.Now.ToString());
        }

        static void countUp3()
        {
            for (double i = 0; i < 1000000000; i++)
            {
                counter3++;
            }

            Console.WriteLine(counter3.ToString());
            Console.WriteLine(DateTime.Now.ToString());
        }

        static void countUp4()
        {
            for (double i = 0; i < 1000000000; i++)
            {
                counter4++;
            }

            Console.WriteLine(counter4.ToString());
            Console.WriteLine(DateTime.Now.ToString());
        }
    }
}

Answer 1

Here's a cause that you might not see coming: false sharing because those 4 ints all sit side by side in memory.

Update - MSDN mags from previous years are only available as .chm files now - so you have to grab the 'October 2008' edition of the MSDN Mag from here and, after downloading, you must remember to right-click and 'unblock' the file from the file properties dialog in Windows Explorer (other OSs are available!) before opening it. You're looking for a column called '.Net Matters' by Stephen Toub, Igor Ostrovsky, and Huseyin Yildiz

The article (read it all - it's brilliant) shows how values that are side by side in memory can end up causing blocking when updated because they all sit on the same cache line. This is very low-level blocking that you can't disable from your .Net code. You can, however force the data to be spaced further apart so that you guarantee, or at least increase the likelihood, that each value will be on a different cache line.

The article uses arrays - but it's just possible it's affecting you here.

To follow up the suggestion below, you might be able to prove/disprove this by changing your code ever-so-slightly:

class Program 
{ 
    class CounterHolder {
       private int[] fakeInts = new int[1024];
       public int Value = 0;
    }
    static CounterHolder counter1 = new CounterHolder(); 
    static CounterHolder counter2 = new CounterHolder(); 
    static CounterHolder counter3 = new CounterHolder(); 
    static CounterHolder counter4 = new CounterHolder(); 

And then modify your thread functions to manipulate the public field Value on each of the counter holders.

I've made those arrays really much bigger than they need to be in the hope that it'll prove it better :)

Answer 2

Andreas Zaltan's is the answer. Take the code

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading;
using System.Diagnostics;
using System.Threading.Tasks;

namespace ConsoleApplication1
{
    class Program
    {
        //static int counter = 0;
        //static int counter2 = 0;
        //static int counter3 = 0;
        //static int counter4 = 0;

        class CounterHolder
        {
            private int[] fakeInts = new int[1024];
            public int Value = 0;
        }
        static CounterHolder counter1 = new CounterHolder();
        static CounterHolder counter2 = new CounterHolder();
        static CounterHolder counter3 = new CounterHolder();
        static CounterHolder counter4 = new CounterHolder(); 

        static void Main(string[] args)
        {
            Console.WriteLine("Without multithreading:");
            Console.WriteLine("Start: " + DateTime.Now.ToString());

            Stopwatch sw = new Stopwatch();
            sw.Start();

            countUp();
            countUp2();
            countUp3();
            countUp4();

            sw.Stop();
            Console.WriteLine("Time taken = " + sw.Elapsed.ToString());

            Console.WriteLine("\nWith multithreading:");
            Console.WriteLine("Start: " + DateTime.Now.ToString());
            sw.Reset();
            sw.Start();

            Task task1 = Task.Factory.StartNew(() => countUp());
            Task task2 = Task.Factory.StartNew(() => countUp2());
            Task task3 = Task.Factory.StartNew(() => countUp3());
            Task task4 = Task.Factory.StartNew(() => countUp4());
            var continuation = Task.Factory.ContinueWhenAll(new[] { task1, task2, task3, task4 }, tasks =>
            {
                Console.WriteLine("Total Time taken = " + sw.Elapsed.ToString());
            });
            Console.Read();
        }

        static void countUp()
        {
            Stopwatch sw = new Stopwatch();
            sw.Start();
            for (double i = 0; i < 1000000000; i++)
                counter1.Value++;
            sw.Stop();
            Console.WriteLine("Task countup took: " + sw.Elapsed.ToString());
        }

        static void countUp2()
        {
            Stopwatch sw = new Stopwatch();
            sw.Start();
            for (double i = 0; i < 1000000000; i++)
                counter2.Value++;
            sw.Stop();
            Console.WriteLine("Task countUP2 took: " + sw.Elapsed.ToString());
        }

        static void countUp3()
        {
            Stopwatch sw = new Stopwatch();
            sw.Start();
            for (double i = 0; i < 1000000000; i++)
                counter3.Value++;
            sw.Stop();
            Console.WriteLine("Task countUP2 took: " + sw.Elapsed.ToString());
        }

        static void countUp4()
        {
            Stopwatch sw = new Stopwatch();
            sw.Start();
            for (double i = 0; i < 1000000000; i++)
                counter4.Value++;
            sw.Stop();
            Console.WriteLine("Task countUP2 took: " + sw.Elapsed.ToString());
        }
    } 
}

Run it with the intergers and you get the multi-thrreadded version running ever so slightly slower.

Serial: 13.88s
Multi-threaded: 14.01

Run it using the suggestion above you get the following

I have posted this for clarity...