How can multithreading speed up an application (when threads can't run concurrently)?

Question

I'm learning about multithreading, but after reading some tutorials I'm sort of confused. I don't understand how multithreading can speed up an application.

By intuition, I would say multithreading slows down an application, because you constantly have to wait for those semaphores.

How and when can multithreading speed up an application, when threads can't run concurrently?

Answer 1

because you constantly have to wait for those semaphores.

Only in a poorly-designed program or in one designed for parallel work on a single-processor machine. In a well-designed program, the threads do useful work in parallel in between the synchronization points, and enough of it to outweigh the overhead of synchronization.

Even without parallel (multicore/multiprocessor) processing, multithreading can be beneficial when the threads do blocking I/O. E.g., the good old CVSup programs used multithreading in the single-core era to make full use of network connections' duplex capabilities. While one thread was waiting for data to arrive over the link, another would be pushing data the other way. Due to network latency, both threads necessarily had to spend a lot of time waiting, during which the other threads could do useful work.

Answer 2

Two ways I can think of, the first of which is probably what you mean by "parallel threading".

• If you have multiple CPUs or cores, they can work simultaneously if you're running multiple threads.
• In the single core case, if your thread ends up waiting for (synchronous) I/O, let's say you call read() to read 100 MB from tape, another thread can get scheduled and get work done while you wait.

Answer 3

The idea behind multithreading is to have as few blocking points as possible. In other words, if a thread has to constantly wait on another thread to finish something, then the benefit of threads is likely lost in that situation.

Obligatory link: http://en.wikipedia.org/wiki/Amdahl's_law

Also, as Mark Ransom said, if your hardware can't actually do more than 1 thing at once, then threads are really just logically running at the same time (by swapping) than actually running at the same time. That can still be useful in situations with IO blocking though.