Thread locality

Question

I have this statement, which came from Goetz's Java Concurrency In Practice:

Runtime overhead of threads due to context switching includes saving and restoring execution context, loss of locality, and CPU time spent scheduling threads instead of running them.

What is meant by "loss of locality"?

Answer 1

When a thread works, it often reads data from memory and from disk. The data is often stored in contiguous or close locations in memory/on the disk (for example, when iterating over an array, or when reading the fields of an object). The hardware bets on that by loading blocks of memory into fast caches so that access to contiguous/close memory locations is faster.

When you have a high number of threads and you switch between them, those caches often need to be flushed and reloaded, which makes the code of a thread take more time than if it was executed all at once, without having to switch to other threads and come back later.

A bit like we humans need some time to get back to a task after being interrupted, find where we were, what we were doing, etc.