Why do programs run slower on first run after compile?

Question

I have this simple hello world program:

#include <stdio.h>

int main() {
  printf("Hello, world!\n");
  return 0;
}

I compiled this program with LLVM Clang (v15.0.1, built from Homebrew, so not Apple's version) like normal, then I ran and timed the outputs. To my surprise, the first time the program ran, it took nearly 10x longer than the second time, but the next three executions run much faster.

$ clang test.c -o test
$ time ./test
Hello, world!

real    0m0.169s
user    0m0.001s
sys 0m0.002s
$ time ./test
Hello, world!

real    0m0.017s
user    0m0.001s
sys 0m0.006s
$ time ./test
Hello, world!

real    0m0.004s
user    0m0.001s
sys 0m0.002s
$ time ./test
Hello, world!

real    0m0.008s
user    0m0.001s
sys 0m0.005s

I'm running this on an Intel Core i5 mac, running macOS Big Sur v11.6.8. The shell is the bash shipped with macOS.

Nothing in my code deals with time, and I don't think there's anything to cache, so I'm not sure why the first execution runs so slow. I suspect that the OS might be doing some kind of optimization, but I don't know what/how. What is the cause of this large discrepancy in runtimes?

Answer 1

There is another factor that could contribute to the effect you observed, and that is the CPU cache. The first time that the program is run, all attempts to fetch instructions and data from the L1, L2, and L3 caches produce cache misses, and the CPU has to then fetch them from RAM or from the disk, both of which are much slower than the CPU caches. However, during the first run, the CPU caches are populated with the program's instructions and data, so that on subsequent runs of the program, the CPU is more likely to find the instructions and data in one of those CPU caches, and the program is likely to run faster as a result.