Initializing Billion Integers to value 1

Question

What is good posix thread design to initialize billion integers using c/c++ on linux platform 8-core CPU with 32GB of DRAM? Thanks for your help.

Answer 1

This is a trivial operation and you need not consider multi-threading. Just do it with a memcpy in a single thread.

Answer 2

The exact number of threads will not be such a limiting factor, but sometimes for this questions it is worth to overcommit, say use 2 threads per physical core.

But the real bottleneck will be IO, writing the data into the RAM. You'd have to take care that the data that is to be replaced will never read before you erase it. Then you should assure that writes to memory appear in large chunks and (if possible) as "write through", mondern CPU have instructions for the later.

Usually something like memcpy with a fixed sized buffer (some pages) that contains the pattern that you want to see should be optimized quite well.

Answer 3

What is that for? Depending on usage, the following scenario might work: you initialize one memory page (that's several KB) to all 1's. Then you map that page into the virtual address space as many times as needed with a copy-on-write flag. This way, on reading you'll get all ones from all those virtual pages, on writing the system will allocate more physical pages as needed.