Running parallel jobs on multicore using GNU parallel

Question

I need to run multiple jobs on a multicore (and multithreaded) machine. I am using the GNU Parallel utility to distribute jobs across the cores to speed up the task. The commands to be executed are available in a file called 'commands'. I use the following command to run the GNU Parallel.

cat commands | parallel -j +0

As per the guidance at this location- gnu parallel, this command is supposed to use all the cores to run this task. My machine has 2 cores and 2 threads per core. The system monitor however shows 4 CPUs (CPU1 and CPU2 belong to core1, CPU3 and CPU4 belong to core2). Each job (simulation) takes about 20 seconds to run on a single core. I ran 2 jobs in parallel using this GNU parallel utility with the command above. I observe in the system monitor that, if the 2 jobs are assigned to cpu1 and cpu2 (that is the same core), there is obviously no speed-up. They take about 40seconds to finish, which is about the time they would take if run sequentially. However, sometimes the tool distributes the 2 jobs to CPU1 and CPU3 or CPU4 (which means, 2 jobs are assigned to 2 different cores). In this case, both jobs finish parallely in 20 seconds.

Now, I want to know if there is a way in which I can force the tool to run on different "cores" and not on different "threads" on the same core. Any help is appreciated. Thanks!

Answer 1

GNU Parallel spawns processes. It does not decide which core to run it on. Your OS does that. GNU/Linux makes it possible to save power by having processes spawn on cores the same physical CPU to save power:

echo 1 > /sys/devices/system/cpu/sched_mc_power_savings

(See https://lesswatts.org/tips/cpu.php)

If you computer is set up for saving power this may be the case for you.

GNU/Linux also makes it possible to control which cores a process should be spawned on. Using taskset you can tell GNU/Linux to spawn on every other core using the mask 10101010(bin)=0xAA; here shown for a 128 core machine:

cat commands | taskset 0xaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa parallel -j +0