Threaded application over OpenMPI only using 1 core on node

Question

I have an mutlithreaded application (not parallel) that I now want to execute on different nodes synchronised using OpenMPI.

When I run the application on the node I get 300 %CPU utilisation (top command). I assume this means that 3 processors are used 100% (4 Core node).

When I run the same process synchronised using OpenMPI I only get 100% CPU utilisation, which I assume means that all my threads are confined to only 1 CPU on the node.

Is there any way that I can get the program to make use of all CPUs on the nodes for the 1 task scheduled on the node?

I have looked at OMP_NUM_THREADS but that does not help, I guess that it is used when OpenMPI spawns its own threads to panellise work.

Answer 1

I found that the affinity of each process can be set using --cpus-per-proc and it did solve the problem.

BUT: --cpus-per-proc is deprecated (using 1.8.3) and I get the following message:

Command line options:
  Deprecated:  --cpus-per-proc, -cpus-per-proc, --cpus-per-rank, -cpus-per-rank
  Replacement: --map-by <obj>:PE=N, default <obj>=NUMA

I had to use the following to get the same functionality

--map-by socket:pe=4