setting an environment variable for a specific MPI process

Question

I am running a Fortran code in MPI. I need to set an environment variable in one particular process. Is there a way to do this? Calling "system" from the Fortran code does not seem to have an effect. I am running the code via "aprun".

Answer 1

Launcher solution

You should do this with MPMD launching. It works with mpirun or aprun.

Here is an example, where one sets the OMP_NUM_THREADS environment variable differently on one process than the others.

aprun -n 1  -e OMP_NUM_THREADS=1  ./mpi-openmp-app.x input_file.in :
      -n 99 -e OMP_NUM_THREADS=10 ./mpi-openmp-app.x input_file.in

This is the heterogeneous equivalent of

aprun -n 100 -e OMP_NUM_THREADS=10 ./mpi-openmp-app.x input_file.in

Please see the aprun man page (or man aprun from the command line) for details.

Note that Cray is in the process of switching many sites from ALPS (i.e. aprun) to SLURM (srun), but I'm sure that SLURM supports the same feature.

MPI's mpirun or mpiexec supports a similar feature. The syntax is not specified by the MPI standard, so you need to read the documentation of your MPI implementation for the specifics.

Source code solution

Assuming your environment variable is parsed after MPI is initialized, you can do something like the following using setenv, if the launcher solution does not work.

int requested=MPI_THREAD_FUNNELED, provided;
MPI_Init_thread(&argc,&argv,requested,&provided);

int rank;
MPI_Comm_rank(MPI_COMM_WORLD,&rank);
if (rank==0) {
  int overwrite = 1;
  int rc = setenv("OMP_NUM_THREADS","1",overwrite);
}