Menu
I have the following problem:
Program 1 has a huge amount of data, say 10GB. The data in question consists of large integer- and double-arrays. Program 2 has 1..n MPI processes that use tiles of this data to compute results.
How can I send the data from program 1 to the MPI Processes?
Using File I/O is out of question. The compute node has sufficient RAM.
861
The processes communicate via calls to MPI communication primitives. Typically, each process executes in its own address space, although shared-memory implementations of MPI are possible. This document specifies the behavior of a parallel program assuming that only MPI calls are used for communication.
Communicator - Holds a group of processes that can communicate with each other. All message passing calls in MPI must have a specific communicator to use the call with. An example of a communicator handle is MPI_COMM_WORLD. MPI_COMM_WORLD is the default communicator that contains all processes available for use.
The Message Passing Interface (MPI) is an Application Program Interface that defines a model of parallel computing where each parallel process has its own local memory, and data must be explicitly shared by passing messages between processes.
It should be possible, depending on your MPI implementation, to run several different programs in the same MPI job. For instance, using OpenMPI you can run
mpirun -n 1 big_program : -n 20 little_program
and you should be able to access both programs using MPI_COMM_WORLD. From there you'd then be able to use the usual MPI functions to pass your data from the big program to the little ones.
177
One answer might be to have the two programs reside in separate communicators; a single executable could launch both sets of apps by utilizing MPI-2's dynamic process management, and the "producer" program communicate through MPI_COMM_WORLD to the "consumer" application. Subsequently, all IPC for the consumer app would have to run inside a subcommunicator that excluded the producer portion. This would mean rewriting to avoid direct calls to MPI_COMM_WORLD, however.
34
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
