How to implement a program in openCL using MPI on a single cpu machine

Question

I'm new to GPU programming , I have laptop without graphics card,i want to develop a matrix multiplication program on intel openCL, and implement this application using MPI..

any guidelines and helpfull links can be posted.

I'm confused about the MPI thing, do we have to write code for MPI , or do we have to use some developed MPIs to run our application?

this is the project proposal of what i want to do

GPU cluster computation (C++, OpenCL and MPI)

• Study MPI for distributing the problem

• Implement OpenCL apps on a single machine (matrix multiplication/ 2D image processing)

• Implement apps with MPI (e.g. large 2D image processing)

Answer 1

So the thing to understand is that MPI and OpenCL for your purposes are completely orthogonal. MPI is for communicating between your GPU nodes; OpenCL is for accelerating your local computation on a single node by using the GPU (or multiple CPU cores). For any of these problems, you'd start with writing a serial C++ version of the code. The next step would be to (in any order) work on an OpenCL implementation for a single node, and work on an MPI version which decomposes the problems (you don't want to user master-slave for any of the above listed problems) onto multiple processes, with each process doing their local part of the computation which contributes to the global solution. Once both of those parts are done, you'd merge the two and have a distributed-memory (the MPI part) GPU (the OpenCL part) version of a code to solve this problem.

It won't quite be that easy, of course, and combining the two will take a fair bit of work, but that's the basic approach to keep in mind. Start with one problem, get it working on a single processor in C++, then try it with one or the other. Don't try to do everything at once or you'll never get anywhere.

For problems like matrix multiplication, there are many many examples on the internet of both GPU and MPI implementations to learn from.

Answer 2

Simplified:

MPI is a library for communicating proccesses, but also a platform for running applications in a cluster. You write a program that use MPI library and then that program should be executed with MPI. MPI fork that application N times in the cluster and allow to communicate that applicacion instances with messages.

The tasks that the make the instances, if they are the same or different workers, and the topology is up to you.

I think 3 ways to use (OpenCL and MPI):

1. MPI start (K+1) instances, one master and K slaves. The master split the data in chunks and the slaves proccess the data in the GPUS using OpenCL. All slaves are the same.
2. MPI start (k+1) instances, one master and k slaves. Each slave compute a specialized problem (slave 1 matrix multiplication, slave 2 block compression, ...etc) and the master direct the data in a workflow kind of task.
3. MPI start (k+1) instances, one master and k slaves. Same that case 1, but the master also send to the slaves the OpenCL program to proccess data.