Getting starting with Parallel programming [closed]

Question

So it looks like multicore and all its associated complications are here to stay. I am planning a software project that will definitely benefit from parallelism. The problem is that I have very little experience writing concurrent software. I studied it at University and understand the concepts and theory very well but have zero hands on useful experience building software to run on on multiple processors since school.

So my question is, what is the best way to get started with multiprocessor programming? I am familiar with mostly Linux development in C/C++ and Obj-C on Mac OS X with almost zero Windows experience. Also my planned software project will require FFT and probably floating point comparisons of a lot of data.

There is OpenCL, OpenMP, MPI, POSIX threads, etc... What technologies should I start with?

Here are a couple stack options I am considering but not sure if they will let me experiment in working towards my goal:

• Should I get Snow Leopard and try to get OpenCL Obj-C programs to run execution on the ATI X1600 GPU on my laptop? or
• Should I get a Playstation and try writing C code to throw across its six available Cell SPE cores? or
• Should I build out a Linux box with an Nvidia card and try working with CUDA?

Thanks in advance for your help.

Answer 1

I'd suggest going OpenMP and MPI initially, not sure it matters which you choose first, but you definitely ought to want (in my opinion :-) ) to learn both shared and distributed memory approaches to parallel computing.

I suggest avoiding OpenCL, CUDA, POSIX threads, at first: get a good grounding in the basics of parallel applications before you start to wrestle with the sub-structure. For example, it's much easier to learn to use broadcast communications in MPI than it is to program them in threads.

I'd stick with C/C++ on your Mac since you are already familiar with them, and there are good open-source OpenMP and MPI libraries for that platform and those languages.

And, and for some of us it's a big plus, whatever you learn about C/C++ and MPI (to a lesser extent it's true of OpenMP too) will serve you well when you graduate to real supercomputers.

All subjective and argumentative, so ignore this if you wish.