Are there any simple solutions for distributing computational work in .NET?

Question

So let's say I have some computers at home.

Let's also say that I have some algorithm I want to run, that generally takes a lot of time to solve. It can be divided in how many parts I want, so I could run part of it in one machine, part of it in another, etc, and in the end i'd just need to merge the results in one of the computers.

My question is if there is any easy and straightforward way in .NET of making use of several computers to do this kind of computations. If yes, how is it called? I don't mean having to code all the IPC code by myself, something similar to BCL's Tasks but that'd allow me to send "work" to other computers, via an IP or something.

Thanks!

Answer 1

In general distributed computing, there are two models: Message Passing and Shared Memory

It sounds like you're interested in Message Passing, because it typically involves multiple computers which communicate over the network. Shared memory uses the memory system to communicate between processes. From my (limited) experience in distributed computing, almost everyone who uses Message Passing as their model has moved to MPI. MPI is a framework that has been implemented in many languages. There are some pretty strong .NET MPI implementations out there.

This whole post assumes you're doing "distributed computing" in a HPC scenario. In such a scenario, you would have many powerful computers, connected over an inter-connection network, in a server room (or similar). If you're looking for some kind of distributed computing between seperated physical machines (such as folding@home), you'll need to find something else or roll your own solution. Therefore, as Leniel Macaferi mentions in his response, you'll need to ensure your data center is running some version of Microsoft HPC Server. Most HPC clusters run some flavor of linux, which will obviously prevent you from deploying a .NET based solution.