Is there a .net framework to assist in this scenario?

Question

I have a requirement to process a lot of data on nightly basis. I have a network of semi trusted PC's, intranet - nothing external, mostly desktops - but also some servers.

My initial thought was use the idle time of desktops, plus some dedicated server resources as follows:

1) Bundle data into encrypted 20 meg chunks (admins don’t want any data written to pc disks, must stay in ram)

2) Distribute .net dll(s) containing the processing to be carried out (the processing changes often, hence distribution of dlls)

3) Decrypt Data

4) Execute dll methods

5) Return results

A server would monitor work-units/jobs, and after a time X, if no result received - check status of the machine that should be working on them, and assign to another machine if needs be.

I would not classify the work as CPU intensive - however the parallel element that would allow a whole database to be checked sooner is a real bonus.

I don't think I need intermachine communication, just client-server. I am happy for the clients to simply poll the server - I don't need anything complex in the way of job notification.

My question is: Are there any .NET frameworks out there that do this kind of thing? or Is this approach completely flawed?

Answer 1

Take a look at this question - although it does specifically mention high CPU usage there may be something there which suits your needs.