I am doing a project in the university which requires running of multiple instances (1000s) of a program I've written (in C++), which runs for quite a while (say 2 hours). The program is very self contained - it does not require input files, and the only dependency I think is boost.
I'm currently using the university-owned cluster of computer. However, it's quite old and the jobs dispatching and monitors services are pretty bad.
So I was wondering whether I can run my jobs elsewhere, for some money. For example, I looked a bit into Google App Engine, but as it seems every job must end after 30 seconds it is not suitable for me. Maybe Amazon EC2?
Do you know of such options?
The entry-level cost for some of the commercial turn-key clusters today is in the $10,000 range. This will get you 8-12 cores, the interconnect, disks, and operating system you need to have a small but functioning cluster.
A cluster is a group of inter-connected computers or hosts that work together to support applications and middleware (e.g. databases). In a cluster, each computer is referred to as a “node”. Unlike grid computers, where each node performs a different task, computer clusters assign the same task to each node.
A Beowulf cluster is a distributed computing system made of normal desktop computers that are connected via commodity network such as Ethernet and are being controlled by free and open source software like Linux.
Some of the popular implementations of cluster computing are Google search engine, Earthquake Simulation, Petroleum Reservoir Simulation, and Weather Forecasting system.
Amazon EC2 is the classic approach for this.
Google App Engine is great, but probably to restrictive for your use case.
EC2 is definitely a very good option, as Peter says. Since you're at a university I'm guessing that cost may be an important factor, so take a look at Rackspace's cloud service as well; depending on what kind of server resources you need, this can work out quite a bit cheaper than EC2. (I don't work for Rackspace).
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With