Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Tools for setting up and running a grid job on Google Compute Engine?

I have the need to set up and run "embarrassingly" parallel jobs on Google Compute Engine. I am looking for tools to facilitate this.

On EC2, I was using MIT's Starcluster to set up the cluster, and then just submitting the job to SGE.

Are there similar tools available yet for Google Compute Engine?

If not, I will probably manually set up a Condor Cluster to do the jobs. Buzz kill.

like image 415
Jose Cortez Avatar asked Jun 20 '13 18:06

Jose Cortez


1 Answers

I didn't find anything out there. Starcluster's current path to GCE support is to wait for official boto support for GCE. That all may take a while, so I just went ahead and wrote some python code to start up and terminate a basic cluster.

I shared it here on github.

I hope someone else finds it useful, at the very least as sample code.

like image 125
Jose Cortez Avatar answered Oct 24 '22 04:10

Jose Cortez