google-compute-enginesungridenginecondorstarclusterembarrassingly-parallel

Tools for setting up and running a grid job on Google Compute Engine?

I have the need to set up and run "embarrassingly" parallel jobs on Google Compute Engine. I am looking for tools to facilitate this.

On EC2, I was using MIT's Starcluster to set up the cluster, and then just submitting the job to SGE.

Are there similar tools available yet for Google Compute Engine?

If not, I will probably manually set up a Condor Cluster to do the jobs. Buzz kill.

Solution

  • I didn't find anything out there. Starcluster's current path to GCE support is to wait for official boto support for GCE. That all may take a while, so I just went ahead and wrote some python code to start up and terminate a basic cluster.

    I shared it here on github.

    I hope someone else finds it useful, at the very least as sample code.