google-cloud-platformgoogle-compute-enginegoogle-api-python-clientgoogle-dl-platform

How to launch a Google Deep learning VM from google python client?


GCP has a deep learning VM available to run on their cloud compute platform. The details about the image is here

So, I am using the google python client to launch my instances and the documentation for this is available here. Now, the way one specifies the disk and the boot image is through this JSON blob:

'disks': [
            {
                'boot': True,
                'autoDelete': True,
                'initializeParams': {
                    'sourceImage': source_disk_image,
                }
            }
        ]

Now the source_disk_image is specified by the path to some public image like:

projects/debian-cloud/global/images/family/debian-9 or some variant of this type. Now, my question is how can I specify some marketplace image to be used for my instance?


Solution

  • If you're not attached to using the marketplace to create the VM, there's a lot of documentation about all the available Google Deep Learning images.

    They live in the deeplearning-platform-release project, so, for example, I think (but am not sure) the default image you are referring to from the Marketplace you linked is projects/deeplearning-platform-release/global/images/tf-1-14-cu100-20191004 but you can also pull them by family and just get the latest versions, for example, projects/deeplearning-platform-release/global/images/family/tf-latest-gpu.

    The gcloud images command is also pretty illuminating to see the description of a given family choice or image, e.g.:

    $ gcloud compute images describe-from-family tf-latest-gpu --project deeplearning-platform-release
    archiveSizeBytes: '322993843200'
    creationTimestamp: '2019-10-06T13:57:56.932-07:00'
    description: "Google, Deep Learning Image: TensorFlow 1.14.0, m36, TensorFlow 1.14.0\
      \ with CUDA 10.0 and Intel\xAE MKL-DNN, Intel\xAE MKL."
    diskSizeGb: '30'
    ...
    

    Which looks a lot like the Marketplace description.

    That said, it looks like the Marketplace might be doing other things though (e.g. there are checkboxes about installing particular drivers separate from choosing the image).

    I think that @Ernesto's tip about creating an instance off the marketplace, and then viewing that instance via the REST link at the bottom of the instance page to find exactly how it was created is also good advice. However, in this case you probably want to view the disk that was created (not the instance, since once it is created it only references the disk resource), click on the rest link, and look for the "sourceImage" portion of the REST response.

    e.g. from a regular old debian-9 disk (I don't have GPU quota so I can't actually create the marketplace deployment):

    Bottom of Disk instance page


    relevant json