amazon-ec2dvccml

How to configure vpc/subnets for ec2 instances via cml runner in aws?


I am setting up a workflow via dvc(https://dvc.org/) and CML (https://cml.dev/) in aws. I am gitlab user. based on documentation provided in CML, cml runner can provision ec2 instances in aws and run training and processing job. how can i pass vpc/subnet configuration for ec2 via cml runner.

deploy:
  image: iterativeai/cml:0-dvc2-base1
  script:
    - cml runner launch
      --cloud=gcp
      --cloud-region=asia-southeast1-a
      ...
      --labels=cml-runner

test:
  needs: [deploy]
  script:
    - ...

Solution

  • Here is an example that you can use:

    cml runner launch \
       ...
       --cloud=aws \
       --cloud-region=xxx \
       --cloud-type=t3.small \
       --cloud-aws-security-group=cml-test-sg \
       --cloud-aws-subnet=subnet-5d7ee225
    

    You can create an AWS security group in the VPC you want to target (you need to expose SSH for cml runner) note that cml-test-sg is the Name of the created security in your target VPC

    Additionally you can target a specific subnet within your VPC by also providing the id for the resource you want, here: subnet-5d7ee225