autoscalingdagster

Does Dagster-cloud support autoscaling directly?


I found in Dagster's documentation a https://docs.dagster.io/deployment/guides/kubernetes/customizing-your-deployment#per-job-kubernetes-configuration

On auto scaling using Kubernetes.

Does Dagster contain a direct method to perform that auto-scaling? (Without Kubernetes)


Solution

  • To my understanding, in ECS, Dagster receives a cluster ID in its running configuration, and uses machines in that cluster of the suitable parameters. (So, if you had cluster A, with machine types X,Y, and request to use X machine, Y machines would be ignored)