airflowmetricsdatadogmwaa

How to send AWS MWAA cluster metrics to DataDog?


AWS's Managed Workflows for Apache Airflow (MWAA) provides cluster metrics like the CPUUtilization and MemoryUtilization for the BaseWorker, Scheduler and WebServer. I would like to have these in DataDog so I can set usage alerts. Here's a example of what they look like in Cloudwatch:

View of MWAA metrics in AWS

However, DataDog's official MWAA integration doesn't have these metrics listed. Is there a way to directly specify the metrics and forward them to DataDog? Or, possibly, are they already being picked up but under an obscure namespace?


Solution

  • After some research I found that you need to activate an option to send custom metrics from AWS.

    You have to follow the next steps: Datadog UIIntegrationsSearch for AWSSelect your accountGo to Metric CollectionTurn on Collect custom metrics from Cloudwatch

    enter image description here

    After some minutes search for something like aws.mwaa.cpuutilization and it should be there for you to filter by cluster or any other type of query

    enter image description here