AWS's Managed Workflows for Apache Airflow (MWAA) provides cluster metrics like the CPUUtilization
and MemoryUtilization
for the BaseWorker
, Scheduler
and WebServer
. I would like to have these in DataDog so I can set usage alerts. Here's a example of what they look like in Cloudwatch:
However, DataDog's official MWAA integration doesn't have these metrics listed. Is there a way to directly specify the metrics and forward them to DataDog? Or, possibly, are they already being picked up but under an obscure namespace?
After some research I found that you need to activate an option to send custom metrics from AWS.
You have to follow the next steps:
Datadog UI
→ Integrations
→ Search for AWS
→ Select your account
→ Go to Metric Collection
→ Turn on Collect custom metrics from Cloudwatch
After some minutes search for something like aws.mwaa.cpuutilization
and it should be there for you to filter by cluster or any other type of query