amazon-web-servicesaws-lambdaamazon-cloudwatchaws-application-load-balancer

Is there a way to monitor/alert on any changes to the private IP address of an ALB in AWS?


I'm aware that AWS Application Load Balancers occasionally change their private IP addresses due to maintenance, from what I understand customers are unable to receive prior warning of these changes. I'm currently trying to find a way to monitor/alert on these changes as and when they happen.

Has anybody implemented a similar solution before using CloudWatch/EventBridge/Lambda?


Solution

  • As this is AWS Managed Service, we have no insight into when this will happen but we can tell when it happens after the fact. In any case this change should not matter and any clients using the ALB as a entry point should respect DNS. Along with this the DNS Record should be set with a TTL of 60 seconds to avoid stale records being returned to the client when the IP does change.

    Some methods to alert on this change;

    1. When the IP address of the ALB changes, a ENI is created/deleted in the VPC/Subnet in which the new IP address is associated with this, so you can create an EventBridge rule to monitor for any 'CreateNetworkInterface || DeleteNetworkInterface' Actions where the "sourceIPAddress": "elasticloadbalancing.amazonaws.com" and/or "requestParameters": {"description": "ELB app/xxxxxx/xxxxxxxxxx"} - where the ELB "ELB app/xxxxxx/xxxxxxxxxx" will be replaced by the ELB you wish to monitor these changes for.

    A pattern such as the following;

    {
      "source": ["aws.ec2"],
      "detail-type": ["AWS API Call via CloudTrail"],
      "detail": {
        "sourceIPAddress": ["elasticloadbalancing.amazonaws.com"],
        "eventName": ["CreateNetworkInterface","DeleteNetworkInterface"],
        "requestParameters": {
          "description": ["ELB app/xxxxxx/xxxxxxxxxx"]
        }
      }
    }
    

    You can then set the target of the EventBridge rule as a SNS Topic to alert on this.

    Note: To create a rule that triggers on an API call via CloudTrail - https://docs.aws.amazon.com/AmazonCloudWatch/latest/events/Create-CloudWatch-Events-CloudTrail-Rule.html

    1. EventBridge to trigger a Lambda function at a fixed rate of 1 time per minute/hour. The Lambda function fetches the latest ALB/CLB IP addresses by performing DNS lookup and compares them to the previously stored IP addresses in a text file stored on AWS S3. If there is a change in the Load Balancer IPs, the Lambda function updates the S3 text file with new IP addresses and sends email notification using SNS.

    2. Utilize something similar to the above, but use local scripts