amazon-web-servicesamazon-ec2elastic-load-balancer

How to make a HTTP call reaching all instances behind amazon AWS load balancer?


I have a web app which runs behind Amazon AWS Elastic Load Balancer with 3 instances attached. The app has a /refresh endpoint to reload reference data. It need to be run whenever new data is available, which happens several times a week.

What I have been doing is assigning public address to all instances, and do refresh independently (using ec2-url/refresh). I agree with Michael's answer on a different topic, EC2 instances behind ELB shouldn't allow direct public access. Now my problem is how can I make elb-url/refresh call reaching all instances behind the load balancer?

And it would be nice if I can collect HTTP responses from multiple instances. But I don't mind doing the refresh blindly for now.


Solution

  • one of the way I'd solve this problem is by

    1. writing the data to an AWS s3 bucket
    2. triggering a AWS Lambda function automatically from the s3 write
    3. using AWS SDK to to identify the instances attached to the ELB from the Lambda function e.g. using boto3 from python or AWS Java SDK
    4. call /refresh on individual instances from Lambda
    5. ensuring when a new instance is created (due to autoscaling or deployment), it fetches the data from the s3 bucket during startup
    6. ensuring that the private subnets the instances are in allows traffic from the subnets attached to the Lambda
    7. ensuring that the security groups attached to the instances allow traffic from the security group attached to the Lambda

    the key wins of this solution are

    hope this is useful.