Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to make a HTTP call reaching all instances behind amazon AWS load balancer?

I have a web app which runs behind Amazon AWS Elastic Load Balancer with 3 instances attached. The app has a /refresh endpoint to reload reference data. It need to be run whenever new data is available, which happens several times a week.

What I have been doing is assigning public address to all instances, and do refresh independently (using ec2-url/refresh). I agree with Michael's answer on a different topic, EC2 instances behind ELB shouldn't allow direct public access. Now my problem is how can I make elb-url/refresh call reaching all instances behind the load balancer?

And it would be nice if I can collect HTTP responses from multiple instances. But I don't mind doing the refresh blindly for now.

like image 556
Click2Death Avatar asked Sep 14 '16 19:09

Click2Death


People also ask

Can a load balancer send a request to all instances?

Application Load Balancing at its most basic provides the same functionality as Classic Load Balancing, in that it distributes requests to multiple instances (servers) across multiple AWS availability zones (AZs) in order to distribute traffic evenly across all instances, keeping the instances as evenly loaded as ...

Which load balancer is best suited for HTTP https Load Balancing traffic?

If you need to load balance HTTP requests, we recommend you use the Application Load Balancer (ALB). For network/transport protocols (layer4 – TCP, UDP) load balancing, and for extreme performance/low latency applications we recommend using Network Load Balancer.

What rule will you implement if you need to forward all traffic coming from a single application to any specific target group?

Elastic load balancer rules:Direct all traffic coming from particular application.

How does the AWS load balancer routes traffic?

A load balancer accepts incoming traffic from clients and routes requests to its registered targets (such as EC2 instances) in one or more Availability Zones. The load balancer also monitors the health of its registered targets and ensures that it routes traffic only to healthy targets.


1 Answers

one of the way I'd solve this problem is by

  1. writing the data to an AWS s3 bucket
  2. triggering a AWS Lambda function automatically from the s3 write
  3. using AWS SDK to to identify the instances attached to the ELB from the Lambda function e.g. using boto3 from python or AWS Java SDK
  4. call /refresh on individual instances from Lambda
  5. ensuring when a new instance is created (due to autoscaling or deployment), it fetches the data from the s3 bucket during startup
  6. ensuring that the private subnets the instances are in allows traffic from the subnets attached to the Lambda
  7. ensuring that the security groups attached to the instances allow traffic from the security group attached to the Lambda

the key wins of this solution are

  • the process is fully automated from the instant the data is written to s3,
  • avoids data inconsistency due to autoscaling/deployment,
  • simple to maintain (you don't have to hardcode instance ip addresses anywhere),
  • you don't have to expose instances outside the VPC
  • highly available (AWS ensures the Lambda is invoked on s3 write, you don't worry about running a script in an instance and ensuring the instance is up and running)

hope this is useful.

like image 101
redoc Avatar answered Oct 15 '22 02:10

redoc