Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

how does multiple EC2 instances (scaling) works on one EBS for data storage?

So, in a simple situation, if there is only one instance, then I can store the data into a EBS volume mounted on that instance. e.g. /mnt/db

However, how does it work if I scale and have multiple instance (either static or dynamic scaling)?

Because one EBS can only attach to one instance, if I have multiple instance, does it mean that I have to attach an EBS volume for each instance? If that's the case, the data on each Instance's EBS volume will be different.

It is obvious that I want all instances to access (R & W) a single volume (as data-storage). and the data in the volume will constantly grow and there is no downtime.

What is the solution? Is there a way that I don't mount the device (EBS), and just call it for accessing the data?

Here is what I can think of: 1) if each instance has its own EBS volume, then each time interval (e.g. 1 hour), all instances will unmount & detach the EBS volume,and attach a new one. Then there is one powerful instance that mount all the EBS volumes just detached, and aggregate all the data. 2) or similar to 1), instead of detach and attach, I just take a snapshot on all volumes for all instances. Then the powerful instance aggregateness the data from the snapshot. And save the result into either another EBS or S3.

These two approach seem to be working.. but require a lot of work. is there a smarter way to approach this problem? thanks.

  • by the way, because of performance issue, I cannot have the instance writes data to S3. :)

OH how about this 3) First, all instances have their own EBS and write data into the EBS. and then each hour, data will be sent to S3. Then another instance will aggregate them.

like image 473
murvinlai Avatar asked Jan 27 '11 00:01

murvinlai


People also ask

Can multiple EC2 instances use the same EBS volume?

AWS released a feature called Multi-Attach, which allows EC2 instances to share a single EBS volume for up to 16 instances and provide higher availability of your applications for Linux workloads. Each instance to which the volume is attached has full read and write permissions to the volume.

How does EC2 Auto Scaling group work?

An Auto Scaling group starts by launching enough instances to meet its desired capacity. It maintains this number of instances by performing periodic health checks on the instances in the group. The Auto Scaling group continues to maintain a fixed number of instances even if an instance becomes unhealthy.

Can an EBS volume be attached to multiple instances at a time?

You can attach multiple EBS volumes to a single instance. The volume and instance must be in the same Availability Zone. Depending on the volume and instance types, you can use Multi-Attach to mount a volume to multiple instances at the same time.

What AWS storage solution can be provided to multiple EC2 instances at the same time?

Use EBS Multi-Attach volumes to enable multiple EC2 instances to simultaneously access a standard file system.


1 Answers

how about having ang NFS instance which can be mounted to the other instances?

like image 176
Rodney Quillo Avatar answered Oct 28 '22 08:10

Rodney Quillo