ECS with persistent data on EFS or EBS with CloudFormation

Question

I am looking for some expert with AWS to help me with this thing. I've spent almost one week trying to deploy my backend docker image to AWS with no 100% of the desired behaviour.

Firstly I was suggested to try out the new Fargate service AWS recently provided. I managed to deploy everything that I needed but it quickly turned out that I need any kind of data persistence which is unavailable with Fargate for the moment from what I've read.

I found those templates which are very helpful because AWS is soooo big and overwhelming so I would do nothing without those and currently tried the deploy using EC2 instances. https://github.com/awslabs/aws-cloudformation-templates/tree/master/aws/services/ECS/EC2LaunchType

I've a question to this kind of deploy:
1: Why this deploy creates 2 instances of EBS for the cluster? One 8GB with snapshot and second with 22GB in size without snapshot.

2: Is it possible to reduce the size of those EBS volumes ?? If so how ?

3: Is it possible to have just only one of those EBS ?

4: Is it possible to mount volue from my docker backend image to those EBS volumes to persist data ? If so how ? I need to mount two volumes for my backend that is /root/.local/share/Bisq /root/.local/share/bisq-api or ~/.local/share/Bisq ~/.local/share/bisq-api im not quite sure how this works with AWS. What are the paths compared to the steps on the local environment.

5: Would it be better to use EBS or EFS for data persistance? Problem with EFS is that i Can't find any related documents how to connect EFS to this kind of ECS deploy. Everything MUST be with use of CloudFormation templates

Overall the requirements that would match 100% desired behaviour are:

1: CloudFormation Template/tempalte's that deploy neccesarry services as less as possible in order to not build huge infrastructure to preserve the COSTs and ability to just click button and get external link to the backend service. (There can't be any manuall configuration everything must be automated)

2: Ability to stop/start EC2 instace for the backend container (EC2 will be running up from few minutes to hours per day / few days a month. (dependent on each user scenario how often he will be using the backend)

3: Ability to preserve the data when user stops the instance and then starts it in the future point in time.

I would apprecieate any help/suggestions because Im starting to lose my HEAD over everything what connects to AWS services. This is really huge problem to understand any use cases for AWS so I would appreciate help

Thank you!

Answer 1

1: Why this deploy creates 2 instances of EBS for the cluster? One 8GB with snapshot and second with 22GB in size without snapshot.

As per the docs:

Amazon ECS-optimized AMIs from version 2015.09.d and later launch with an 8-GiB volume for the operating system that is attached at /dev/xvda and mounted as the root of the file system. There is an additional 22-GiB volume that is attached at /dev/xvdcz that Docker uses for image and metadata storage.

Here the reference: ecs-ami-storage-config

2: Is it possible to reduce the size of those EBS volumes ?? If so how ?

Also from the same docs:

You can increase these default volume sizes by changing the block device mapping settings for your instances when you launch them; however, you cannot specify a smaller volume size than the default.

3: Is it possible to have just only one of those EBS ?

For this you can better use a custom AMI and configure it as needed.

4: Is it possible to mount volume from my docker backend image to those EBS volumes to persist data ? If so how ? I need to mount two volumes for my backend that is /root/.local/share/Bisq /root/.local/share/bisq-api or ~/.local/share/Bisq ~/.local/share/bisq-api im not quite sure how this works with AWS. What are the paths compared to the steps on the local environment.

This is configured in the task definition: AWS::ECS::TaskDefinition

Type: AWS::ECS::TaskDefinition
Properties: 
  Volumes:
    - Volume Definition
  ...

5: Would it be better to use EBS or EFS for data persistance? Problem with EFS is that i Can't find any related documents how to connect EFS to this kind of ECS deploy. Everything MUST be with use of CloudFormation templates

It depends on your use case scenario. Each one has advantages and disadvantages depending on your needs, so better read the docs for each and chose the best one accordingly. In the templates you found, you can customize the LaunchConfiguration UserData to run the attach commands. You can do all this in CloudFormation.

Additionally I will leave you this documentation for mounting automatically an EFS: Mounting Your Amazon EFS File System Automatically