Copying File From S3 To EC2 by User Data Approach

Question

I have been searching solution for this task, all I find CLI approaches which I don't want.

I simply want:

I have an S3 Bucket, which has one private file, file can be an image/zip file anything.

And I want when I launch any EC2 instance it should have taken that file from S3 bucket to EC2 instance directory.

And for this, I want to use only EC2 User Data Approach.

Answer 1

The User Data field in Amazon EC2 passes information to the instance that is accessible to applications running on the instance.

Amazon EC2 instances launched with Amazon-provided AMIs (eg Amazon Linux 2) include a program called Cloud-Init that looks at the User Data and, if a script is provided, runs that script the first time that the instance is booted.

Therefore, you can configure a script (passed via User Data) that will run when the instance is first launched. The script will run as the root user. Your script could copy a file from Amazon S3 by using the AWS Command-Line Interface (CLI), like this:

#!
aws s3 cp s3://my-bucket/foo.txt /home/ec2-user/foo.txt
chown ec2-user foo.txt

Please note that you will need to assign an IAM Role to the instance that has permission to access the bucket. The AWS CLI will use these permissions for the file copy.

You mention that you do not wish to use the AWS CLI. You could, instead, write a program that calls the Amazon S3 API using a preferred programming language (eg Python), but using the CLI is much simpler.

Answer 2

The accepted answer is awesome, but just for clarity's sake.

1. create an IAM role and add the IAMReadOnlyAccess permission to it.
2. create a S3 bucket with a suitable bucket policy and put your file(s) there.
3. Create an instance which IAM role is the one you just created.
4. In the EC2 instance's user data copy your file(s) from the bucket to wherever you need them to be.
5. Launch your instance and test.

Here's the simple user data I tested with...

#!/bin/bash
yum update -y
yum install httpd -y
aws s3 cp s3://clarius-content/index.html /var/www/html
service httpd start

I used this bucket policy to allow the EC2 instance to access my bucket via its (the instance's) IAM role.

{
    "Version": "2012-10-17",
    "Id": "Policy1686172385847",
    "Statement": [
        {
            "Sid": "Stmt1686172384560",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::<account-number>:role/EC2-IAM-ReadOnly"
            },
            "Action": "s3:GetObject",
            "Resource": "arn:aws:s3:::clarius-content/*"
        }
    ]
}

So, why do we need both a bucket policy and an IAM role to make this work? Well, the default security principal is least privilege, so we need an IAM role that allows the EC2 instance to make S3 calls, and a bucket which policy will accept those S3 calls.

FYI user data is stored as meta data, you won't find it anywhere on the instance.