How do you specify AWS credentials when running AWS CLI from a Dockerfile in an AWS SAM pipeline?

Question

I have an app using:

• SAM
• AWS S3
• AWS Lambda based on Docker
• AWS SAM pipeline
• Github function

In the Dockerfile I have:

RUN aws s3 cp s3://mylambda/distilBERT distilBERT.tar.gz

Resulting in the error message:

Step 6/8 : RUN aws s3 cp s3://mylambda/distilBERT distilBERT.tar.gz
 ---> Running in 786873b916db
fatal error: Unable to locate credentials
Error: InferenceFunction failed to build: The command '/bin/sh -c aws s3 cp s3://mylambda/distilBERT distilBERT.tar.gz' returned a non-zero code: 1

I need to find a way to store the credential in a secured manner. Is it possible with GitHub secrets or something?

Thanks

Answer 1

My solution may be a bit longer but I feel it solves your problem, and

1. It does not expose any secrets
2. It does not require any manual work
3. It is easy to change your AWS keys later if required.

Steps:

1. You can add the environment variables in Github actions(since you already mentioned Github actions) as secrets.

2. In your Github CI/CD flow, when you build the Dockerfile, you can create a aws credentials file.

      - name: Configure AWS credentials
        echo "
        [default]
        aws_access_key_id = $ACCESS_KEY
        aws_secret_access_key = $SECRET_ACCESS_KEY
        " > credentials
        with:
          ACCESS_KEY: ${{ secrets.AWS_ACCESS_KEY_ID }}
          SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}

3. In your Dockerfile, you can add instructions to COPY this credentials file and store it

COPY credentials credentials
RUN mkdir ~/.aws
RUN mv credentials ~/.aws/credentials

Changing your credentials requires just changing your github actions.

Answer 2

Docker by default does not have access to the .aws folder running on the host machine. You could either pass the AWS credentials as environment variables to the Docker image:

ENV AWS_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE
ENV AWS_SECRET_ACCESS_KEY=...

Keep in mind, hardcoding AWS credentials in a Dockerfile is a bad practice. In order to avoid this, you can pass the environment variables at runtime with using docker run -e MYVAR1 or docker run --env MYVAR2=foo arguments. Other solution would be to use an .env file for the environment variables.

A more involved solution would be to map a volume for the ~/.aws folder from the host machine in the Docker image.