How to specify sensitive environment variables at deploy time with Elastic Beanstalk

Question

I am deploying a Python Flask application with Elastic Beanstalk. I have a config file /.ebextensions/01.config where among other things I set some environment variables - some of which should be secret.

The file looks something like this:

packages:   yum:     gcc: []     git: []     postgresql93-devel: []  option_settings:   "aws:elasticbeanstalk:application:environment":     SECRET_KEY: "sensitive"     MAIL_USERNAME: "sensitive"     MAIL_PASSWORD: "sensitive"     SQLALCHEMY_DATABASE_URI: "sensitive"   "aws:elasticbeanstalk:container:python:staticfiles":     "/static/": "app/static/" 

What are the best practices for keeping certain values secret? Currently the .ebextensions folder is under source control and I like this because it is shared with everyone, but at the same time I do not want to keep sensitive values under source control.

Is there a way to specify some environment variables through the EB CLI tool when deploying (e.g. eb deploy -config ...)? Or how is this use case covered by the AWS deployment tools?

Answer 1

The AWS documentation recommends storing sensitive information in S3 because environment variables may be exposed in various ways:

Providing connection information to your application with environment properties is a good way to keep passwords out of your code, but it's not a perfect solution. Environment properties are discoverable in the Environment Management Console, and can be viewed by any user that has permission to describe configuration settings on your environment. Depending on the platform, environment properties may also appear in instance logs.

The example below is from the documentation, to which you should refer for full details. In short, you need to:

1. Upload the file to S3 with minimal permissions, possibly encrypted.

2. Grant read access to the role of the instance profile for your Elastic Beanstalk autoscaling group. The policy would be like:

   {     "Version": "2012-10-17",     "Statement": [         {             "Sid": "database",             "Action": [                 "s3:GetObject"             ],             "Effect": "Allow",             "Resource": [                 "arn:aws:s3:::my-secret-bucket-123456789012/beanstalk-database.json"             ]         }     ] } 

3. Add a file with a name like s3-connection-info-file.config to /.ebextensions in your application bundle root with these contents:

   Resources:   AWSEBAutoScalingGroup:     Metadata:       AWS::CloudFormation::Authentication:         S3Auth:           type: "s3"           buckets: ["my-secret-bucket-123456789012"]           roleName: "aws-elasticbeanstalk-ec2-role"  files:   "/tmp/beanstalk-database.json" :     mode: "000644"     owner: root     group: root     authentication: "S3Auth"     source: https://s3-us-west-2.amazonaws.com/my-secret-bucket-123456789012/beanstalk-database.json 

Then update your application code to extract the values from the file /tmp/beanstalk-database.json (or wherever you decide to put it in your actual config.)

Answer 2

This question already has an answer, but I want to contribute an alternative solution to this problem. Instead of having to keep secrets in environment variables (which then have to be managed and stored somewhere out of version control, plus you need to remember to set them at deployment), I put all my secrets in an encrypted S3 bucket only accessible from the role the EB is running as. I then fetch the secrets at startup. This has the benefit of completely decoupling deployment from configuration, and you never ever have to fiddle with secrets in the command line again.

If needed (for example if secrets are needed during app setup, such as keys to repositories where code is fetched) you can also use an .ebextensions config file with an S3Auth directive to easily copy the contents of said S3 bucket to your local instance; otherwise just use the AWS SDK to fetch all secrets from the app at startup.

EDIT: As of April 2018 AWS offers a dedicated managed service for secrets management; the AWS Secrets Manager. It offers convenient secure storage of secrets in string or json format, versioning, stages, rotation and more. It also eliminates some of the configuration when it comes to KMS, IAM etc for a quicker setup. I see no real reason using any other AWS service for storing static sensitive data such as private keys, passwords etc.