Securely storing environment variables in GAE with app.yaml

Question

I need to store API keys and other sensitive information in app.yaml as environment variables for deployment on GAE. The issue with this is that if I push app.yaml to GitHub, this information becomes public (not good). I don't want to store the info in a datastore as it does not suit the project. Rather, I'd like to swap out the values from a file that is listed in .gitignore on each deployment of the app.

Here is my app.yaml file:

application: myapp version: 3  runtime: python27 api_version: 1 threadsafe: true  libraries: - name: webapp2   version: latest - name: jinja2   version: latest  handlers: - url: /static   static_dir: static  - url: /.*   script: main.application     login: required   secure: always # auth_fail_action: unauthorized  env_variables:   CLIENT_ID: ${CLIENT_ID}   CLIENT_SECRET: ${CLIENT_SECRET}   ORG: ${ORG}   ACCESS_TOKEN: ${ACCESS_TOKEN}   SESSION_SECRET: ${SESSION_SECRET} 

Any ideas?

Answer 1

This solution is simple but may not suit all different teams.

First, put the environment variables in an env_variables.yaml, e.g.,

env_variables:   SECRET: 'my_secret' 

Then, include this env_variables.yaml in the app.yaml

includes:   - env_variables.yaml 

Finally, add the env_variables.yaml to .gitignore, so that the secret variables won't exist in the repository.

In this case, the env_variables.yaml needs to be shared among the deployment managers.

Answer 2

If it's sensitive data, you should not store it in source code as it will be checked into source control. The wrong people (inside or outside your organization) may find it there. Also, your development environment probably uses different config values from your production environment. If these values are stored in code, you will have to run different code in development and production, which is messy and bad practice.

In my projects, I put config data in the datastore using this class:

from google.appengine.ext import ndb  class Settings(ndb.Model):   name = ndb.StringProperty()   value = ndb.StringProperty()    @staticmethod   def get(name):     NOT_SET_VALUE = "NOT SET"     retval = Settings.query(Settings.name == name).get()     if not retval:       retval = Settings()       retval.name = name       retval.value = NOT_SET_VALUE       retval.put()     if retval.value == NOT_SET_VALUE:       raise Exception(('Setting %s not found in the database. A placeholder ' +         'record has been created. Go to the Developers Console for your app ' +         'in App Engine, look up the Settings record with name=%s and enter ' +         'its value in that record\'s value field.') % (name, name))     return retval.value 

Your application would do this to get a value:

API_KEY = Settings.get('API_KEY') 

If there is a value for that key in the datastore, you will get it. If there isn't, a placeholder record will be created and an exception will be thrown. The exception will remind you to go to the Developers Console and update the placeholder record.

I find this takes the guessing out of setting config values. If you are unsure of what config values to set, just run the code and it will tell you!

The code above uses the ndb library which uses memcache and the datastore under the hood, so it's fast.

---

Update:

jelder asked for how to find the Datastore values in the App Engine console and set them. Here is how:

1. Go to https://console.cloud.google.com/datastore/

2. Select your project at the top of the page if it's not already selected.

3. In the Kind dropdown box, select Settings.

4. If you ran the code above, your keys will show up. They will all have the value NOT SET. Click each one and set its value.

Hope this helps!