How to keep secret key information out of Git repository

Question

I have some files in my repository, and one contains a secret Adafruit key. I want to use Git to store my repository, but I don't want to publish the key.

What's the best way to keep it secret, without having to blank it out everytime I commit and push something?

Answer 1

Depending on what you're trying to achieve you could choose one of those methods:

• keep file in the tree managed by git but ignore it with entry in gitignore
• keep file content in environment variable,
• don't use file with key at all, keep key content elsewhere (external systems like hashicorp's vault, database, cloud (iffy, I wouldn't recommend that), etc.)

First approach is easy and doesn't require much work, but you still has the problem of passing the secret key to different location where you'd use the same repository in a secure manner. Second approach requires slightly more work, has the same drawback as the first one.

Third requires certainly more work then 1st and 2nd, but could lead to a setup that's really secure.

Answer 2

This highly depends on the requirements of the project

Basically, the best strategy security-wise is not to store keys, passwords and in general any vulnerable information inside the source control system. If its the goal, there are many different approaches:

• "Supply" this kind of information in Runtime and keep it somewhere else:

  ./runMyApp.sh -db.password=

• Use specialized tools (like, for example Vault by Hashicorp) to manage secrets

• Encode the secret value offline and store in git the encoded value. Without a secret key used for decoding, this encoded value alone is useless. Decode the value again in runtime, using some kind of shared keys infra / asymmetric key pair, in this case, you can use a public key for encoding, a private key for decoding

Answer 3

I want to use Git to store my repository, but I don't want to publish the key.

For something as critical as a secret key, I would use a dedicated keyring infrastructure located outside the development environment, optionally coupled to a secret passphrase.

Aside this case, I personnaly use submodules for this. Check out :

git submodule

In particular, I declare a global Git repository, in which I declare in turn another Git repository that will contain the actual project that will go public. This enables us to store at top level everything that is related to the given project, but not forcibly relevant to it and that is not to be published. This could be, for instance, all my drafts, automation scripts, worknotes, project specifications, tests, bug reports, etc.

Among all the advantages this facility provides, we can highlight the fact that you can declare as a submodule an already existing repository, this repository being located inside or outside the parent one.

And what's really interesting with this is that both main repository and submodules remains distinct Git repositories, that still can be configured independently. This means that you don't need your parent repository to have its remote servers configured.

Doing that way, you get all the benefits of a versioning system wherever you work, while still ensuring yourself that you'll never accidentally push outside something that is not stored inside the public submodule.

Answer 4

If you don't have too many secrets to manage, and you do want to keep the secrets in version control, I make the parent repository private. It contains 2 folders - a secrets folder, and a gitsubmodule for the public repository (in another folder). I use ansible crypt to encrypt anything in the secrets folder, and a bash script to pass the decrypted contents, and load those secrets as environment vars to ensure the secrets file always stays encrypted.

Ansible crypt can encrypt and decrypt an environment variable, which I wrap in a bash script to do these functions like so-

testsecret=$(echo 'this is a test secret' | ./scripts/ansible-encrypt.sh --vault-id $vault_key --encrypt)
result=$(./scripts/ansible-encrypt.sh --vault-id $vault_key --decrypt $testsecret)
echo $result

testsecret here is the encrypted base64 result, and it can be stored safely in a text file. Later you could source that file to keep the encrypted result in memory, and finally when you need to use the secret, you can decrypt it ./scripts/ansible-encrypt.sh --vault-id $vault_key --decrypt $testsecret

This bash script referenced above is below (ansible-encrypt.sh). It wraps ansible crypt functions in a way that can store encrypted variables in base64 which resolves some problems that can occur with encoding.

#!/bin/bash

# This scripts encrypts an input hidden from the shell and base 64 encodes it so it can be stored as an environment variable
# Optionally can also decrypt an environment variable

vault_id_func () {
    if [[ "$verbose" == true ]]; then
        echo "Parsing vault_id_func option: '--${opt}', value: '${val}'" >&2;
    fi
    vault_key="${val}"
}
secret_name=secret
secret_name_func () {
    if [[ "$verbose" == true ]]; then
        echo "Parsing secret_name option: '--${opt}', value: '${val}'" >&2;
    fi
    secret_name="${val}"
}
decrypt=false
decrypt_func () {
    if [[ "$verbose" == true ]]; then
        echo "Parsing secret_name option: '--${opt}', value: '${val}'" >&2;
    fi
    decrypt=true
    encrypted_secret="${val}"
}

IFS='
'
optspec=":hv-:t:"

encrypt=false

parse_opts () {
    local OPTIND
    OPTIND=0
    while getopts "$optspec" optchar; do
        case "${optchar}" in
            -)
                case "${OPTARG}" in
                    vault-id)
                        val="${!OPTIND}"; OPTIND=$(( $OPTIND + 1 ))
                        opt="${OPTARG}"
                        vault_id_func
                        ;;
                    vault-id=*)
                        val=${OPTARG#*=}
                        opt=${OPTARG%=$val}
                        vault_id_func
                        ;;
                    secret-name)
                        val="${!OPTIND}"; OPTIND=$(( $OPTIND + 1 ))
                        opt="${OPTARG}"
                        secret_name_func
                        ;;
                    secret-name=*)
                        val=${OPTARG#*=}
                        opt=${OPTARG%=$val}
                        secret_name_func
                        ;;
                    decrypt)
                        val="${!OPTIND}"; OPTIND=$(( $OPTIND + 1 ))
                        opt="${OPTARG}"
                        decrypt_func
                        ;;
                    decrypt=*)
                        val=${OPTARG#*=}
                        opt=${OPTARG%=$val}
                        decrypt_func
                        ;;
                    encrypt)
                        encrypt=true
                        ;;
                    *)
                        if [ "$OPTERR" = 1 ] && [ "${optspec:0:1}" != ":" ]; then
                            echo "Unknown option --${OPTARG}" >&2
                        fi
                        ;;
                esac;;
            h)
                help
                ;;
            *)
                if [ "$OPTERR" != 1 ] || [ "${optspec:0:1}" = ":" ]; then
                    echo "Non-option argument: '-${OPTARG}'" >&2
                fi
                ;;
        esac
    done
}
parse_opts "$@"

if [[ "$encrypt" = true ]]; then
    read -s -p "Enter the string to encrypt: `echo $'\n> '`";
    secret=$(echo -n "$REPLY" | ansible-vault encrypt_string --vault-id $vault_key --stdin-name $secret_name | base64 -w 0)
    unset REPLY
    echo $secret
elif [[ "$decrypt" = true ]]; then
    result=$(echo $encrypted_secret | base64 -d | /snap/bin/yq r - "$secret_name" | ansible-vault decrypt --vault-id $vault_key)
    echo $result
else
    # if no arg is passed to encrypt or decrypt, then we a ssume the function will decrypt the firehawksecret env var
    encrypted_secret="${firehawksecret}"
    result=$(echo $encrypted_secret | base64 -d | /snap/bin/yq r - "$secret_name" | ansible-vault decrypt --vault-id $vault_key)
    echo $result
fi

Storing encrypted values as environment variables is much more secure than decrypting something at rest and leaving the plaintext result in memory. That's extremely easy for any process to siphon off.

If you only wish to share the code with others and not the secrets, you can use a git template for the parent private repo structure so that others can inherit that structure, but use their own secrets. This also allows CI to pickup everything you would need for your tests.

Alternatively, If you don't want your secrets in version control, you can simply use git ignore on a containing folder that will house your secrets.

Personally, this makes me nervous, its possible for user error to still result in publicly committed secrets, since those files are still under the root of a public repo, any number of things could go wrong that could be embarrassing with that approach.