Authenticating Firebase connection in GitHub Action

Question

Background

I have a Python script that reads data from an Excel file and uploads each row as a separate document to a collection in Firestore. I want this script to run when I push a new version of the Excel file to GitHub.

Setup

I placed the necessary credentials in GitHub repo secrets and setup the following workflow to run on push to my data/ directory:

name: update_firestore

on:
  push:
    branches:
      - main
    paths:
      - data/**.xlsx

jobs:
  build:
    runs-on: ubuntu-latest

    steps:
      - name: checkout repo content
        uses: actions/checkout@v2 # checkout the repository content to github runner.

      - name: setup python
        uses: actions/setup-python@v4
        with:
          python-version: '3.*' # install the latest python version

      - name: install python packages
        run: |
          python -m pip install --upgrade pip
          pip install -r requirements.txt

      - name: execute python script
        env:
          TYPE: service_account
          PROJECT_ID: ${{ secrets.PROJECT_ID }}
          PRIVATE_KEY_ID: ${{ secrets.PRIVATE_KEY_ID }}
          PRIVATE_KEY: ${{ secrets.PRIVATE_KEY }}
          CLIENT_EMAIL: ${{ secrets.CLIENT_EMAIL }}
          TOKEN_URI: ${{ secrets.TOKEN_URI }}
        run: python src/update_database.py -n ideas -delete -add

The Problem

I keep getting the following error:

Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.10.7/x64/lib/python3.10/site-packages/firebase_admin/credentials.py", line 96, in __init__
    self._g_credential = service_account.Credentials.from_service_account_info(
  File "/opt/hostedtoolcache/Python/3.10.7/x64/lib/python3.10/site-packages/google/oauth2/service_account.py", line 221, in from_service_account_info
    signer = _service_account_info.from_dict(
  File "/opt/hostedtoolcache/Python/3.10.7/x64/lib/python3.10/site-packages/google/auth/_service_account_info.py", line 58, in from_dict
    signer = crypt.RSASigner.from_service_account_info(data)
  File "/opt/hostedtoolcache/Python/3.10.7/x64/lib/python3.10/site-packages/google/auth/crypt/base.py", line 113, in from_service_account_info
    return cls.from_string(
  File "/opt/hostedtoolcache/Python/3.10.7/x64/lib/python3.10/site-packages/google/auth/crypt/_python_rsa.py", line 171, in from_string
    raise ValueError("No key could be detected.")
ValueError: No key could be detected.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/runner/work/IRIS/IRIS/src/update_database.py", line 9, in <module>
    import fire
  File "/home/runner/work/IRIS/IRIS/src/fire/__init__.py", line 35, in <module>
    cred = credentials.Certificate(create_keyfile_dict())
  File "/opt/hostedtoolcache/Python/3.10.7/x64/lib/python3.10/site-packages/firebase_admin/credentials.py", line 99, in __init__
    raise ValueError('Failed to initialize a certificate credential. '
ValueError: Failed to initialize a certificate credential. Caused by: "No key could be detected."
Error: Process completed with exit code 1.

My Attempted Solutions

I have tried a variety of approaches including what I show above, just hardcoding each of the secrets, and copying the .json formatted credentials directly as a single secret. I know there are some issues dealing with multiline environment variables which the PRIVATE_KEY is. I have tried:

1. Pasting the PRIVATE_KEY str directly from the download firebase provides which includes \n
2. Removing escape characters and formatting the secret like:

-----BEGIN PRIVATE KEY-----
BunC40fL3773R5AndNumb3r5
...
rAndomLettersANDNumb3R5==
-----END PRIVATE KEY-----

I feel like the solution should be pretty straight-forward but have been struggling and my knowledge with all this is a bit limited.

Thank you in advance!

Answer 1

After hours of research, I found an easy way to store the Firestore service account JSON as a Github Secret.

Step 1 : Convert your service account JSON to base-64

Let's name the base-64 encoded JSON SERVICE_ACCOUNT_KEY. There are two ways to get this value:

Method 1 : Using command line

cat path-to-your-service-account.json | base64 | xargs

This will return a single line representing the encoded service account JSON. Copy this value.

Method 2 : Using python

import json
import base64


service_key = {
    "type": "service_account",
    "project_id": "xxx",
    "private_key_id": "xxx",
    "private_key": "-----BEGIN PRIVATE KEY-----\nxxxxx\n-----END PRIVATE KEY-----\n",
    "client_email": "xxxx.com",
    "client_id": "xxxx",
    "auth_uri": "xxxx",
    "token_uri": "xxxx",
    "auth_provider_x509_cert_url": "xxxx",
    "client_x509_cert_url": "xxxx"
}

# convert json to a string
service_key = json.dumps(service_key)

# encode service key
SERVICE_ACCOUNT_KEY= base64.b64encode(service_key.encode('utf-8'))

print(SERVICE_ACCOUNT_KEY)
# FORMAT: b'a_long_string'

Copy only the value between the quotes. (copy a_long_string instead of b'a_long_string')

Step 2 : Create your environment variable

I am using dotenv library to read environment variables. You will have to install it first using pip install python-dotenv. Also add this dependency in your requirements.txt for github actions.

• Create a Github repository secret SERVICE_ACCOUNT_KEY which will store the base-64 value.
• In your Github YML file, add the environment variable:

      - name: execute py script 
        env:
          SERVICE_ACCOUNT_KEY: ${{ secrets.SERVICE_ACCOUNT_KEY }}
        run: python src/main.py 

• To be able to test your program locally, you might also want to add SERVICE_ACCOUNT_KEY together with its value to your .env file (which should be in the root directory of your project). Remember to add .env to your .gitignore file to avoid exposing your key on Github.

Step 3 : Decoding the service key

You will now need to get the value of SERVICE_ACCOUNT_KEY in your Python code and convert this value back to a JSON. I am using the dotenv library to get the value of the SERVICE_ACCOUNT_KEY.

import json
import base64
import os
from dotenv import load_dotenv, find_dotenv

# get the value of `SERVICE_ACCOUNT_KEY`environment variable
load_dotenv(find_dotenv())
encoded_key = os.getenv("SERVICE_ACCOUNT_KEY")

# decode
SERVICE_ACCOUNT_JSON = json.loads(base64.b64decode(encoded_key).decode('utf-8'))

# Use `SERVICE_ACCOUNT_JSON` later to initialse firestore db:
# cred = credentials.Certificate(SERVICE_ACCOUNT_JSON)
# firebase_admin.initialize_app(cred)