How can a Google Cloud Python function access a private Python package

Question

1. I am using Google Cloud Function using Python. Several other functions are in production.
2. However, for this, I have additionally created a custom Python package that is available on github as a private repo.
3. I need to install the package in the Google Function

WHAT I HAVE DONE

1. I run the Google Function in local using functions-framework
2. I have a requirements.txt which has a link to the package. This is done by adding the following line to requirements.txt:

    -e git+https://github.com/<repo>/<project>#egg=<project>

3. I run pip install -r requirements.txt. And the package is successfully installed.
4. Now in the python code of the function using import <pkg-name> works and I am able to access all the functions.

CHALLENGES WHEN PUSHING THE FUNCTION TO THE CLOUD

1. As per the documentation, to push the Cloud function to Google Cloud, I issue the command:

gcloud functions \
  deploy <function-name> \
  --trigger-http  \
  --entry-point <entry-function> \
  --runtime python37 \
  --project=<my-project>

As expected this gives an error because it does not have access to the private repo in git.

2. I created a Google Cloud Repository and linked it to the git repo, hoping that in some way I could specify that in the requirements.txt. Just do not know how.

3. I tried setting environment variables for username and password (not a good idea, I agree) in Google Cloud Function and specify them in the requirements.txt as:

    -e git+https://${AUTH_USER}:${AUTH_PASSWORD}@github.com/<repo>/<project>#egg=<project>

That too gave an error.

Any help or direction will be greatly appreciated.

Answer 1

While @marian.viadoi was correct in the above comment viz. Google Cloud Function cannot access a private git repo, I have implemented a workaround and sharing just in case it is suitable. Here is what has been done:

1. The Python package was prepared as per the documentation in (https://packaging.python.org/tutorials/packaging-projects/)
2. A binary distribution "whl" file was created using python3 setup.py bdist_wheel
3. This whl file was put in the google functions folder. I chose to put it under a dist folder of it.
4. In the requirements.txt I added the line: ./dist/xxxx.whl to other dependencies required by the cloud function
5. Pushed the cloud function to GCP via the gcloud deploy ..
6. On successful installation, the package and its dependencies get automatically installed in the virtual environment of the cloud function

This does not answer how to use a private repo in Google Functions (which is not possible), but the above steps ensure that a private package can be easily shared and updated amongst different cloud functions.