How to Install Private Python Package as Part of Build

Question

I have a fairly large private python package I just finished creating. I'd like to install it as part of my build process for an app in a Docker container (though this isn't so important). The package source is quite large, so ideally I'd avoid downloading/keeping the whole source.

Right now, I've been just passing around the package source along with my app, but this is unwieldy and hopefully temporary. What's a better way? git submodule/subtree? I'm pretty new to this.

Answer 1

When you have a requirements.txt file and you want to do pip install -r requirements.txt in your Dockerfile, what I do is:

• Create a GITHUB_TOKEN env var. Put it in .env
• Add it as an ARG in your docker-compose.yml file
• Before installing your pip packages:

RUN git config --global url."https://${GITHUB_OAUTH_TOKEN}@github.com/".insteadOf "https://github.com/"

That allows to patch all URLs coming from requirements.txt

Answer 2

If you don't want to mess with ssh keys use http and token authentication, but you don't want to expose your token in your repo I suggest to pass the token as an environment variable.

Add to your Dockerfile:

ENV GITOKEN "$GITOKEN"
RUN bash -c "pip install -r <(envsubst < requirements.txt)"

Set your requirements as this:

+plotly==2.7.0
+git+https://[email protected]/githubuser/your_package.git@master