Upload to pypi from Gitlab Pipelines

Question

I'm trying to upload a package to pypi using a Gitlab CI job, but I cannot make it work :/ Anyone has a working example?

What I have tried so far in my .gitlab-ci.yaml (from my local machine all of them are working):

1. Twine with a .pypirc file

   - echo "[distutils]" >> ~/.pypirc
   - echo "index-servers =" >> ~/.pypirc
   - echo "    pypi" >> ~/.pypirc
   - echo "" >> ~/.pypirc
   - echo "[pypi]" >> ~/.pypirc
   - 'echo "repository: https://upload.pypi.org/legacy/" >> ~/.pypirc'
   - 'echo "username: ${PYPI_USER}" >> ~/.pypirc'
   - 'echo "password: ${PYPI_PASSWORD}" >> ~/.pypirc'
   - python3 setup.py check sdist bdist  # This will fail if your creds are bad.
   - cat ~/.pypirc
   - twine upload dist/* --config-file ~/.pypirc
   

2. Same as before but with $VARIABLE

   [...]
   - 'echo "username: $PYPI_USER" >> ~/.pypirc'
   - 'echo "password: $PYPI_PASSWORD" >> ~/.pypirc'
   [...]
   

3. Two options before but using python setup.py ... upload

4. twine upload dist/* -u $PYPI_USER -p $PYPI_PASSWORD
5. twine upload dist/* wiht TWINE_USERNAME and TWINE_PASSWORD environment variables.

... and always get a 403 Client Error: Invalid or non-existent authentication information. I'm running out of options...

Answer 1

I am simply using the TWINE_USERNAME and TWINE_PASSWORD variables, it worked out of the box.

This is the relevant part in my gitlab-ci.yml (replace the image with your desired one and of course change all the other stuff like stage, cache etc. to your needs):

pypi:
    image: docker.km3net.de/base/python:3
    stage: release
    cache: {}
    script:
        - pip install -U twine
        - python setup.py sdist
        - twine upload dist/*
    only:
        - tags

And add the environment variables in GitLab under Settings->CI/CD->Variables (https://your-gitlab-instance.oerg/GIT_NAMESPACE/GIT_PROJECT/settings/ci_cd):

Here is the successful pipeline:

Answer 2

I got this working, using a modified version of your code:

pypi:
  stage: upload
  script:
  - pip install twine
  - rm -rf dist
  - echo "[distutils]" >> ~/.pypirc
  - echo "index-servers =" >> ~/.pypirc
  - echo "    nexus" >> ~/.pypirc
  - echo "" >> ~/.pypirc
  - echo "[nexus]" >> ~/.pypirc
  - echo "${PYPI_REPO}" >> ~/.pypirc
  - echo "${PYPI_USER}" >> ~/.pypirc
  - echo "${PYPI_PASSWORD}" >> ~/.pypirc
  - python3 setup.py check sdist bdist  # This will fail if your creds are bad.
  - python setup.py sdist bdist_wheel
  - twine upload -r nexus dist/*.tar.gz

The difference is I didn't use the "'" and got rid of the colons in the yaml; instead I set the values of the secrets as e.g., username: myuser

Answer 3

If problems with EOF appears, make sure to change Settings/Repository/Tags to be protected, so they will work again. I've posted here a more complete description.