Coverage badge in Gitlab CI with Python coverage always unknown

Question

I am trying to show a coverage badge for a Python project in a private Gitlab CE installation (v11.8.6), using coverage.py for Python. However, the badge always says unknown.

This is the relevant job in my .gitlab-ci.yaml file:

coverage:
    stage: test
    before_script:
        - pip3.6 install coverage
        - mkdir -p public
    script:
        - coverage run --source=my_service setup.py test
        - coverage report | tee public/coverage.txt
    artifacts:
        paths:
            - public/coverage.txt
    coverage: '/TOTAL\s+\d+\s+\d+\s+(\d+%)/'

I expected the badge to show the actual coverage at this URL, so this is what I have entered in the project settings under General/Badges:

http://<privategitlaburl>/%{project_path}/badges/%{default_branch}/coverage.svg?job=coverage

I read these instructions using Gitlab pages. However, I do not want to use pages just for this purpose, and I am dealing with a Python project.

According to the example in the CI/CD settings, and in this post, the regex in the coverage entry should work. which I could confirm by trying it locally:

$ grep -P "TOTAL\s+\d+\s+\d+\s+(\d+%)" public/coverage.txt
TOTAL                                           289     53    82%

I also tried the same regex in the field Test coverage parsing in the project settings under CI/CD/Pipeline settings, but the badge shown on that same page keeps showing unknown.

The documentation is not quite clear to me, as it does not describe the whole procedure. It is clear how to use a badge once created, and there is a manual for publishing a coverage report to pages, but there seems to be no clear path from extracting the score to displaying the badge.

Should I use the coverage entry in my .gitlab-ci.yaml file or fill in the regex in the pipeline settings?

Either way, is Gitlab CI supposed to update the coverage badge based on that, or do I need to use additional tools like coverage-badge to do so?

Where is the extracted coverage score supposed to be reported; how can I find out if my regex works?

Answer 1

I finally got the coverage badge displaying a percentage instead of unknown today for my python project. Here's the relevant content from my .gitlab-ci.yml:

job:
 script:
  - 'python -m venv venv'
  - '.\venv\Scripts\activate'
  - 'python -m pip install -r requirements.txt'
  - 'coverage run --source=python_project -m unittest discover ./tests'
  - 'coverage report --omit=things_that_arent_mine/*'
  - 'coverage xml'
 artifacts:
  reports:
   cobertura: 'coverage.xml'

I'm also using the regex for gcovr listed in the repo CI/CD Settings > General Pipelines > Test coverage parsing which I found after reading this as well as the second to last comment on this:

^TOTAL.*\s+(\d+\%)$

In the repo General Settings > Badges, my badge link is:

http://gitlab-server/%{project_path}/-/jobs

and my badge image url is:

http://gitlab-server/%{project_path}/badges/%{default_branch}/coverage.svg

I don't quite know what the Cobertura report artifact is for (I think it specifically has to do with merge requests) but I have it in there because the tutorials took me down that road. I did confirm that removing the following from the .gitlab-ci.yml doesn't break the badge or coverage number on the jobs page:

  - 'coverage xml'
 artifacts:
  reports:
   cobertura: 'coverage.xml'

While removing or commenting:

- 'coverage report --omit=things_that_arent_mine/*'

does break the badge as well as the coverage number displayed on the CI/CD jobs page of the repo. I also tried some regex variations that I tested in rubular but the only one that didn't cause gitlab to barf was the gcovr one.

Hopefully this helps out, it was kind of difficult and arduous to piece together what was needed for this badge to work on a python project.

EDIT: Also just figured out how to add some sexy precision to the coverage percentage number. If you change the coverage report line of the .gitlab-ci.yml to:

- 'coverage report --omit=things_that_arent_mine/* --precision=2'

And the regex in CI/CD Settings > General Pipelines > Test coverage parsing to:

^TOTAL.+?(\d+.\d+\%)$

That should give you a very precise coverage number that effectively no one but you and I will care about. But by gosh we'll know for sure if coverage is 99.99% or 100%.