Show Coverage in github PR

Question

We run tests on each pull request in github.

It would be very cool if the coverage would be visible if you are looking at the changes of a pull request.

While looking at the diff in the PR review I would like to see which line was covered and which line not.

We use pytest and travis.

I found this: https://github.com/TheKevJames/coveralls-python But AFAIK this shows the coverage reports somewhere else, not in the "Files changed" tab of the PR.

This question is not about reaching a overall coverage of X percent. This question is about seeing which lines where covered and which lines where not covered.

Answer 1

You can achieve this using Github Actions and the Github Check runs API.

Generic flow of the Github Action workflow will be:

1. Run pytest on the changed files and then generate coverage report.
2. Use coverage report to annotate the commit.

The generated report looks something like this:

I have created a repository : pytest-annotate-pr to demonstrate how this can be done. Note that, this is just a demonstration of how this can be done, and it's far from production ready. For example, this currently only works when a commit is pushed in a PR :) It does not work when branches are merged into main. The above screenshot is from one of the PR raised against main.

If you work on this and make a production ready workflow, please consider making it into a Github Action.

Answer 2

A partial solution: In one of the projects I work in, we have a codecov action set up. This action has a feature to annotate lines that were not covered in the "Files changed" tab:

This only shows you which lines were not covered, but maybe that's better than nothing?

Edit: I just now noticed that Julian already suggested codecov in the comments and that you said you wanted something non-commercial. I can only add there that codecov is free for public repos as well as teams with up to five members. If you have a private repo and a bigger team and do not want to pay for this service, this doesn't help you.

I think I should still keep the answer here in case someone finds this question when searching for a solution for an open source project.