Viewing Unit Test and Coverage Reports Generated in AWS CodeBuild

Question

I am using AWS CodeBuild to run unit tests for my python project using pytest. I am using the --junitxml argument and the pytest-cov package to generate test reports and coverage reports that I've listed as artifacts in my buildspec.yml.

I've used Jenkins in the past to do this and Jenkins provides some nice graphs and tables showing test result history and coverage history as well as results from the most recent test.

Is there a good way to view the reports generated by my CodeBuild project? I haven't found anything in CodeBuild or CodePipeline directly. Do I have to use a separate tool that can ingest the report files? If so, what are some tools for this?

Answer 1

CodeBuild recently announced support for test reports. Copied from the blog post:

The reports can be in the JUnit XML or Cucumber JSON format. You can view metrics such as Pass Rate %, Test Run Duration, and number of Passed versus Failed/Error test cases in one location. Builders can use any testing frameworks as long as the reports are generated in the supported formats.

---

Two things needs to be updated to accomplish this. First, add some configuration to the buildspec.yml file:

reports:
  SurefireReports: # CodeBuild will create a report group called "SurefireReports".
    files: #Store all of the files
      - '**/*'
    base-directory: 'target/surefire-reports' # Location of the reports 

Secondly, CodeBuild needs some additional IAM permissions:

{
    "Statement": [
        {
            "Resource": "arn:aws:codebuild:your-region:your-aws-account-id:report-group/my-project-*", 
            "Effect": "Allow",
            "Action": [
                "codebuild:CreateReportGroup",
                "codebuild:CreateReport",
                "codebuild:UpdateReport",
                "codebuild:BatchPutTestCases"
            ]
        }
    ]
}