Aggregating results of downstream is ‘no test’ in Jenkins

Question

After running the main project, every downstream project has test result, but the "Latest Aggregated Test Result" is no tests. How to configure the Jenkins to make all the test results display in aggregated list?

Answer 1

Aggregate downstream test results is not obvious, and not documented. The steps below are synthesized from How To Aggregate Downstream Test Results in Hudson.

To aggregate, you need to archive an artifact in the upstream job, fingerprint the artifact, and then pass the artifact from the upstream job to the downstream job. In my own words:

the shared, finger-printed artifact "ties" the jobs together and allows the upstream job to see the downstream test results

To show this, we can make a very simple flow between two free-style jobs, Job_A and Job_B.

Upstream

Job_A will run and create an artifact named some_file.txt. We're not aggregating the value/contents of some_file.txt, but it needs to be finger-printed and so it cannot be empty. Job_A will then trigger a build of Job_B.

Job_A's configuration:

1. Execute shell:

   echo $(date) > some_file.txt
   

2. Archive the artifacts:

   • set Files to archive to the file, some_file.text

3. Aggregate downstream test results:

   • check the Automatically aggregate... option

4. Build other projects:

   • set Projects to build to Job_B

5. Record fingerprints of files to track usage:

   • set Files to fingerprint to some_file.txt

Downstream

Job_B will run, copy the file some_file.txt from the upstream job that triggered this run, echo out some mock test results to an XML file, then publish that XML result file. It's the published results that will get aggregated back into Job_A.

Job_B's configuration:

1. Copy artifacts from another project:

   Project name	Job_A
   Which build	Upstream build that triggered this job
   Artifacts to copy	some_file.txt
   Fingerprint Artifacts	✔

2. Execute shell:

   XML_VAR='<testsuite tests="3">
     <testcase classname="foo" name="ASuccessfulTest"/>
     <testcase classname="foo" name="AnotherSuccessfulTest"/>
     <testcase classname="foo" name="AFailingTest">
       <failure type="ValueError">Not enough foo!!</failure>
     </testcase>
   </testsuite>'
   
   echo $XML_VAR > results.xml
   

3. Publish JUnit test result report:

   • set Test report XMLs with the file, results.xml

This should be sufficient to have Job_A aggregate Job_B's test results. I'm not sure if there's a way/plugin to change Job_A's status based on downstream results (like if Job_B failed, then Job_A would retroactively fail).

Answer 2

For Scripted Pipeline, Say I have:

one upstream job - mainJob

two downstream jobs - downStreamJob1 and downStreamJob2.

To aggregate test result from downstreamJob1 and downStreamJob2, here is what the Jenkinsfile will look like:

downStreamJob1 Jenkinsfile - Archive and fingerprint the test result xml

archiveArtifacts allowEmptyArchive: true, 
artifacts: **/test-results/test/*.xml, 
fingerprint: true, defaultExcludes: false

downStreamJob2 Jenkinsfile - Archive and fingerprint the test result xml

archiveArtifacts allowEmptyArchive: true, 
artifacts: **/output/junit-report/*.xml,  
fingerprint: true, defaultExcludes: false

The artifacts path used Fileset to grab all test report XML. Read more about fileset HERE

mainJob Jenkinsfile - Copy artifact from each of the downstream jobs

copyArtifacts filter: 'build/test-result/test/*.xml', fingerprintArtifacts: true, projectName: 'downStreamJob1', selector: lastCompleted()

copyArtifacts filter: 'output/junit-report/*.xml', fingerprintArtifacts: true, projectName: 'downStreamJob2', selector: lastCompleted()

The best way to make sure you have the right path for filter and artifacts is to navigate to the artifact in each downstream job using this url $BUILD_URL/artifact/ where BUILD_URL is Full URL of this build, like http://server:port/jenkins/job/foo/15/