Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Make Android jenkins build fail if tests fail

I have seen several post regarding making a build in Jenkins fail if the unit test execution fail (e.g. this one). It turns out that by default Jenkins reports builds with failing tests as unstable and some people do not like that. This, however, will be perfectly fine for me. I just want to be able to easily differentiate builds with passing tests from such with failing tests.

And here is the catch: I am developing for Android so my build is configured following this page. Basically the tests are run with the following command:

ant all clean emma debug install test

As result coverage report is generated and published in Jenkins.

All posts I have read about configuring the Jenkins result according to tests results were dealing with ant task manipulation. However, if we look at the android build.xml the Android tests are run with adb command: adb shell am instrument .... I don't know how to configure this command to print the tests results. It can be configured to print the coverage report. I have already done that, but was never able to make the build fail according to the coverage report.

I hope somebody else also faced the same problem and managed to solve it. Any guidance will be most appreciated.

like image 642
Boris Strandjev Avatar asked Apr 03 '12 16:04

Boris Strandjev


People also ask

Why your tests may pass locally but fail in Jenkins?

This is most likely due to cached dependencies on the Jenkins server and the new dependencies didn't have their versions updated. If you have internal libraries or domains, then they may be causing this issue.

How does Jenkins handle build failure?

In Jenkins, in the pipeline where failure occurred, in the pane, select the latest build, and click Console Output. On the Console Output page, check the logs to find the reason for the failure. If required, update the parameters in the deployment input configuration file.

How do you fail a Jenkins job?

The easiest option would be running your JMeter test using Taurus tool as a wrapper. Taurus has powerful and flexible Pass/Fail Criteria subsystem where you can configure any combination of test failure thresholds and if any failure occurs - Taurus will exit with non-zero exit status.


2 Answers

Consider using 'the-missing-android-xml-junit-test-runner' - the idea being to produce jUnit reports that Jenkins understands. Disclaimer: I do have no personal experience with adb. However, I do know from experience that Jenkins deals with properly generated jUnit reports very gracefully.

like image 175
malenkiy_scot Avatar answered Oct 11 '22 19:10

malenkiy_scot


This is possible, configure the server using the following documentation:

https://wiki.jenkins-ci.org/display/JENKINS/Android+Emulator+Plugin

See "Parsing monkey output" in the link.

Addition, Jenkins can test instrumentation, below have a link of a good place to start in the learning of instrumentation:

http://developer.android.com/resources/tutorials/testing/helloandroid_test.html

I hope it is useful.

screenshot of my project jenkins

like image 25
Guillermo Tobar Avatar answered Oct 11 '22 18:10

Guillermo Tobar