Jenkins Build Script exits after Google Test execution

Question

I am building a Qt GUI application via Jenkins. I added 3 build steps:

• Building the test executable
• Running the test executable
• compiling a coverage report with gcovr

For some reason, the shell task for running the test executable stops after execution. Even a simple echo does not run after. The tests are written with Google Test and output xUnit XML files, which are analyzed after the build. Some tests start the applications user interface, so I installed the jenkins xvnc plugin to get them to run.

The build tasks are as follows:

Build

cd $WORKSPACE/projectfiles/QMake
sh createbin.sh

Test

cd $WORKSPACE/bin
./Application --gtest_output=xml

Coverage Report

cd $WORKSPACE/projectfiles/QMake/out
gcovr -x -o coverage.xml

Now, an echo at the end of the first build task is correctly printed, but an echo at the end of the second is not. The third build task is therefore not even run, although the Google Test output is visible. I thought that maybe the problem is that some of the Google Tests fail, but why whould the script stop executing just because the tests fail?

Maybe someone can give me a hint on why the second task stops.

Edit

The console output looks like this:

Updating svn://repo/ to revision '2012-11-15T06:43:15.228 -0800'
At revision 2053
no change for svn://repo/ since the previous build
Starting xvnc
[VG5] $ vncserver :10

New 'ubuntu:10 (jenkins)' desktop is ubuntu:10

Starting applications specified in /var/lib/jenkins/.vnc/xstartup
Log file is /var/lib/jenkins/.vnc/ubuntu:10.log

[VG5] $ /bin/sh -xe /tmp/hudson7777833632767565513.sh
+ cd /var/lib/jenkins/workspace/projectfiles/QMake
+ sh createbin.sh
... Compiler output ...
+ echo Build Done
Build Done
[VG5] $ /bin/sh -xe /tmp/hudson4729703161621217344.sh
+ cd /var/lib/jenkins/workspace/VG5/bin
+ ./Application --gtest_output=xml
Xlib:  extension "XInputExtension" missing on display ":10".
[==========] Running 29 tests from 8 test cases.
... Test output ...
 3 FAILED TESTS
Build step 'Execute shell' marked build as failure
Terminating xvnc.
$ vncserver -kill :10
Killing Xvnc4 process ID 1953
Recording test results
Skipping Cobertura coverage report as build was not UNSTABLE or better ...
Finished: FAILURE

Answer 1

Generally, if one Build Step fails, the rest will not be executed.

Pay attention to this line from your log:

[VG5] $ /bin/sh -xe

The -x makes the shell print each command in console before execution.
The -e makes the shell exit with error if any of the commands failed.

A "fail" in this case, would be a return code of not 0 from any of the individual commands.
You can verify this by running this directly on the machine:

./Application --gtest_output=xml
echo $?

If the echo $? displays 0, it indicates successful completion of the previous command. If it displays anything else, it indicates an error code from the previous command (from ./Application), and Jenkins treats it as such.

Now, there are several things at play here. First is that your second Build Step (essentially a temporary shell script /tmp/hudson4729703161621217344.sh) is set to fail if one command fails (the default behaviour). When the Build Step fails, Jenkins will stop and fail the whole job.

You can fix this particular behaviour by adding set +e to the top of your second Build Step. This will not cause the script (Build Step) to fail due to individual command failure (it will display an error for the command, and continue).

However, the overall result of the script (Build Step) is the exit code of the last command. Since in your OP, you only have 2 commands in the script, and the last is failing, it will cause the whole script (Build Step) to be considered a failure, despite the +x that you've added. Note that if you add an echo as the 3rd command, this would actually work, since the last script command (echo) was successful, however this "workaround" is not what you need.

What you need is proper error handling added to your script. Consider this:

set +e
cd $WORKSPACE/bin && ./Application --gtest_output=xml
if ! [ $? -eq 0 ]; then
    echo "Tests failed, however we are continuing"
else
    echo "All tests passed"
fi

Three things are happening in the script:

1. First, we are telling shell not to exit on failure of individual commands

2. Then i've added basic error handling in the second line. The && means "execute ./Application if-and-only-if the previous cd was successful. You never know, maybe the bin folder is missing, or whatever else can happen. BTW, the && internally works on the same error code equals 0 principle

3. Lastly, there is now proper error handling for the result of ./Application. If the result is not 0, then we show that it had failed, else we show that it had passed. Note, this since the last command is not a (potentially) failing ./Application, but an echo from either of if-else possibilities, the overall result of the script (Built Step) will be a success (i.e 0), and the next Build Step will be executed.

BTW, you can as well put all 3 of your build steps into a single build step with proper error handling.

Yes... this answer may be a little longer than what's required, but i wanted you to understand how Jenkins and shell treat exit codes.