How to mark the jenkins build unstable with groovy postbuild

Question

I'm running test cases with nosetests in Jenkins. In general, it will have 100 test cases and I want to mark the build unstable when less than 20 test cases failed. If more than 20 test cases failed, then, mark the build failed.

The command I ran:

nosetests test.py --tc-file config.yml --tc-format yaml

First of all, I tried to just change the status of the build to Unstable but it still failed.

The groovy script I used:

manager.addWarningBadge("Thou shalt not use deprecated methods.")
manager.createSummary("warning.gif").appendText("<h1>You have been warned!</h1>", false, false, false, "red")
manager.buildUnstable()

The first two lines of code are executed, but the job is still marked as Failed.

Is there anything wrong with my jenkins config? Or the groovy postbuild plugin does not work with nosetest?

This is the console output:

FAILED (failures=2)
Build step 'Execute shell' marked build as failure
Build step 'Groovy Postbuild' marked build as failure
Finished: FAILURE

Answer 1

As DevD outlined, FAILED is a more significant build state than UNSTABLE. This means calling manager.buildUnstable() or manager.build.setResult(hudson.model.Result.UNSTABLE) after a step failed will still leave the build result FAILED.

However, you can override a failed build result state to be UNSTABLE by using reflection:

manager.build.@result = hudson.model.Result.UNSTABLE

---

Below example iterates over the build log lines looking for particular regex. If found it which will change (downgrade) build status, add badges & append to the build summary.

errpattern = ~/TIMEOUT - Batch \w+ did not complete within \d+ minutes.*/;
pattern = ~/INSERT COMPLETE - Batch of \d+ records was inserted to.*/;
manager.build.logFile.eachLine{ line ->
    errmatcher=errpattern.matcher(line)
    matcher=pattern.matcher(line)
    if (errmatcher.find()) {
        // warning message
        String errMatchStr = errmatcher.group(0) // line matched
        manager.addWarningBadge(errMatchStr);
        manager.createSummary("warning.gif").appendText("<h4>${errMatchStr}</h4>", false, false, false, "red");
        manager.buildUnstable();
        // explicitly set build result
        manager.build.@result = hudson.model.Result.UNSTABLE
    } else if (matcher.find()) {
        // ok
        String matchStr = matcher.group(0) // line matched
        manager.addInfoBadge(matchStr);
        manager.createSummary("clipboard.gif").appendText("<h4>${matchStr}</h4>", false, false, false, "black");
    }
}

Note: this iterates over every line, so assumes that these matches are unique, or you want a badge & summary appended for every matched line!

Post-build result is:

Build step 'Execute Groovy script' marked build as failure
Archiving artifacts
Build step 'Groovy Postbuild' changed build result to UNSTABLE
Email was triggered for: Unstable

Answer 2

Actually It is the intended way to work.

Preference FAILED -> UNSTABLE -> SUCCESS

using groovy post build we can change the lower result(SUCCESS) to higher preference(FAILED/UNSTABLE).. not vise versa.

as workaround after your Nosetest ,add an execute shell and "exit 0". so always your result will be the lower preference. now by your post build groovy script decide your exit criteria based on test results. This is actually a tweak.. will explore more and update you on this.