Prevent Bamboo from failing job when failing tests are moved to quarantine

Question

On Bamboo I have MSBuild job for building and running tests.

It works fine - but I want to use test quarantine option so when I have some test moved to quarantine (and this moved tests are only what fail) job will be marked as successful

I moved one failing test to quarantine but job is marked as failed

Configuration:

In Job -> Tasks I have Command named Test runner:

Executable = VSTest.Console

Argument = "WebServiceTestClient.dll" /Logger:trx

And after running job with failing test quarantined I get log

01-Apr-2014 10:59:44  Total tests: 62. Passed: 61. Failed: 1. Skipped: 0. 
01-Apr-2014 10:59:44  Test Run Failed. 
01-Apr-2014 10:59:44  Test execution time: 10.1296 Minutes 
01-Apr-2014 10:59:44  Failing task since return code of [C:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\IDE\CommonExtensions\Microsoft\TestWindow\VSTest.Console.exe "WebServiceTestClient.dll" /Logger:trx] was 1 while expected 0 
01-Apr-2014 10:59:44  Finished task 'Test runner' 
01-Apr-2014 10:59:44  Starting task 'Parse test logs' of type 'com.atlassian.bamboo.plugin.dotnet:mstest' 
01-Apr-2014 10:59:44  Parsing test results... 
01-Apr-2014 10:59:45  Failing task since 1 failing test cases were found. 

Job is marked failed but then:

01-Apr-2014 10:59:45  Changing Task Result to SUCCESS as all failed tests were quarantined. 
01-Apr-2014 10:59:45  Finished task 'Parse test logs' 
01-Apr-2014 10:59:45  Running post build plugin 'NCover Results Collector' 
01-Apr-2014 10:59:45  Running post build plugin 'Clover Results Collector' 
01-Apr-2014 10:59:45  Running post build plugin 'Artifact Copier' 
01-Apr-2014 10:59:45  Finalising the build... 

Unfortunately the output is:

Answer 1

The problem is the job result is a failure.

You can run the tests via an ant script and as long as the target executes successfully you will pass. In my bamboo setup I run all the unit tests like this and after running the test I look for the test results and generate a failed test result if one is missing (for ex, if the cppunit exe threw an exception and didn't write out its results).

Then the final stage is set to check for the test results (this can be the same stage you run the tests from or a later one).