PHPUnit: Multiple assertions in a single test, only first failure seen

Question

The next weirdness I'm seeing with PHPUnit:

class DummyTest extends PHPUnit_Framework_TestCase {
    public function testDummy() {
        $this->assertTrue(false, 'assert1');
        $this->assertTrue(false, 'assert2');
    }

    public function testDummy2() {
        $this->assertTrue(false, 'assert3');
    }
}

As soon as the first assertion fails in a test, the rest of the test is ignored.

So (with a simple call of phpunit DummyTest.php):

• The above code will display 2 tests, 2 assertions, 2 failures. What?

• If I make all the tests pass, then I'll get OK (2 tests, 3 assertions). Good.

• If I only make all the tests pass except for assert2, I get 2 tests, 3 assertions, 1 failure. Good.

I don't get it, but PHPUnit's been around for ages, surely it has to be me?

Not only are the counts not what I'd expect, only the error message for the first failed assert in the code above is displayed.

(BTW, I'm analyzing the xml format generated by PHPUnit for CI rather than testing real code, hence the practice of multiple assertions in the one test.)

Answer 1

First off: That is expected behavior.

Every test method will stop executing once an assertion fails.

For an example where the opposite will be very annoying*:

class DummyTest extends PHPUnit_Framework_TestCase {
    public function testDummy() {
        $foo = get_me_my_foo();
        $this->assertInstanceOf("MyObject", $foo);
        $this->assertTrue($foo->doStuff());
    } 
}

if phpunit wouldn't stop after the first assertion you'd get an E_FATAL (call to a non member function) and the whole PHP process would die.

So to formulate nice assertions and small tests it's more practical that way.

For another example:

When "asserting that an array has a size of X, then asserting that it contains a,b and c" you don't care about the fact that it doesn't contain those values if the size is 0.

If a test fails you usually just need the message "why it failed" and then, while fixing it, you'll automatically make sure the other assertions also pass.

---

On an additional note there are also people arguing that you should only have One Asssertion per Test case while I don't practice (and I'm not sure if i like it) I wanted to note it ;)