TestNG continues to execute testclasses even though a previous class failed

Question

Why does TestNG continue to execute the test if one TestCase (Class) has failed?

For example the testoutput is:

1 test passed, 1 test failed.(3599,0 s)
    TestSuite FAILED
        run FAILED: check5=3 Expected: <3> got: <5>
        run passed (1.0s)    // <--- this should not have been executed

why is the second run even executed? my testngsuite.xml:

<suite name="TestSuite_03">
    <test name="TestCase_17">
        <groups>
            <run><include name="functest"/></run>
        </groups>
        <classes>
            <class name="TestStep_003" desc="will fail" />
            <class name="TestStep_012" desc="will pass" />
        </classes> ...

I am using Maven, TestNG and Java via NetBeans

my structure:

public abstract class TestCommon
{
    @BeforeSuite(groups={"functest})
    public void BeforeSuite()
    {
        // clean report folder
    }
    @BeforeTest(groups={"functest})
    public void BeforeTest()
    {
        // start selenium browser
    }
    @AfterMethod(groups={"functest})
    public void AfterMethod()
    {
        // check for failure and capture screenshot
    }
    @AfterTest(groups={"functest})
    public void AfterTest()
    {
        // close browser
    }
}


public class TestStep_003 extends TestCommon
{
    @Test(groups = {functest})
    public void run()
    {
        assertThat(5, Matchers.equalTo(3)); // will fail
    }
}
public class TestStep_012 extends TestCommon
{
    @Test(groups = {functest})
    public void run()
    {
        assertThat(5, Matchers.equalTo(5)); // will pass
    }
}

Answer 1

You should design your test cases to be independent, because modularity decreases complexity: If tests are dependent, you would always need to run all test cases if the last one failed, and needed to check all test cases whether they might have caused the error/failure.

With independent tests, you can run all of them, e.g. during a nightly build, even if some of them fail. Or execute a specific one in isolation. And you need not worry about the ordering of tests. That is why almost all test runners execute all tests and make no guarantee about their order.

This applies especially for unit tests. For a few integration tests and acceptance tests some dependency might be acceptable. In that case, use TestNG's dependency feature.