Selenium Java (maven project): TestNG results differs from ReportNG

Question

I tried running testng.xml and the results were:

===============================================

Default test

Tests run: 14, Failures: 6, Skips: 0

Default suite

Total tests run: 14, Failures: 6, Skips: 0

===============================================

Now, I disabled deafult TestNG listener and added ReportNG listner in testng.xml. The testng.xml. looks like this:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="Suite">
    <listeners>
        <listener class-name="org.uncommons.reportng.HTMLReporter" />
        <listener class-name="org.uncommons.reportng.JUnitXMLReporter" />
    </listeners>
    <test name="Test">
        <classes>
            <class name=".URL_Daily" />
        </classes>
    </test> <!-- Test -->
</suite> <!-- Suite -->

Following the steps, I added velocity, guice and reportng dependencies in maven's pom.xml.

Upon executing the testsuite testng.xml, following folders were created (marked in red box).

Upon opening index.html the results are like this:

As expected, an output folder should have been created by ReportNG which is not noticed in my scenario. Secondly, the results are different.

In addition, the report index.html doesn't look how actually it should be. Can somebody tell what's wrong?

Some details for your reference:

OS: Windows 7

Guice.jar version: guice-4.1.0

ReportNG Version: reportng-1.1.4

Velocity version: velocity-dep-1.4

TestNG Version: testng-6.11

Selenium Version: selenium-java-3.5.3

Eclipse: eclipse oxygen

My test case is as follows:

public class MwSites {
WebDriver driver;

@BeforeTest     
public void setup ()    
{
    System.setProperty("webdriver.chrome.driver", "F:\\Automation\\Drivers\\Selenium Drivers\\chromedriver_win32\\chromedriver.exe");
    ChromeOptions options = new ChromeOptions();
    options.addArguments("test-type");
    options.addArguments("start-maximized");
    options.addArguments("--js-flags=--expose-gc");  
    options.addArguments("--enable-precise-memory-info"); 
    options.addArguments("--disable-popup-blocking");
    options.addArguments("--disable-default-apps");
    options.addArguments("test-type=browser");
    options.addArguments("disable-infobars");
    driver = new ChromeDriver(options);
    driver.manage().timeouts().implicitlyWait(60, TimeUnit.SECONDS);
}

@AfterTest
public void Quit () throws InterruptedException
{
    driver.quit();
}

@Test(priority = 0)     
public void MI_Demo () throws InterruptedException
{
    driver.navigate().to("http://demo.movingwalls.com/mi/#/login");
    Assert.assertEquals("Login", driver.getTitle());
    if (driver.getTitle()=="Login"){
        System.out.println("Failed to access MI in demo environment");
    }
    else{
        System.out.println("MI is successfully accessed in demo environment");
    }
}

Answer 1

As your tests got skipped, you may want to check test results xml first to find out the root cause (exception).

On the other hand, I'm just wondering what's the initial intention of using a library, which is not maintained for 4 years? ReportNG is dead. It's a fact. And if you take a look at its sources, you'll see which TestNG version it uses (6.8). You could easily assume that its reporter is based on old TestNG API.

When you add TestNG 6.11 as a dependency, the probability of a jar hell occurrence is quite high. And I wouldn't be surprised, if it's a root cause of your issue.

If you haven't decided which reporting system to use yet, I'd recommend to take a look at a modern Allure 2 framework.

Otherwise, just check a stacktrace first to understand the root cause of your issue.