Logo Questions Linux Laravel Mysql Ubuntu Git Menu
HTML CSS JAVASCRIPT SQL PYTHON PHP BOOTSTRAP JAVA JQUERY R React Kotlin
 

Jenkins/Maven job - running the test suite twice?

Tags:

maven

jenkins

webdriver

testng

I have 2 tests, in a Maven project and I have Xvfb starting along the tests. What I want to achieve is the following: - start Xvfb so the tests run headless - run the tests.

But what happens, when I start the job is: - The tests run and fail, because they can't find a display, because xvfb isnt running - Xvfb starts - The tests run again

Here is my pom.xml

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <groupId>com.pragmaticqa.tests</groupId>
    <artifactId>functionalTests</artifactId>
    <version>1.0-SNAPSHOT</version>
    <packaging>jar</packaging>
    <name>functionalTests</name>
    <url>http://maven.apache.org</url>
    <properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <displayProps>target/selenium/display.properties</displayProps>
    </properties>
    <dependencies>
        <dependency>
            <groupId>org.seleniumhq.selenium</groupId>
            <artifactId>selenium-firefox-driver</artifactId>
            <version>2.32.0</version>
        </dependency>
        <dependency>
            <groupId>org.seleniumhq.selenium</groupId>
            <artifactId>selenium-java</artifactId>
            <version>2.32.0</version>
        </dependency>
        <dependency>
            <groupId>org.testng</groupId>
            <artifactId>testng</artifactId>
            <version>6.8</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.apache.maven.surefire</groupId>
            <artifactId>surefire</artifactId>
            <version>2.5</version>
            <type>pom</type>
        </dependency>
    </dependencies>
    <build>
        <!-- <pluginManagement> -->
        <plugins>
            <plugin>
                <groupId>org.eclipse.m2e</groupId>
                <artifactId>lifecycle-mapping</artifactId>
                <version>1.0.0</version>
                <configuration>
                    <lifecycleMappingMetadata>
                        <pluginExecutions>
                            <pluginExecution>
                                <pluginExecutionFilter>
                                    <groupId>org.codehaus.mojo</groupId>
                                    <artifactId>aspectj-maven-plugin</artifactId>
                                    <versionRange>[1.0,)</versionRange>
                                    <goals>
                                        <goal>test-compile</goal>
                                        <goal>compile</goal>
                                    </goals>
                                </pluginExecutionFilter>
                                <action>
                                    <execute />
                                </action>
                            </pluginExecution>
                        </pluginExecutions>
                    </lifecycleMappingMetadata>
                </configuration>
            </plugin>
            <plugin>
                <groupId>org.codehaus.mojo</groupId>
                <artifactId>selenium-maven-plugin</artifactId>
                <version>2.3</version>
                <executions>
                    <execution>
                        <id>xvfb</id>
                        <phase>pre-integration-test</phase>
                        <goals>
                            <goal>xvfb</goal>
                        </goals>
                        <configuration>
                            <display>:20</display>
                        </configuration>
                    </execution>
                    <execution>
                        <id>selenium</id>
                        <phase>pre-integration-test</phase>
                        <goals>
                            <goal>start-server</goal>
                        </goals>
                        <configuration>
                            <background>true</background>
                        </configuration>
                    </execution>
                </executions>
            </plugin>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-surefire-plugin</artifactId>
                <version>2.5</version>
                <configuration>
                    <includes>
                        <include>**/*Test*.java</include>
                    </includes>
                    <systemPropertyVariables>
                        <!-- <display.props>${displayProps}</display.props> -->
                        <!-- <display>:21</display> -->
                    </systemPropertyVariables>
                </configuration>
                <executions>
                    <execution>
                        <phase>integration-test</phase>
                        <goals>
                            <goal>test</goal>
                        </goals>
                        <configuration>
                            <skip>false</skip>
                        </configuration>
                    </execution>
                </executions>
            </plugin>

        </plugins>
        <!-- </pluginManagement> -->
    </build>
</project>

Here is my testng.xml (I dont know whether it has some role in a maven project):

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<test name="NewTest" preserve-order="true">
    <classes>
        <class name="AppTest">
            <methods>
                <include name="aNewPlaceIsBornAndThenEdited" />
            </methods>
        </class>
        <class name="AppTest2">
            <methods>
                <include name="twoUsersSignUp" />
            </methods>
        </class>
    </classes>
</test>

My github repository consists of the whole maven project with a target folder, where some copies of the test classes are always stored.

My jenkins job settings:

  • asksing for a git repository, link provided
  • Build whenever a SNAPSHOT dependency is built - this is on, but it doesnt make a difference whether its on or off
  • Maven options: -Xms256m -XX:MaxPermSize=256m -Xmx256m - this has also nothing to do with anything

console output:

Building in workspace /mnt/www/jenkins/jobs/Integration tests/workspace
Checkout:workspace / /mnt/www/jenkins/jobs/Integration tests/workspace - hudson.remoting.LocalChannel@7cd2da43
Using strategy: Default
Last Built Revision: Revision ac0ccfc8f2d3ab2a8c079cddec073c03dfb57dce (origin/HEAD, origin/master)
Fetching changes from 1 remote Git repository
Fetching upstream changes from origin
Seen branch in repository origin/HEAD
Seen branch in repository origin/master
Seen 2 remote branches
Commencing build of Revision ac0ccfc8f2d3ab2a8c079cddec073c03dfb57dce (origin/HEAD, origin/master)
Checking out Revision ac0ccfc8f2d3ab2a8c079cddec073c03dfb57dce (origin/HEAD, origin/master)
Warning : There are multiple branch changesets here
Parsing POMs
[workspace] $ /usr/lib/jvm/java-6-openjdk-amd64/bin/java -Xms256m -XX:MaxPermSize=256m -Xmx256m -Djava.awt.headless=true -cp /mnt/www/jenkins/plugins/maven-plugin/WEB-INF/lib/maven3-agent-1.3.jar:/mnt/www/jenkins/tools/hudson.tasks.Maven_MavenInstallation/Maven_3.0.5/boot/plexus-classworlds-2.4.jar org.jvnet.hudson.maven3.agent.Maven3Main /mnt/www/jenkins/tools/hudson.tasks.Maven_MavenInstallation/Maven_3.0.5 /var/cache/jenkins/war/WEB-INF/lib/remoting-2.28.jar /mnt/www/jenkins/plugins/maven-plugin/WEB-INF/lib/maven3-interceptor-1.3.jar 60610

<===[JENKINS REMOTING CAPACITY]===>channel started

log4j:WARN No appenders could be found for logger (org.apache.commons.beanutils.converters.BooleanConverter).
log4j:WARN Please initialize the log4j system properly.

Executing Maven:  -B -f /mnt/www/jenkins/jobs/Integration tests/workspace/pom.xml install

[INFO] Scanning for projects...
[INFO]                                                                         
[INFO] ------------------------------------------------------------------------
[INFO] Building functionalTests 1.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------

[INFO] 
[INFO] --- maven-resources-plugin:2.5:resources (default-resources) @ functionalTests ---

[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 1 resource

[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:compile (default-compile) @ functionalTests ---
[INFO] Nothing to compile - all classes are up to date
[INFO] 
[INFO] --- maven-resources-plugin:2.5:testResources (default-testResources) @ functionalTests ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 1 resource

[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:testCompile (default-testCompile) @ functionalTests ---
[INFO] Nothing to compile - all classes are up to date
[INFO] 
[INFO] --- maven-surefire-plugin:2.5:test (default-test) @ functionalTests ---
[INFO] Surefire report directory: /mnt/www/jenkins/jobs/Integration tests/workspace/target/surefire-reports

-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running TestSuite

org.openqa.selenium.firefox.NotConnectedException: Unable to connect to host 127.0.0.1 on port 7055 after 45000 ms. Firefox console output:
Error: cannot open display: :20
Error: cannot open display: :20

    at org.openqa.selenium.firefox.internal.NewProfileExtensionConnection.start(NewProfileExtensionConnection.java:106)
    at org.openqa.selenium.firefox.FirefoxDriver.startClient(FirefoxDriver.java:244)
    at org.openqa.selenium.remote.RemoteWebDriver.<init>(RemoteWebDriver.java:110)
    at org.openqa.selenium.firefox.FirefoxDriver.<init>(FirefoxDriver.java:188)
    at org.openqa.selenium.firefox.FirefoxDriver.<init>(FirefoxDriver.java:183)
    at org.openqa.selenium.firefox.FirefoxDriver.<init>(FirefoxDriver.java:179)
    at com.pragmaticqa.tests.AppTestData.setUp(AppTestData.java:37)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:616)
    at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:80)
    at org.testng.internal.MethodInvocationHelper.invokeDataProvider(MethodInvocationHelper.java:117)
    at org.testng.internal.Parameters.handleParameters(Parameters.java:426)
    at org.testng.internal.Invoker.handleParameters(Invoker.java:1383)
    at org.testng.internal.Invoker.createParameters(Invoker.java:1075)
    at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1180)
    at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:127)
    at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:111)
    at org.testng.TestRunner.privateRun(TestRunner.java:767)
    at org.testng.TestRunner.run(TestRunner.java:617)
    at org.testng.SuiteRunner.runTest(SuiteRunner.java:334)
    at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:329)
    at org.testng.SuiteRunner.privateRun(SuiteRunner.java:291)
    at org.testng.SuiteRunner.run(SuiteRunner.java:240)
    at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52)
    at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:86)
    at org.testng.TestNG.runSuitesSequentially(TestNG.java:1198)
    at org.testng.TestNG.runSuitesLocally(TestNG.java:1123)
    at org.testng.TestNG.run(TestNG.java:1031)
    at org.apache.maven.surefire.testng.TestNGExecutor.run(TestNGExecutor.java:62)
    at org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.execute(TestNGDirectoryTestSuite.java:141)
    at org.apache.maven.surefire.Surefire.run(Surefire.java:180)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:616)
    at org.apache.maven.surefire.booter.SurefireBooter.runSuitesInProcess(SurefireBooter.java:350)
    at org.apache.maven.surefire.booter.SurefireBooter.main(SurefireBooter.java:1021)

org.openqa.selenium.firefox.NotConnectedException: Unable to connect to host 127.0.0.1 on port 7055 after 45000 ms. Firefox console output:
Error: cannot open display: :20
Error: cannot open display: :20

    at org.openqa.selenium.firefox.internal.NewProfileExtensionConnection.start(NewProfileExtensionConnection.java:106)
    at org.openqa.selenium.firefox.FirefoxDriver.startClient(FirefoxDriver.java:244)
    at org.openqa.selenium.remote.RemoteWebDriver.<init>(RemoteWebDriver.java:110)
    at org.openqa.selenium.firefox.FirefoxDriver.<init>(FirefoxDriver.java:188)
    at org.openqa.selenium.firefox.FirefoxDriver.<init>(FirefoxDriver.java:183)
    at org.openqa.selenium.firefox.FirefoxDriver.<init>(FirefoxDriver.java:179)
    at com.pragmaticqa.tests.AppTestData.setUp(AppTestData.java:37)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:616)
    at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:80)
    at org.testng.internal.MethodInvocationHelper.invokeDataProvider(MethodInvocationHelper.java:117)
    at org.testng.internal.Parameters.handleParameters(Parameters.java:426)
    at org.testng.internal.Invoker.handleParameters(Invoker.java:1383)
    at org.testng.internal.Invoker.createParameters(Invoker.java:1075)
    at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1180)
    at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:127)
    at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:111)
    at org.testng.TestRunner.privateRun(TestRunner.java:767)
    at org.testng.TestRunner.run(TestRunner.java:617)
    at org.testng.SuiteRunner.runTest(SuiteRunner.java:334)
    at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:329)
    at org.testng.SuiteRunner.privateRun(SuiteRunner.java:291)
    at org.testng.SuiteRunner.run(SuiteRunner.java:240)
    at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52)
    at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:86)
    at org.testng.TestNG.runSuitesSequentially(TestNG.java:1198)
    at org.testng.TestNG.runSuitesLocally(TestNG.java:1123)
    at org.testng.TestNG.run(TestNG.java:1031)
    at org.apache.maven.surefire.testng.TestNGExecutor.run(TestNGExecutor.java:62)
    at org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.execute(TestNGDirectoryTestSuite.java:141)
    at org.apache.maven.surefire.Surefire.run(Surefire.java:180)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:616)
    at org.apache.maven.surefire.booter.SurefireBooter.runSuitesInProcess(SurefireBooter.java:350)
    at org.apache.maven.surefire.booter.SurefireBooter.main(SurefireBooter.java:1021)
Tests run: 2, Failures: 0, Errors: 0, Skipped: 2, Time elapsed: 93.916 sec

Results :

Tests run: 2, Failures: 0, Errors: 0, Skipped: 2

[JENKINS] Recording test results

[INFO] 
[INFO] --- maven-jar-plugin:2.3.2:jar (default-jar) @ functionalTests ---

[INFO] Building jar: /mnt/www/jenkins/jobs/Integration tests/workspace/target/functionalTests-1.0-SNAPSHOT.jar
[INFO] 
[INFO] --- selenium-maven-plugin:2.3:xvfb (xvfb) @ functionalTests ---

[INFO] Starting Xvfb...
[INFO] Using display: :20
[INFO] Using Xauthority file: /tmp/Xvfb450292168912752014.Xauthority
Deleting: /tmp/Xvfb450292168912752014.Xauthority
xauth:  file /tmp/Xvfb450292168912752014.Xauthority does not exist
Launching Xvfb
Waiting for Xvfb...
[INFO] Redirecting output to: /mnt/www/jenkins/jobs/Integration tests/workspace/target/selenium/xvfb.log

Xvfb started
[INFO] 
[INFO] --- selenium-maven-plugin:2.3:start-server (selenium) @ functionalTests ---
Launching Selenium Server
Waiting for Selenium Server...
[INFO] Including display properties from: /mnt/www/jenkins/jobs/Integration tests/workspace/target/selenium/display.properties
[INFO] User extensions: /mnt/www/jenkins/jobs/Integration tests/workspace/target/selenium/user-extensions.js

08:51:30,509 INFO  [org.openqa.selenium.server.SeleniumServer] Java: Sun Microsystems Inc. 20.0-b12
08:51:30,510 INFO  [org.openqa.selenium.server.SeleniumServer] OS: Linux 3.2.0-31-virtual amd64
08:51:30,521 INFO  [org.openqa.selenium.server.SeleniumServer] v2.21.0, with Core v2.21.0. Built from revision 16551
08:51:30,793 INFO  [org.openqa.selenium.server.SeleniumServer] RemoteWebDriver instances should connect to: http://127.0.0.1:4444/wd/hub
08:51:30,795 INFO  [org.openqa.jetty.http.HttpServer] Version Jetty/5.1.x
08:51:30,796 INFO  [org.openqa.jetty.util.Container] Started HttpContext[/selenium-server/driver,/selenium-server/driver]
08:51:30,797 INFO  [org.openqa.jetty.util.Container] Started HttpContext[/selenium-server,/selenium-server]
08:51:30,797 INFO  [org.openqa.jetty.util.Container] Started HttpContext[/,/]

08:51:30,951 INFO  [org.openqa.jetty.util.Container] Started org.openqa.jetty.jetty.servlet.ServletHandler@6a8c436b
08:51:30,951 INFO  [org.openqa.jetty.util.Container] Started HttpContext[/wd,/wd]
08:51:30,957 INFO  [org.openqa.jetty.http.SocketListener] Started SocketListener on 0.0.0.0:4444
08:51:30,957 INFO  [org.openqa.jetty.util.Container] Started org.openqa.jetty.jetty.Server@1bc74f37
08:51:31.693 INFO - Checking Resource aliases
Selenium Server started
[INFO] [INFO] Surefire report directory: /mnt/www/jenkins/jobs/Integration tests/workspace/target/surefire-reports

[INFO] --- maven-surefire-plugin:2.5:test (default) @ functionalTests ---

-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running TestSuite

Tests run: 2, Failures: 2, Errors: 0, Skipped: 0, Time elapsed: 319.362 sec <<< FAILURE!
[JENKINS] Recording test results
Results :

Failed tests: 
  aNewPlaceIsBornAndThenEdited(com.pragmaticqa.tests.AppTest)
  twoUsersSignUp(com.pragmaticqa.tests.AppTest2)

Tests run: 2, Failures: 2, Errors: 0, Skipped: 0


[ERROR] There are test failures.

Please refer to /mnt/www/jenkins/jobs/Integration tests/workspace/target/surefire-reports for the individual test results.
[INFO] 
[INFO] --- maven-install-plugin:2.3.1:install (default-install) @ functionalTests ---
[INFO] Installing /mnt/www/jenkins/jobs/Integration tests/workspace/target/functionalTests-1.0-SNAPSHOT.jar to /var/lib/jenkins/.m2/repository/com/pragmaticqa/tests/functionalTests/1.0-SNAPSHOT/functionalTests-1.0-SNAPSHOT.jar
[INFO] Installing /mnt/www/jenkins/jobs/Integration tests/workspace/pom.xml to /var/lib/jenkins/.m2/repository/com/pragmaticqa/tests/functionalTests/1.0-SNAPSHOT/functionalTests-1.0-SNAPSHOT.pom
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 7:11.329s
[INFO] Finished at: Fri Aug 02 08:56:53 UTC 2013

[INFO] Final Memory: 16M/247M
[INFO] ------------------------------------------------------------------------
[JENKINS] Archiving /mnt/www/jenkins/jobs/Integration tests/workspace/pom.xml to /mnt/www/jenkins/jobs/Integration tests/modules/com.pragmaticqa.tests$functionalTests/builds/2013-08-02_08-49-31/archive/com.pragmaticqa.tests/functionalTests/1.0-SNAPSHOT/functionalTests-1.0-SNAPSHOT.pom
[JENKINS] Archiving /mnt/www/jenkins/jobs/Integration tests/workspace/target/functionalTests-1.0-SNAPSHOT.jar to /mnt/www/jenkins/jobs/Integration tests/modules/com.pragmaticqa.tests$functionalTests/builds/2013-08-02_08-49-31/archive/com.pragmaticqa.tests/functionalTests/1.0-SNAPSHOT/functionalTests-1.0-SNAPSHOT.jar
Waiting for Jenkins to finish collecting data
channel stopped

Finished: SUCCESS
like image 272
Kaloyan Roussev Avatar asked Aug 02 '13 08:08

Kaloyan Roussev

1 Answers

Looks like at some stage the tests are being run and we have the chance to stop them from running again, by configuring skip to true

     <executions>
            <execution>
                <phase>integration-test</phase>
                <goals>
                    <goal>test</goal>
                </goals>
                <configuration>
                    <skip>false</skip>
                </configuration>
     </execution>
like image 172
Kaloyan Roussev Avatar answered Nov 15 '22 05:11

Kaloyan Roussev
Sign in to Comment

Related questions

Configuring Perforce scm into maven project to get latest changelist
How to document private elements in scaladoc with scala-maven-plugin?
Maven unable to find packages in jar
Maven (MVN) return value on failure?
Incremental build in Jenkins
How to find unused properties in a pom
MyFaces tomcat7:run goal cannot start ProtocolHandler "http-bio-8080"
org.springframework.test cannot be resolved also with maven dependency present
Maven assembly plugin: Failed to retrieve numeric file attributes using: '/bin/sh -c ls -1nlaR
How do I get depend on all artifacts from a group of a version from maven?
Where should I put test support code for a Java library
jaxws-maven-plugin resolving WSDL location relative to class location, why?
Is it possible to use the maven-release-plugin with a specific revision?
maven deploy: how to restrict to deploy only the artifacts? (multi-module environment)
Maven Resources Plugin: how to copy resources to target folder and keep time stamps?
How to create Restful service and deploy it OSGi container?
PowerMockRunner causes ExceptionInInitializerError when running trivial test
How can I add meta data to a maven pom
Eclipse/Maven and "Resolve dependencies from workspace projects" can't mix jars and source?
Duplicate classes R & BuildConfig after "Reimport All Maven Projects"

Donate For Us

If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!