Why doesn't running test-with-groovy from ant command line in a netbeans project run the tests?

Question

I've got a netbeans project, and I've got groovy in it as well for spock testing. When I right-click on the project and say test, it runs a task called

test-with-groovy

but when I run ant test-with-groovy the tests are compiled but not run. I feel like something must be added by the netbeans ide but I have no idea what and half a day of searching has netted no results.

Can anyone help me?

Here is how you can get the results I am getting:

I created a simple java project with a simple main in netbeans 8.0.2

package simpleantjava;

public class SimpleAntJava {

    public static void main(String[] args) {
        // TODO code application logic here
        System.out.println("Main Ran");
    }

}

then I created two test files, one a junit java file:

package simpleantjava;

import org.junit.After;
import org.junit.AfterClass;
import org.junit.Before;
import org.junit.BeforeClass;
import org.junit.Test;
import static org.junit.Assert.*;

/**
 *
 * @author vextorspace
 */
public class SimpleAntJavaTest {

    public SimpleAntJavaTest() {
    }

    @BeforeClass
    public static void setUpClass() {
    }

    @AfterClass
    public static void tearDownClass() {
    }

    @Before
    public void setUp() {
    }

    @After
    public void tearDown() {
    }

    /**
     * Test of main method, of class SimpleAntJava.
     */
    @Test
    public void testMain() {
        System.out.println("main");
        String[] args = null;
        SimpleAntJava.main(args);
        // TODO review the generated test code and remove the default call to fail.
        fail("Main JUnit Test is a prototype.");
    }

}

and one a groovy based spock test:

package simpleantjava

import org.junit.Test
import spock.lang.Specification

/**
 *
 * @author vextorspace
 */
class NewGroovyTest extends Specification{
    @Test
    def "Our groovy test should run"() {
        expect:
          true
    }

    @Test
    def "Failing tests should fail"() {
        expect:
          false
    }
}

if I run the tests from netbeans, the output says it is running:

ant -f /home/vextorspace/NetBeansProjects/SimpleAntJava -Dtest.binarytestincludes=**/*Test.class -Dtest.binaryincludes= -Dtest.binaryexcludes=**/*$* test-with-groovy

but if I run that in the ant command line it doesn't run any tests (though it gives no errors either) It does compile both test files into class files in build/test/classes.

If I run ant clean test it builds both test files but does not run the groovy test, only the java test.

the build-impl.xml has the definitions for test (I won't include the whole file, but it is the standard one created by netbeans: here are the relevant sections I believe:

<target if="${nb.junit.single}" name="-init-macrodef-junit-single" unless="${nb.junit.batch}">
    <macrodef name="junit" uri="http://www.netbeans.org/ns/j2se-project/3">
        <attribute default="${includes}" name="includes"/>
        <attribute default="${excludes}" name="excludes"/>
        <attribute default="**" name="testincludes"/>
        <attribute default="" name="testmethods"/>
        <element name="customize" optional="true"/>
        <sequential>
            <property name="junit.forkmode" value="perTest"/>
            <junit dir="${work.dir}" errorproperty="tests.failed" failureproperty="tests.failed" fork="true" forkmode="${junit.forkmode}" showoutput="true" tempdir="${build.dir}">
                <test methods="@{testmethods}" name="@{testincludes}" todir="${build.test.results.dir}"/>
                <syspropertyset>
                    <propertyref prefix="test-sys-prop."/>
                    <mapper from="test-sys-prop.*" to="*" type="glob"/>
                </syspropertyset>
                <formatter type="brief" usefile="false"/>
                <formatter type="xml"/>
                <jvmarg value="-ea"/>
                <customize/>
            </junit>
        </sequential>
    </macrodef>
</target>
<target depends="-init-test-properties" if="${nb.junit.batch}" name="-init-macrodef-junit-batch" unless="${nb.junit.single}">
    <macrodef name="junit" uri="http://www.netbeans.org/ns/j2se-project/3">
        <attribute default="${includes}" name="includes"/>
        <attribute default="${excludes}" name="excludes"/>
        <attribute default="**" name="testincludes"/>
        <attribute default="" name="testmethods"/>
        <element name="customize" optional="true"/>
        <sequential>
            <property name="junit.forkmode" value="perTest"/>
            <junit dir="${work.dir}" errorproperty="tests.failed" failureproperty="tests.failed" fork="true" forkmode="${junit.forkmode}" showoutput="true" tempdir="${build.dir}">
                <batchtest todir="${build.test.results.dir}">
                    <fileset dir="${test.src.dir}" excludes="@{excludes},${excludes}" includes="@{includes}">
                        <filename name="@{testincludes}"/>
                    </fileset>
                    <fileset dir="${build.test.classes.dir}" excludes="@{excludes},${excludes},${test.binaryexcludes}" includes="${test.binaryincludes}">
                        <filename name="${test.binarytestincludes}"/>
                    </fileset>
                </batchtest>
                <syspropertyset>
                    <propertyref prefix="test-sys-prop."/>
                    <mapper from="test-sys-prop.*" to="*" type="glob"/>
                </syspropertyset>
                <formatter type="brief" usefile="false"/>
                <formatter type="xml"/>
                <jvmarg value="-ea"/>
                <customize/>
            </junit>
        </sequential>
    </macrodef>
</target>
<target depends="-init-macrodef-junit-init,-init-macrodef-junit-single, -init-macrodef-junit-batch" if="${junit.available}" name="-init-macrodef-junit"/>

...

<!--                                                                                                                                                                                                            
            =======================                                                                                                                                                                             
            TEST EXECUTION SECTION                                                                                                                                                                              
            =======================                                                                                                                                                                             
        -->
<target depends="init" if="have.tests" name="-pre-test-run">
<mkdir dir="${build.test.results.dir}"/>
</target>
<target depends="init,compile-test,-pre-test-run" if="have.tests" name="-do-test-run">
<j2seproject3:test includes="${includes}" testincludes="**/*Test.java"/>
</target>
<target depends="init,compile-test,-pre-test-run,-do-test-run" if="have.tests" name="-post-test-run">
    <fail if="tests.failed" unless="ignore.failing.tests">Some tests failed; see details above.</fail>
</target>
<target depends="init" if="have.tests" name="test-report"/>
<target depends="init" if="netbeans.home+have.tests" name="-test-browse"/>
<target depends="init,compile-test,-pre-test-run,-do-test-run,test-report,-post-test-run,-test-browse" description="Run unit tests." name="test"/>
<target depends="init" if="have.tests" name="-pre-test-run-single">
<mkdir dir="${build.test.results.dir}"/>
</target>

and the test-with-groovy ant target is in the included groovy-build.xml:

<target depends="init,compile-test,-pre-test-run" if="have.tests" name="-do-test-run-with-groovy">
    <j2seproject3:test testincludes=""/>
</target>
<target depends="init,compile-test,-pre-test-run,-do-test-run-with-groovy" if="have.tests" name="-post-test-run-with-groovy">
    <fail if="tests.failed" unless="ignore.failing.tests">Some tests failed; see details above.</fail>
</target>
<target depends="init,compile-test,-pre-test-run,-do-test-run-with-groovy,test-report,-post-test-run-with-groovy,-test-browse" description="Run unit tests." name="test-with-groovy"/>
<target depends="init,compile-test-single,-pre-test-run-single" if="have.tests" name="-do-test-run-single-groovy">
    <fail unless="test.binarytestincludes">Must select some files in the IDE or set test.includes</fail>
    <j2seproject3:test testincludes=""/>
</target>

I must confess I don't know what the macrodefs are really doing. But I've tried changing **/*Test.java to **/*Test.groovy, **/*Test.class and **/Test. to no avail.

Does anyone know how to make this work? I don't want to throw out the whole build process because I'm actually working on a legacy project that is 6 years old with the jogl plugin and a lot of custom stuff in the ant file so I'd like to figure out how to make this work.

Thank You!

Answer 1

NewGroovyTest is completely independent test using Spock and no relationship to other classes.

In order to include classes that are defined in test/**/*Test.groovy files I took the compiled classes (as all Groovy compiles into Java) and am feeding them into JUnit. As said earlier the default behavior is to only use classes that come from *Test.java files.

This link will help you: Groovy Unit Tests

Specifically, adding the following:

<target name="-post-test-run-with-groovy">
<junit dir="${work.dir}" errorproperty="tests.failed"
       failureproperty="tests.failed" fork="true"
       showoutput="true">
  <batchtest todir="${build.test.results.dir}">
    <fileset dir="${build.test.classes.dir}">
      <include name="**/*Test.class"/>
      <exclude name="**/*$*"/>
    </fileset>
  </batchtest>
  <classpath>
    <path path="${run.test.classpath}"/>
  </classpath>
  <syspropertyset>
  <propertyref prefix="test-sys-prop."/>
  <mapper from="test-sys-prop.*" to="*" type="glob"/>
  </syspropertyset>
  <formatter type="brief" usefile="false"/>
  <formatter type="xml"/>
  <jvmarg line="${run.jvmargs}"/>
</junit>

<mkdir dir="${build.test.results.dir}/../report"/>
<mkdir dir="${build.test.results.dir}/../report/html"/>

<junitreport todir="${build.test.results.dir}/../report">
  <fileset dir="${build.test.results.dir}">
    <include name="TEST-*.xml"/>
  </fileset>
  <report format="frames" todir="${build.test.results.dir}/../report/html"/>
</junitreport>

to the build.xml file allows the tests to be run with the command

ant -f ~/NetBeansProjects/jdesigner -Dtest.binarytestincludes=**/*Test.class -Dtest.binaryincludes= -Dtest.binaryexcludes=**/* test-with-groovy