My JUnit tests works when run in Eclipse, but sometimes randomly fails via Ant

Question

The core of my question is that I am concerned that my Ant build file is missing something that will allow a test to finish and clean itself up. The details are below.

I have a suite of tests that always passes when I run it through Eclipse, but sometimes passes or fails when I run it using my Ant build. The tests use openCL via JOCL so I have limited memory on the GPU and it has to be managed correctly. I get this in my output sometimes when I run my Ant build,

[junit]     Caused an ERROR
[junit] CL_MEM_OBJECT_ALLOCATION_FAILURE
[junit] org.jocl.CLException: CL_MEM_OBJECT_ALLOCATION_FAILURE

The problem can not be in the test itself. I think it is that my most memory hungry test is invoked at the end of the suite. When this last test is invoked, somehow the GPU is left in a bad state from my previous tests. This doesn't happen when I run the tests through Eclipse. It has never failed in my Ant build when I make the memory hungry test the first test in the suite. Is this a familiar case? Why does running the tests through Eclipse always work? Is there anything I can try?

Here is the testing target in my Ant build:

<target name="test" if="testing.enabled">
    <mkdir dir="${test.bin.dir}" />
    <javac srcdir="test" destdir="${test.bin.dir}" debug="true" classpathref="testclasspath" source="1.6"/>
    <junit haltonerror="true" haltonfailure="true">
        <classpath refid="testclasspath"/>
        <formatter type="plain" usefile="false" />
        <batchtest>
            <fileset dir="test">
                <include name="*Test.java"/>
            </fileset>
        </batchtest>
    </junit>
</target>

Answer 1

If you are really sure no left-over cleanup is missed in your code, you can create a JUnit test suite and run that from both eclipse and ant. By creating a test suite you make yourself independent of the sequence of tests that eclipse (order within the project?) and ant (order within the filesystem?) use and determine the order yourself in both cases.

If you are not really really sure your code is issue-free you could make a test suite which starts of by calling Collections.shuffle() on the list of test classes to introduce unknown test execution order in both eclipse see if your tests still never fail.