Automating unit tests (junit) for Eclipse Plugin development

Question

I am developing Eclipse plugins, and I need to be able to automate the building and execution of the test suite for each plugin. (Using Junit)

Test are working within Eclipse, and I can break the plugins into the actual plugin and a fragment plugin for unit testing as described here, here and in a couple places here.

However, each of the approaches above results in the same issue: The java ant task/commandline command that issues the build or should trigger the test, generates no observable side effects, and returns the value "13". I've tried everything I can find, and I've learned a fair bit about how Eclipse starts up (eg: since v3.3 you can no longer use startup.jar -- it doesn't exist -- but you should use org.eclipse.equinox.launcher). Unfortunately, while that is apparently necessary information, it is far from sufficient.

I am working with Eclipse 3.4, Junit 4.3.1 (the org.junit4 bundle, but I would much rather use JUnit 4.4. See here.)

So, my question is: How exactly do you automate the build and testing of Eclipse plugins?

Edit: To clarify, I want to use something like ant + cruise control, but I can't even get the unit tests to run at all outside of Eclipse. I say "something like" because there are other technologies that accomplish the same thing, and I am not so picky as to discard a solution that works just because it's using say, Maven or Buckminster, if those technologies make this substantially easier.

Edit2: The 'Java Result 13' mentioned above seems to be caused by the inability to find the coretestrunner. From the log:

java.lang.RuntimeException: Application "org.eclipse.test.coretestapplication" could not be found in the registry. The applications available are: org.eclipse.equinox.app.error, com.rcpquickstart.helloworld.application.
    at org.eclipse.equinox.internal.app.EclipseAppContainer.startDefaultApp(EclipseAppContainer.java:242)
    at org.eclipse.equinox.internal.app.MainApplicationLauncher.run(MainApplicationLauncher.java:29)
    at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:110)
    at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:79)
    at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:382)
    at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:179)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java:549)
    at org.eclipse.equinox.launcher.Main.basicRun(Main.java:504)
    at org.eclipse.equinox.launcher.Main.run(Main.java:1236)
    at org.eclipse.equinox.launcher.Main.main(Main.java:1212)
    at org.eclipse.core.launcher.Main.main(Main.java:30)

!ENTRY org.eclipse.osgi 2 0 2008-11-04 21:02:10.514
!MESSAGE The following is a complete list of bundles which are not resolved, see the prior log entry for the root cause if it exists:
!SUBENTRY 1 org.eclipse.osgi 2 0 2008-11-04 21:02:10.515
!MESSAGE Bundle update@plugins/org.eclipse.test_3.2.0/ [34] was not resolved.
!SUBENTRY 2 org.eclipse.test 2 0 2008-11-04 21:02:10.516
!MESSAGE Missing required bundle org.apache.ant_0.0.0.
!SUBENTRY 2 org.eclipse.test 2 0 2008-11-04 21:02:10.516
!MESSAGE Missing required bundle org.eclipse.ui.ide.application_0.0.0.
!SUBENTRY 1 org.eclipse.osgi 2 0 2008-11-04 21:02:10.518
!MESSAGE Bundle update@plugins/org.eclipse.ant.optional.junit_3.2.100.jar [60] was not resolved.
!SUBENTRY 2 org.eclipse.ant.optional.junit 2 0 2008-11-04 21:02:10.519
!MESSAGE Missing host org.apache.ant_[1.6.5,2.0.0).
!SUBENTRY 2 org.eclipse.ant.optional.junit 2 0 2008-11-04 21:02:10.519
!MESSAGE Missing required bundle org.eclipse.core.runtime.compatibility_0.0.0.

Answer 1

I have just got JUnit testing working as part of the headless build for our RCP application.

I found this article - Automating Eclipse PDE Unit Tests using Ant incredibly helpful. It provides code and approach to get you started. However, a number of things that I discovered:

About the article's code

• there was only one bundle under tests (we have separated out our build process from the code, using Buckminster)
• there was only one test class.
• these were both effectively hardcoded into the build script

About Eclipse PDE

• the uitestapplication requires another testApplication. Using coretestapplication does not.
• as these applications are both in bundles that have dependencies on SWT. This is a deal killer in most circumstances, though not if your build machine is a Windows box. I would love to see these split into non-UI bundles.

I found that the code provided was a good starting point, but had a number of the above assumptions implicit in their implementation.

Having discovered these assumptions, doing the work was relatively straight forward.

Our new and shiny setup

• buckminster builds the bundles.
• target copies the bundles from the target platform, the org.eclipse.pde.runtime and org.eclipse.jdt.junit into a "tester-eclipse-install". This should take care of your Java Result 13 problem.
• find the test fragments from looking at the workspace
• find the fragment host from looking at the manifest
• find the test classes from looking at the project in the workspace.
• register a PDETestListener modified to handle multiple test classes
• invoke the tester-eclipse-install with the multiple test classes.

I also read Build and Test Automation for plug-ins and features but we are not using PDE-Build directly.