Run TestNG/JUnit Integration Tests on Remote Server from within IDE

Question

Inside my IDE (Eclipse or NetBeans, doesn't matter), i have some TestNG testclasses (But my question also refers to remote JUnit tests), which are Integration tests. These tests need an integration server to run, they cannot be run on a local machine. They need the complete environment of the integration server - not only JavaEE container related stuff (=> no Arquillian nor JEEUnit).

Now I want to be able to run these tests from within my IDE (Eclipse) - preferrably with the TestNG Plugin - but when I launch them they should actually be run on the remote integration server.

Is it possible to launch integration tests on a remote server from within my IDE? I like the idea of having some kind of Agent on the remote Server which waits for test-requests and executes them. But as I said, it would be nice if this runs from inside the TestNG Plugin.

Do I need some kind of workaround, for example Ant scripts (hopefully not) or some Maven magic? What are the best practices?

I know I could also create Webservices for my application, then I can call them from local unit tests. But I'd like to know if there are also possibilities without Webservices.

Answer 1

How you will accomplish this depends a lot on what integration server and IDE you're using, and what you're application is. I'll assume you're using Eclipse; and I'll assume you're using Jenkins, since it will IMO be the easiest to get it to do what you want.

Most of this will work out of the box. However, there is a little bit that will require a some of additional work.

You'll want to do the following:

1. Set up a job in Jenkins named something like "Integrated Testing" (the name doesn't matter). Configure it to run the tests you want to be able to run from your IDE, and to allow remote machines to trigger builds.
2. Make this job a parameterized job, and add a file parameter. Lets suppose this is called 'TestingProgram'. This will be the program that the job will test.
3. Make the Integrated Testing job use the TestingProgram file to run the tests. If you're program requires more than one file, than assume this will be a zip file containing all of the necessary files.
4. Configure your Eclipse project to produce these same file(s) that the Integration Testing job expects (I assume this is already happening, since it is presumably building some sort of bin version of your program)

Now comes the slightly more complicated part of hooking up Eclipse to Jenkins. Unfortunately, I don't think there are any preexisting tools that will do exactly what you want. The good news is that it should be very simple to achieve with a custom script. All the script has to do is:

1. If your program has multiple files required, and eclipse does not already do so, it will need to zip these files up
2. It will need to take the file and read it into some variable, and then base64 encode it. There are lots of libraries that will take care of most of this for you, eg this one. Lets assume this file is read into a variable called $programFile
3. It will need to send a HTTP request to http://<your-jenkins-server>:8080/<integrated-testing-job-name>/buildwithparameters?TestingProgram=$programFile

You can read up more on triggering Jenkins remote builds with parameters in the jenkins docs.

The script that accomplishes these steps can be pretty much anything. Given that you want to ultimately incorporate it into your IDE, it seems like the most logical choices would be an ant script or an Eclipse plugin. Neither would be too complicated - the ant script would just do those steps, and you could import the ant script into an Eclipse project specifically for doing testing - and a plugin could just add a menu item which, when run within a project, would execute the above steps for that project.

NOTE: There are actually several different ways to trigger a build with parameters using Jenkins. For example, you can do a POST request, use json to pass the parameter as described in the jenkins docs I linked to, use the Jenkins CLI, etc. Not all of them work with file parameters, but you would use them all in a very similar way - as a step in your custom script, you would execute a remote build on Jenkins, and pass the file you want to use to test with. For example, the explanation I gave assumes that the testing file is very small; if that's not the case, you might want to do a POST request instead. If you run into problems using one method, it should be pretty easy to switch it to use a different method that works better.

Answer 2

Unfortunately I do not have a ready solution but I think I can give you some tips since I spent some time thinking about this.

I do not know about TestNG but JUnit has ability to plugin your own test executor. I am sure that TestNG has appropriate functionality. So, find it and be familiar with it. Since you can control how your test is being invoked you can even do something else instead of invoking the test case's methods. For example call some remote API that will make the test to run remotely.

Obviously it can be web service that makes remote Agent (according to your suggestion) to run test on remote machine. It is fine, but I like agent-less or semi agent-less solutions more. What do I mean? If for example your remote machine is Unix you can perform SSH or Telnet connection and run command line. In this case you can create mvn or ant script, copy to remote machine using ssh and then run it. The script will run your tests. So it is almost agent less. You just require java installation and SSH support.

If remote machine is windows you can use either Telnet or WMI. So, if security is not an issue but you need cross platform support use Telnet.

Please do not hesitate to contact me if you need more assistance concerning SSH/Telnet.

Answer 3

I haven't done it myself (my tests are local), but here are some stuff that could work:

• Cargo can be used to run a server and deploy modules into it
• Running the tests can be done using Cactus (quite old) or JUnitEE. In bothe cases, you implement the tests using JUnit and run them remotely. Another option is to write the tests to interact with a dedicated API on your server which runs the tests and reports back.

Answer 4

In my solution, I will lunch a local JNDI server and add a remote object (an object implement the Remote interface), then you could use the remote object to sync your local IDE and remote server or other place.