Automate Selenium tests on TeamCity Continuous Integration server

Question

I have a Visual Studio solution which includes a Test project having Selenium Tests.

(I already have a compilation build triggered by version control checkin (Mercurial))

But I want to have a separate (nightly) Build Configuration which runs the Selenium tests, ideally under MSTest.

I assume I need Selenium Server for this? If so, what's the best way to fire it up before running the tests? Should I do this from the MSBuild script or use a Build Step from Team City itself? Do I need to fire up Cassini\WebDev.WebServer first of all so the following can run:

selenium = new DefaultSelenium("localhost", 4444, "*firefox", "http://localhost:49192/");

?

I have a Build Server with TeamCity 6.5.1 installed. I have a VS2010 installed.

Surely someone has done this! Desperate for some help here guys. If any one could offer any examples, that would be appreciated.

Answer 1

I am just about to set up Selenium tests in our TeamCity Server and am still Googling. These are the pages that I found interesting (apart from your StackOverflow question):

Regression testing for any web application with TeamCity, Selenium, and JUnit

Selenium browser UnitTesting from TeamCity

So there is one solution that converts the html files recorded with Selenium IDE to Java to be ran with JUnit in TeamCity and the other approach is creating your test scripts in C#.

And this topic, "Running Selenium Tests through Teamcity, can it be done?" describes a problem while running the Build Agent as a service... I hope that I can continue running the agent as a service.

Answer 2

We are using Selenium for nightly tests of our companys external web site. For this we use Selenium RC and dynamically created test suites.

Our process, which seems more complicated than Ross' is like this:

1. Install Selenium RC on the TeamCity server
2. Use the Firefox IDE plugin to create tests
3. Create a web page that generates a test suite html file with links to all tests in a certain web accessible folder (e.g. http://www.mysite.com/selenium/generateTests.aspx)
4. Create a Powershell build step in TeamCity, running a script that downloads the suite and corresponding tests to the build server
5. Add a "XML report processing" build feature to scan for test results. This makes TeamCity able to tell you about the test results.

In the powershell script:

1. Execute the Selenium RC runtime with inputs that targets the downloaded test suite and which url that is to be the base url. [*1]
2. Use XSLT to transform the output to NUnit format [*2]

[*1] java -jar C:\Selenium\selenium-remote-control-1.0.3\selenium-server-1.0.3\selenium-server.jar -log C:\Selenium\www.mysite.com\selenium-log.log -userExtensions 'user-extensions.js' -firefoxProfileTemplate 'Selenium FireFox Profile' -htmlSuite *firefox http://www.mysite.com C:\Selenium\www.mysite.com\generated\GeneratedSuite.htm C:\Selenium\www.mysite.com\TestResults.html

[*2] nxslt3 $seleniumXmlTestReportPath nunit.xslt -o $nunitReportPath baseUrl=$testBaseUrl

Answer 3

We use TeamCity to build and test our C#-based web application. We run the Selenium Server all the time, using the Java service launcher to start it. The tests connect to localhost, just like you show in your question.

We use TeamCity's "NAnt Runner" to start the test jobs, and use NAnt's <nunit2> task to run the tests under control of NUnit. Because we do that, NUnit finds and runs any public method annotated with a [Test] attribute - it's a very simple, very powerful tool.

This setup works very well for us.