Selenium Critique

Question

I just wanted some opinions from people that have run Selenium (http://selenium.openqa.org) I have had a lot of experience with WaTiN and even wrote a recording suite for it. I had it producing some well-structured code but being only maintained by me it seems my company all but abandoned it.

If you have run selenium have you had a lot of success?

I will be using .NET 3.5, does Selenium work well with it?

Is the code produced clean or simply a list of all the interaction? (http://blogs.conchango.com/richardgriffin/archive/2006/11/14/Testing-Design-Pattern-for-using-WATiR_2F00_N.aspx)

How well does the distributed testing suite fair?

Any other gripes or compliments on the system would be greatly appreciated!

Answer 1

If you are using Selenium IDE to generate code, then you just get a list of every action that selenium will execute. To me, Selenium IDE is a good way to start or do a fast "try and see" test. But, when you think about maintainability and more readable code, you must write your own code.

A good way to achieve good selenium code is to use the Page Object Pattern in a way that the code represents your navigation flow. Here is a good example that I see in Coding Dojo Floripa (from Brazil):

public class GoogleTest {

    private Selenium selenium;

    @Before
    public void setUp() throws Exception {
            selenium = new DefaultSelenium("localhost", 4444, "*firefox",
                            "http://www.google.com/webhp?hl=en");
            selenium.start();
    }

    @Test
    public void codingDojoShouldBeInFirstPageOfResults() {
            GoogleHomePage home = new GoogleHomePage(selenium);
            GoogleSearchResults searchResults = home.searchFor("coding dojo");
            String firstEntry = searchResults.getResult(0);
            assertEquals("Coding Dojo Wiki: FrontPage", firstEntry);
    }

    @After
    public void tearDown() throws Exception {
            selenium.stop();
    }

}


public class GoogleHomePage {

    private final Selenium selenium;

    public GoogleHomePage(Selenium selenium) {
            this.selenium = selenium;
            this.selenium.open("http://www.google.com/webhp?hl=en");
            if (!"Google".equals(selenium.getTitle())) {
                    throw new IllegalStateException("Not the Google Home Page");
            }
    }

    public GoogleSearchResults searchFor(String string) {
            selenium.type("q", string);
            selenium.click("btnG");
            selenium.waitForPageToLoad("5000");
            return new GoogleSearchResults(string, selenium);
    }
}

public class GoogleSearchResults {

    private final Selenium selenium;

    public GoogleSearchResults(String string, Selenium selenium) {
            this.selenium = selenium;
            if (!(string + " - Google Search").equals(selenium.getTitle())) {
                    throw new IllegalStateException(
                                    "This is not the Google Results Page");
            }
    }

    public String getResult(int i) {
            String nameXPath = "xpath=id('res')/div[1]/div[" + (i + 1) + "]/h2/a";
            return selenium.getText(nameXPath);
    }
}

Hope that Helps

Answer 2

I'm using Selenium Remote Control in order to test ASP.Net apps (which is what I'm assuming you'll be targetting as well), and it works great.

If you've never used Selenium, watch some of the screencasts for using Selenium IDE. This will give you a good idea of how 'Selenium' works. The IDE is a firefox plugin that basically lets you develop quick record-and-play tests as you go. For larger test suites or for writing really maintainable tests though, I'd recommend Selenium Remote Control. (The IDE is terrific if you're just getting a start though.)

Selenium Remote Control lets you use your favourite language and unit testing framework to drive a web browser in order to execute your tests. If you're most comfortable with C#/NUnit, you can write your tests that way and use all the NUnit goodies that you like. (For example, the Test-Driven.net plugin). Also, since your tests are written in a high level language, you're able to do things like inherit from a particular test class which you can use to make your actual test method code much cleaner. (Or at least thats the way I write my tests. It lets me test complex scenarios which keeping my test method line-count at a reasonable number.)

You mention distributed testing. Unfortunately I haven't found a way to use the Selenium Grid project with NUnit. Selenium Grid allows you to execute your test suite over a number different machines and browser instances. So rather than running through say 200 test methods one after another (ie, serially), you could spread the load out over say four Grid instances (ie, running in four different browsers instances at a time) on a single machine or multiple machines depending on how distributed you want to get.

If you write your tests in Java or PHP though, you might have better luck. I'm expecting this to be available via NUnit with the release of NUnit2.5 which will include pNUnit for parallel testing.

If you have any further questions about selenium, just clarify your original question and I'll be happy to try and help you out. (Selenium is just one of those tools that I use everyday so I enjoy helping to get new people started with it..)