UI Testing Tool? [closed]

Question

Looking for a tool to UI testing of a Windows (.NET WinForms) application. The idea is that the analysts will record the tests via some UI and it will be able to be played back over and over again.

Commercial tools are fine but bonus points for free ones.

Answer 1

Some others:

• NUnitForms
• Quail

I believe they're both free, and Quail looks really nice!

Answer 2

(I was going to just comment on an answer above, but ran out of room.) We bought Ranorex, and I wouldn't do it again. Their licensing for 1.5 (what we're using) was unclear. Their written license was per user, their marketing said per machine. Still, enforcement was key-based, and not a big deal when a machine got wiped or a replacement tester came in.

For the 2.0 product, their licensing is now tied to the machine. Reimage a machine and you get to relicense it. I'm just not going to support that kind of hassle with my company's thousands of dollars, and we didn't upgrade.

For what they want for the product, a lot of the functionality could be easily written using the UI Automation Framework. Ranorex is a decent product, but I question its value for the money. We're moving a lot of our new tests to just coding to the UI Automation Framework, as we often end up modifying Ranorex-generated code anyway.

Answer 3

TestComplete.

You can definitely use capture-replay to capture and run the test script. But I would suggest that you must at least manually edit your scripts to make them

1. more readable
2. easier to maintain.

The good thing about TestComplete is that it is able to look into your form's properties, capture those properties so that you can refer to those properties by their name, not by just screen coordinates.

Answer 4

Like Tom E stated, do take caution while considering going down the record/playback path for test automation.

See Uncle Bob's article on Ruining your Test Automation Strategy.

The main problem is that the record/playback tools couple the tests to the GUI which makes them very fragile.

Uncle Bob's article does point out that some testing needs to occur on the GUI...but that he recommends stubbing out the business rule code.

Sorry I can't provide you with a specific UI test automation tool...but hopefully this caveat will help you make the best decision on how to employ the tool that you eventually use.

Answer 5

So far I have found:

• Ranorex which looks really good.
• Test Automation FX, also looks good and seems very well priced.
• Microsoft's UI Automation Framework, which does not have the recorder but if I had to I could code one using this.
• white which looks similar to the UI Automation Framework, but has an alpha quality recorder.

Comments please if you have used any of these.

Answer 6

At my company, we decided to go with http://www.sikuli.org/. We felt it was the perfect mix of cost (free), ease-of-use, functionality and extend-ability.

Sikuli is Python based (Jython, really) which is great and its open-source. There's a tiny bit of coding required, but it can be as easy as just calling functions. Their IDE makes it really easy to get started. It is not a record and playback tool. It functions based on computer vision algorithms - you give it screenshots of what to look for and it finds it on the screen and then performs the requested action (click, type, etc) on what it found. This is true independent testing since Sikuli knows nothing about the software its testing. It does not know about underlying APIs - it just does what a real human would do.

We have integrated Sikuli with Robot Framework, http://robotframework.org/, and have created lots of custom python code to build a robust testing platform. This may not be as easy as licensing a tool from a vendor but the time and talent investment in these two open source tools has been well worth it.

Answer 7

There are a bunch of similar questions on SO:

• automate-interaction-with-a-gui-interface (Edit: no longer available)
• automated-testing-of-windows-forms (Edit: no longer available)
• automated-testing-of-gui

In my experience, there are a lot of good open source tools for the web, but not so much selection for open source thick client test automation tools. If you want good support with robust functionality, especially recording, you will need to look at the commercial tools (QTP, RFT, TestPartner, etc...)

Answer 8

You should have a look at http://opensourcetesting.org/functional.php A lot of tools are listed here and you should find something that meet your needs.

Answer 9

Visual Studio Team Test 2010 is coming with a tool for recording and playing back UI tests. You'll find some pointers at:

Introduction to Record and Playback Engine in VSTT 2010