Has anybody used SIKULI for testing their GUI-based apps?

Question

SIKULI seems to have an enormous amount of potential. Has anybody tried to use this as a tool for testing? Or would it be better suited for automating actions for users?

Answer 1

Quoting Unit Testing for GUI (in the project Documentation):

Sikuli is designed to support unit testing for GUI by integrating with junit. The unit testing panel can be opened by clicking on View/Unit Test or by shortcut Cmd-U on Mac (or Ctrl-U on Windows/Linux).

So, while my understanding is that SIKULI is initially aimed at GUI automation, it can definitely be used for GUI testing (which is closely related if you consider that GUI testing = GUI automation + verification framework). Have a look at Unit testing for GUI (JEdit) for a full example (and see the assertXXX on images).

And indeed, I see a big potential in SIKULI for testing as it seems to make writing tests very easy, even without a single line of the real application written (just using some initial mockups for example). SIKULI could become a great companion for various flavor of testing (BDD, acceptance testing, etc).

It' really an amazing piece of software, very impressive.

Answer 2

I am using Sikuli extensively for UI test automation. I am "late" to the Sikuli party, having discovered it in January 2011. I am actually glad I discovered it late, because while it was promising earlier, I don't think until Sikuli x1.0-rc1 (which happened in December) was released that it was ready for primetime.

Previously, I have used TestQuest and EggPlant for UI test automation. In my opinion, Sikuli beats both of them hands down. I truly believe it has the potential to dramatically change how folks perform UI test automation for the better and will be evangelizing it to folks around me.

Using Sikuli correctly is means that you are not following a "record and playback" model. Rather, you have to approach developing test automation with Sikuli - as you need to with all tools - as a software development task.

We are currently in the process of porting a UI automation DSL (Domain Specific Language) we built for EggPlant to Sikuli. One of the key features we will be leveraging in our DSL is Sikuli's text recognition capabilities. This will allow us to run the same script across various localized versions of our product.

Because Sikuli builds on OpenCV (for image recognition) and tesseract-ocr (for text recognition), it has an incredibly amount of power and flexibility.