BDD good for UI automation?

Question

We are about to make a decision of choosing the best approach for UI automation framework.

We have 2 options:

1. TestNG with webdriver for UI automation
2. Inhouse tool built with BDD.

We are in a critical situation to decide which technology is better for our project.

Our project is huge application with 8 modules and lot of integrations with 3rd party tools.

1. Is BDD good for huge projects?
2. Does it introduce maintenance effort when the number of testcases grow ?
3. Will there be duplicacy of given when then's when the number of testcases grow ?

Thanks in advance for all the thoughts.

Answer 1

BDD is about talking through examples of how a piece of code, an application, or a system should behave, then capturing those examples, often using a BDD tool and / or automation framework. Plenty of people get value just from capturing the examples on a wiki. There are four things I like to see in place before people move to the tools, particularly if you're about to do this at an application level:

• An eye on the big picture
• The ability to question and explore
• The ability to spot and embrace uncertainty
• Great relationships between people.

If you don't have these, treat automation with a great deal of care. If you're not capturing genuine conversations, I recommend steering clear of the English-language frameworks like Cucumber and JBehave. The additional overhead of wiring up the English G/W/T, together with the lack of refactoring tools for English, means that it's only worth using these if your business are involved in the reading or writing of scenarios.

I like to use a little DSL like this one, which you could create in TestNG as easily as I've done it in NUnit here. Just group your concerns into the Given / When / Then steps, and don't be afraid to refactor. It's easier to maintain than the English-language tools. The downside is that it's harder for the business to read, and tends to focus more on the testing side than the conversational side.

So, to answer your questions:

1. Yes, BDD is good for huge projects. There are some projects far bigger than yours which are using it.

2. Yes. As you've guessed, large amounts of scenarios will indeed become rapidly unmaintainable. If you haven't come across the Testing Pyramid, I recommend getting a copy of Agile Testing, which will explain it better than I can here, and give you some hints for getting it. In basic terms, you want few automated UI scenarios, more integration / API tests, and a ton of smaller unit tests at the bottom. If you can work out what the value of the code is to a user and provide a couple of examples of that, it's normally enough at the UI level. For instance, if you're writing validation, just take a couple of examples in which the user is guided to fill in the form, then write tests for the rest of the validation at the class level.
   
   Another way to reduce the number of scenarios needed is to spot when areas of the code are stabilising, then draw them out into their own libraries and / or micro-services. This will let you take the scenarios out of the main build. Try not to write scenarios for anything which is so utterly boring that it's simply going to work and never be touched again (eg: login, email gateways, etc.)
   
   If you're feeling generous, you can write a little toy app which shows how to use the services and make the scenarios for those services run over the toy app; this will also help future developers understand how to use your services too. JBehave itself was written using this technique.
   
   You can then use API tests or integration tests to check that your applications are hooked up correctly to the services, without worrying about the user-facing functionality.

3. Sometimes. See number 2 for techniques to reduce the number of testcases. Also, put the interesting scenarios at the top, not the bottom, and don't be afraid to delete anything which is obvious once you've read more interesting scenarios.
   
   eg: "The one where Fred bought a microwave with a discount and got a refund" is more interesting and covers all the functionality of "The one where Fred bought a microwave without a discount and got a refund".

If you're not sure which route to go down, I highly recommend duplicating your effort for a short while. Try both the TestNG approach and your own tool. If you use the Page Object pattern, this can be reused for both so the overhead will be small. From that, you'll be able to work out which one is best for your project.

I can confirm that it's a bit easier to move from the code-based DSL to an English-languge framework than the other way around, so if in doubt, go for code.