Menu
I am currently working on a project that has been in production for over two years. The project makes extensive use of unit testing and scripted UI tests. Initialy unit tests covered the system framework, business rules and state transitions (or workflow). Test scripts are used for black box testing. However, over time the cost of maintaining our full set of unit tests has become increasingly expensive especially those relating to state.
After a bit of investigation we have found that test scripts are more effective (that is, provide better coverage) and are cheaper to maintain than unit tests relating to workflow. This isn't to say the value of unit tests has been completely negated but it does raise this question whether some classes of unit tests can be dropped in favour of test scripts.
Our project is run on an iterative incremental model.
375
Simple − You should write a test script that only asks testers to do one unique activity. This ensures that each function is thoroughly evaluated and that no phases in the software testing process are overlooked.
A test script is a line-by-line description of all the actions and data needed to properly perform a test. The script includes detailed explanations of the steps needed to achieve a specific goal within the program, as well as descriptions of the results that are expected for each step.
Tests can be set to run either a one-time check at a certain time interval or can be run immediately in real-time to review changes. In short, unit tests help developers detect problems immediately, then fix them quickly. With fewer resources spent finding bugs, teams can move on to the next phase of a project.
Generally speaking, unit tests are cheaper. They're easier to write—unless you're trying to add them to an existing app—which means developers don't spend quite as much time writing them. They're also cheaper to run: they don't usually require you to do special things for your environment or obtain external resources.
In more ways than one I have experienced the same kind of pain that you have vis-a-vis unit tests, especially in projects wherein the members are not at all enthusiastic about unit testing and many of them simply ignore or comment-out tests to be able to cheat source control, save time, etc. One former colleague even coined the term "Deadline Driven Development" for this.
In my opinion when facing this kind of challenge, the following are some guidelines vis-a-vis unit testing:
That being said, I still think that unit testing should not be discarded completely. Test scripts and unit tests have their own purposes. But a balance should be struck between an over-zealous attempt to maintain TDD and facing the realities of enterprise application development.
121
One of the answers on SO to the question of 'the limitations of Unit Testing' was that a unit testing becomes convoluted when it's used to test anything to do with INTEGRATION rather than function. Connecting to and using external services (database, SSH'ing to a another server, etc.) and User Interfaces were two of the examples used.
It's not that you CANT use Unit Testing for these things, its just that the difficulty involved in covering all the bases makes using this method of testing not worth it except in cases where reliability is paramount.
I use "script tests" for all my custom JavaScript UI code (template engine, effects and animations, etc.) and I find it quick and reliable if done right.
6
You normally use Unit Tests to do exactly this: test units. More exactly, to test if a unit complies to its interface specification/contract. If a unit test fails, you know exactly where the problem is: it is within the tested unit. This makes it easier to debug, especially since the result of a unit test can be processed automatically. Automatic regression tests come to mind here.
You start using scripted UI tests if you either want to leave the scope of unit tests or want to test things that cannot be tested well with unit tests. E.g. when you test code that interfaces with lots of external APIs that you cannot mock. Now you can provoke certain errors but tracking down where exactly in the code the failure is buried is much harder.
3
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
