page object model: why not include assertions in page methods?

Question

First-time poster. I've been working in UI automation for many years, but was only recently introduced to/instructed to work with the Page Object Model. Most of it is common sense and includes techniques I've been using already, but there's a particular fine point which I haven't been able to justify in my own mind, despite searching extensively for a well-reasoned explanation. I'm hoping someone here might enlighten me, as this question has caused some consternation as I try to integrate the POM with my own best practices.

From http://code.google.com/p/selenium/wiki/PageObjects:

The code presented above shows an important point: the tests, not the PageObjects, should be responsible for making assertions about the state of a page.... Of course, as with every guideline there are exceptions...

From http://seleniumhq.org/docs/06_test_design_considerations.html#chapter06-reference:

There is a lot of flexibility in how the page objects may be designed, but there are a few basic rules for getting the desired maintainability of your test code. Page objects themselves should never be make verifications or assertions. This is part of your test and should always be within the test’s code, never in an page object. The page object will contain the representation of the page, and the services the page provides via methods but no code related to what is being tested should be within the page object.

There is one, single, verification which can, and should, be within the page object and that is to verify that the page, and possibly critical elements on the page, were loaded correctly. This verification should be done while instantiating the page object.

Both of these "guidelines" allow for potential exceptions, but I couldn't disagree more with the basic premise. I'm accustomed to doing a considerable amount of verification within "page methods", and I think the presence of verification there is a powerful technique for finding issues in a variety of contexts (i.e., verification occurs every time the method is called) rather than only occurring in the limited context of particular tests.

For example, let's imagine that when you login to your AUT, some text appears that says "logged in as USER". It's appropriate to have a single test validate this specifically, but why wouldn't you want to verify it every time login is called? This artifact is not directly related to whether the page "loaded correctly" or not, and it's not related to "what is being tested" in general, so according to the POM guidelines above, it clearly SHOULDN'T be in a page method... but it seems to me that it clearly SHOULD be there, to maximize the power of automation by verifying important artifacts as often as possible, with as little forethought as possible. Putting verification code in page methods multiplies the power of automation by allowing you to get a lot of verification "for free", without having to worry about it in your tests, and such frequent verification in different contexts often finds issues which you would NOT find if the verification were limited to, say, a single test for that artifact.

In other words, I tend to distinguish between test-specific verification and "general" verification, and I think it's perfectly appropriate/desirable for the latter to be included - extensively - in page methods. This promotes thinner tests and thicker page objects, which generally increases test maintainability by reusing more code - despite the opposite contention in these guidelines. Am I missing the point? What's the real rationale for NOT wanting verification in page methods? Is the situation I've described actually one of the 'exceptions' described in these guidelines, and therefore actually NOT inconsistent with the POM? Thanks in advance for your thoughts. -jn-

Answer 1

As a guideline, assertions should be done in tests and not in page objects. Of course, there are times when this isn't a pragmatic approach, but those times are infrequent enough for the above guideline to be right. Here are the reasons why I dislike having assertions in page objects:

1. It is quite frustrating to read a test that just calls verify methods where assertions are buried elsewhere in page objects. Where possible, it should be obvious what a test is asserting; this is best achieved when assertions are directly in a test. By hiding the assertions somewhere outside of a test, the intent of the test is not so clear.

2. Assertions in browser tests can be expensive - they can really slow your tests down. When you have hundreds or thousands of tests, minutes/hours can be added to your test execution time; this is A Bad Thing. If you move the assertions to just the tests that care about those particular assertions you'll find that you'll have much quicker tests and you will still catch the relevant defects. The question included the following:

   Putting verification code in page methods multiplies the power of automation by allow you to get a lot of verification "for free"

   Well, "Freedom Isn't Free" :) What you're actually multiplying is your test execution time.

3. Having assertions all over the place violates another good guideline; "One Assertion Per Test" ( http://blog.jayfields.com/2007/06/testing-one-assertion-per-test.html ). I don't stick religiously to it, but I try to follow the principle. Where possible, a test should be interested in one thing only.

4. The value of tests is reduced because one bug will cause loads of tests to fail thus preventing them from testing what they should be testing.

   For example, let's imagine that when you login to your AUT, some text appears that says "logged in as USER". It's appropriate to have a single test validate this specifically, but why wouldn't you want to verify it every time login is called?

   If you have the assertion in the page object class and the expected text changes, all tests that log in will fail. If instead the assertion is in the test then only one test will fail - the one that specifically tests for the correct message - leaving all the other tests to continue running to find other bugs. You don't need 5,000 tests to tell you that the login message is wrong; 1 test will do ;)

5. Having a class do more than one thing violates 'S' in SOLID, ie: 'Single Responsibility Principle' (SRP). A class should be responsible for one thing, and one thing only. In this instance a page-object class should be responsible for modelling a page (or section thereof) and nothing more. If it does any more than that (eg: including assertions) then you're violating SRP.

Answer 2

I too have struggled at times with this recommendation. I believe the reason behind this guideline is to keep your page objects reusable, and putting asserts inside your page objects could possibly limit their ability to be reused by a large number of unrelated tests. That said, I have put certain verification methods on my page objects like testing the caption for a header - in my experience, that is a better way to encapsulate test logic for elements of a page that don't change.

Another note - I have seen MVC applications that have domain models reused as page objects. When done correctly, this can significantly reduce redundant code in your testing library. With this pattern, the view models have no reference to a testing framework, so obviously, you could not put any asserts in them.