Do shoulda-matchers' ActiveRecord matchers violate the "test behavior not implementation" rule?

Question

For example, if I am using should validate_presence_of in my spec, that's only testing that I have the validate_presence_of piece of code inside my model, and that's testing implementation. More importantly, isn't that spec totally useless for testing the real problem, which is "if I don't fill out a certain field, will the model be saved successfully?"

Answer 1

Some of shoulda-matchers' matchers don't test implementation, they test behavior. For example, look at the source for allow_value (which validate_presence_of uses): #matches? actually sets the instance's attribute to the value and checks to see if that causes an error. All of shoulda-matchers' matchers that test validations (its ActiveModel matchers) that I've looked at work the same way; they actually test that the model rejects bad values.

Note that if you trust ActiveModel and ActiveRecord to be fully tested, it shouldn't matter whether the matchers test behavior or just test that the macros are used.

Unit-testing a model's validations is definitely useful. Suppose you're doing BDD and implementing a simple form that creates an instance of a model. You would first write an acceptance test (a Cucumber or rspec scenario) that tests the happy path of filling out the form correctly and successfully creating the instance. You would then write a second acceptance test with an error in the form that demonstrates that when there's an error in the form no instance is saved and the form is redisplayed with the appropriate error message.

Once you've got that error-path scenario for one of the errors it is possible to make in the form, you will find that if you write more error-path scenarios for the other errors they will be very repetitive -- the only things that will be different are the erroneous field value and the error message. And then you'll have a lot of full-stack scenarios, which take a long time to run. So don't write more than that first error-path scenario. Instead, just write unit tests for the validations that would catch each error. Now most of your tests are simple and fast. (This is a specific example of the general BDD technique of dropping down from acceptance tests to unit tests to handle details.)

However, I don't find shoulda-matchers' ActiveRecord matchers very useful. Considering the matchers that test associations, I find that my acceptance tests always force me to add all the associations to my models that I need, and there's nothing left to do in unit tests. The ActiveRecord matchers that test database features that are invisible to the app (e.g. have_db_index) are useful if you're strictly test-driven, but I tend to slack off there. Also, for what it's worth, the ActiveRecord matchers don't test behavior (which would be hard to implement), only that the corresponding macros are used.

One exception where I do find a shoulda-matchers ActiveRecord matcher useful is deletion of dependent objects. I sometimes find that no acceptance spec has already forced me to handle what happens to associated objects when an object is deleted. The ActiveRecord way to make that happen is to add the :dependent option to the belongs_to, has_many or has_one association. Writing an example that uses shoulda-matchers' belong_to or have_many or have_one matcher with the .dependent option is the most convenient way I know to test that.