Is it a good or a bad thing that a suite of quickcheck tests match the implementations?

Question

I'm trying to get started with Haskell's QuickCheck, and while I am familiar with the concepts behind the testing methodology, this is the first time I am trying to put it to use on a project that goes beyond testing stuff like reverse . reverse == id and that kind of thing. I want to know if it is useful to apply it to business logic (I think it very much could be).

So a couple of existing business logic type functions that I would like to test look like the following:

shouldDiscountProduct :: User -> Product -> Bool
shouldDiscountProduct user product =
  if M.isNothing (userDiscountCode user)
     then False
     else if (productDiscount product) then True
                                       else False

For this function I can write a QuickCheck spec like the following:

data ShouldDiscountProductParams
  = ShouldDiscountProductParams User Product

instance Show ShouldDiscountProductParams where
  show (ShouldDiscountProductParams u p) =
    "ShouldDiscountProductParams:\n\n" <>
    "- " <> show u <> "\n\n" <>
    "- " <> show p

instance Arbitrary ShouldDiscountProductParams where
  arbitrary = ShouldDiscountProductParams <$> arbitrary <*> arbitrary

shouldDiscountProduct :: Spec
shouldDiscountProduct = it behavior (property verify)
  where
    behavior =
      "when product elegible for discount\n"
      <> " and user has discount code"

    verify (ShouldDiscountProductParams p t) =
      subject p t `shouldBe` expectation p t

    subject =
      SUT.shouldDiscountProduct

    expectation User{..} Product{..} =
      case (userDiscountCode, productDiscount) of
        (Just _, Just _) -> True
        _ -> False

And what I end up with is a function expectation that verifies the current implementation of shouldDiscountProduct, just more elegantly. So now I have a test, I can refactor my original function. But my natural inclination would be to change it to the implementation in expectation:

shouldDiscountProduct User{..} Product{..} =
  case (userDiscountCode, productDiscount) of
    (Just _, Just _) -> True
    _ -> False

But this is fine right? If I want to change this function again in future I have the same function ready to verify my changes are appropriate and not inadvertently breaking something.

Or is this overkill / double bookkeeping? I suppose I have had ingrained into me from OOP testing that you should try and avoid mirroring the implementation details as much as possible, this literally couldn't be any further than that, it is the implementation!

I then think as I go through my project and add these kinds of tests, I am effectively going to be addding these tests, and then refactoring to the cleaner implementation I implement in the expectation assertion. Obviously this isn't going to be the case for more complex functions than these, but in the round I think will be the case.

What are people experiences with using property based testing for business logic-type functions? Are there any good resources out there for this kind of thing? I guess I just want to verify that I am using QC in an appropriate way, and its just my OOP past throwing doubts in my mind about this...

Answer 1

I'm sorry to jump in a few months later, but as this question easily pops on Google I think it needs a better answer.

Ivan's answer is about unit tests while you are talking about property tests, so let's disregard it.

Dfeuer tells you when it's acceptable to mirror the implementation, but not what to do for your use case.

It's a common mistake with Property based tests (PBT) to rewrite the implementation code at first. But this is not what PBT are for. They exist to check properties of your function. Hey, don't worry, we all do this mistake the first few times we write PBT :D

A type of property you could check here is whether your function response is consistent with its input:

if SUT.shouldDiscountProduct p t 
then isJust (userDiscountCode p) && isJust (productDiscount t) 
else isNothing (userDiscountCode p) || isNothing (productDiscount t)

This one is subtle in your particular use case, but pay attention, we reversed the logic. Your test checks the input, and based on this, asserts on the output. My test checks on the output, and based on this, asserts on the input. In other use cases this could be much less symmetric. Most of the code can also be refactored, I let you this exercise ;)

But you may find other types of properties! E.g. invariance properties:

SUT.shouldDiscountProduct p{userDiscountCode = Nothing} t == False
SUT.shouldDiscountProduct p{productDiscount = Nothing} t == False

See what we did here? We fixed one part of the input (e.g. the user discount code is always empty) and we assert that no matter how everything else varies, the output is invariant (always false). Same goes for product discount.

One last example: you could use an analogous property to check your old code and your new code behave exactly the same:

shouldDiscountProduct user product =
  if M.isNothing (userDiscountCode user)
     then False
     else if (productDiscount product) then True
                                       else False

shouldDiscountProduct' user product
  | Just _ <- userDiscountCode user
  , Just _ <- productDiscount product
  = True
  | otherwise = False

SUT.shouldDiscountProduct p t = SUT.shouldDiscountProduct' p t

Which reads as "No matter the input, the rewritten function must always return the same value as the old function". This is so cool when refactoring!

I hope this helps you grasp the idea behind Property based tests: stop worrying so much about the value returned by your function, and start wondering about some behaviors your function has.

Note, PBT are not an enemy of unit tests, they actually fit well together. You could use 1 or 2 unit tests if it makes you feel safer about actual values, then write Property test(s) to assert your function has some behaviors, no matter the input.

Answer 2

Basically the only times it makes sense for property checking to compare two implementations of the same function are when:

1. Both function are part of the API, and they should each implement a certain function. For example, we generally want liftEq (==) = (==). So we should test that liftEq for the type we're defining satisfies this property.

2. One implementation is obviously correct, but inefficient, while another is efficient but not obviously correct. In this case, the test suite should define the obviously correct version and check the efficient version against it.

For typical "business logic", neither of these apply. There might, however, be some special cases where they do. For example, you could have two different functions you call under different circumstances that are supposed to agree under certain conditions.