testing in functional programming

Question

in object oriented programming i have objects and state. so i can mock all dependencies of an object and test the object. but functional programming (especially the pure) is about composing functions

it's easy to test function that doesn't depend on other functions. we just pass parameter and check the result. but what about function that takes another functions and returns functions?

let's say i have the code g = h1 ∘ h2 ∘ h3 ∘ h4. should i test just function g? but that's integration/functional testing. it's impossible to test all branches with only integration tests. what about unit testing? and it's getting more complicated when a function takes more parameters.

should i create custom functions and use them as mocks? wouldn't it be to expensive and error prone?

and what about monads? for example how to test console output or disk operations in haskell?

Answer 1

I too have been thinking about testing in functional code. I don't have all the answers, but I'll write a little bit here.

Functional programs are put together differently, and that demands different testing approaches.

If you take even the most cursory look at Haskell testing, you will inevitably come across QuickCheck and SmallCheck, two very well-known Haskell testing libraries. These both do "property-based testing".

In an OO language you would laboriously write individual tests to set up half a dozen mock objects, call a method or two, and verify that the expected external methods were called with the right data and / or the method ultimately returned the right answer. That's quite a bit of work. You probably only do this with one or two test cases.

QuickCheck is something else. You might write a property that says something like "if I sort this list, the output should have the same number of elements as the input". This is a one-liner. The QuickCheck library will then automatically build hundreds of randomly-generated lists, and check that the specified condition holds for every single one of them. And if it doesn't, it'll spit out the exact input on which the test failed.

(Both QuickCheck and SmallCheck do roughly the same thing. QuickCheck generates random tests, whereas SmallCheck systematically tries all combinations up to a specified size limit.)

You say you're worried about the combinatorial explosion of possible flow control paths to test, but with tools like this generating the test cases for you dynamically, manually writing enough tests isn't a problem. The only problem is coming up with enough data to test all flow paths.

Haskell can help with that too. I read a paper about a library [I don't know if it ever got released] which actually uses Haskell's lazy evaluation to detect what the code under test is doing with the input data. As in, it can detect whether the function you're testing looks at the contents of a list, or only the size of that list. It can detect which fields of that Customer record are being touched. And so forth. In this way, it automatically generates data, but doesn't waste hours generating different random variations of parts of the data that aren't even relevant for this particular code. (E.g., if you're sorting Customers by ID, it doesn't matter what's in the Name field.)

As for testing functions that take or produce functions... yeah, I don't have an answer to that.

Answer 2

In your example, you can test h1, h2, h3 and h4 separately, no problem, because they don't actually depend on each other. There's nothing stopping you testing g either. But is g a 'unit'? Well a very good definition of a unit test is given by Michael Feathers in his famous unit-testing book, Working Effectively With Legacy Code. He says unit tests are fast and reliable to run in the commit phase of your build pipeline and fast enough for developers to run. So g is a 'unit' by this measure. The other excellent perspective on unit testing is from Hexagonal Architecture, see TDD Where Did It All Go Wrong? They say that you want to test your application's API via the 'ports' it uses to interface to the outside world. Your g is a unit by this definition also. But what do they mean by a 'port' and can we relate this to Haskell? Well a typical port might be the database connection that the application uses to store things in a database. In Hexagonal, you would want to test that interface, likely by a mock. In Haskell terms, the core of the application is pure code and the ports are IO. The point is, you want to introduce your 'seams' (such as mocks) at the IO interface. So you probably don't want to worry about splitting g up.

But how do you introduce 'seams' for testing in Haskell? After all, there is no dependency injection framework (and nor should there be). Well the go-to answer to this is, as always in Haskell, to use functions and parameterisation. For example, suppose you have a function foo that's defined in terms of a function bar. You want to vary bar so it is a test double in your test and the regular value the rest of the time. Just make bar a parameter like this:

Module Foo
 foo bar = ... bar ...

Module Test
 foo = Foo.foo testBar

Module Real
  foo = Foo.foo realBar

You don't need to do it exactly like that but the point is that parameterisation gets you further than you'd think.

Alright, but what about testing IO in Haskell? How do we 'mock out' those IO actions? One way is to do it like you would in JavaScript: create data structures full of IO actions (they call them 'objects' ;-) ) and pass them around. Another way is to not use the IO type directly but instead access it through one of two monadic types - the real and the test type that are both instances of the same type class that defines the actions you want to swap out. Or you can make a Free Monad (using the free or operational packages) and write two interpreters - the test one and the real one.

In summary, testing pure code is so easy that practically anything you try will work. Testing IO code is harder, which is why we isolate it as much as possible.