How can I have a common test suite for multiple packages in go?

Question

When I'm writing an interface, its often convenient to define my tests in the same package as the interface, and then define multiple packages that implement the interface set, eg.

package/
package/impl/x <-- Implementation X
package/impl/y <-- Implementation Y

Is there an easy way to run the same test suite (in this case, located in package/*_test.go) in the sub packages?

The best solution I've come up with so far is to add a test package:

package/tests/

Which implements the test suite, and a test in each of the implementations to run the tests, but this has two downsides:

1) The tests in package/tests are not in _test.go files, and end up being part of the actual library, documented by godoc, etc.

2) The tests in package/tests are run by a custom test runner, which has to basically duplicate all the functionality of go test to scan for go tests and run them.

Seems like a pretty tacky solution.

Is there is a better way of doing this?

Answer 1

I don't really dislike the idea to use a separate testing library. If you have an interface and you have generic tests for each interface, other people that implement that interface might like to use these tests as well.

You could create a package "package/test" that contains a function

// functions needed for each implementation to test it
type Tester struct {
    func New() package.Interface
    func (*package.Interface) Done()
    // whatever you need. Leave nil if function does not apply
}

func TestInterface(t *testing.T, tester Tester)

Notice that the signature of TestInterface does not match to what go test expects. Now, for each package package/impl/x you add one file generic_test.go:

package x

import "testing"
import "package/test"

// run generic tests on this particular implementation
func TestInterface(t *testing.T) {
    test.TestInterface(t,test.Tester{New:New})
}

Where New() is the constructor function of your implementation. The advantage with this scheme is that

1. Your tests are reusable for whoever implements your interface, even from other packages
2. It is immediately obvious that you run the generic test suite
3. The test cases are where the implementation is and not at another, obscure place
4. The code can be adapted easily if one implementation needs special initialization or similar stuff
5. It's go test compatible (big plus!)

Of course, in some cases you need a more complicated TestInterface function, but this is the basic idea.

Answer 2

If you share a piece of code for reuse by different packages then yes, it is a library by definition. Even when used only for testing from *_test.go files. It's no different from importing "testing" of "fmt" in the _test.go file. And having the API documented by godoc is a plus, not minus IMHO.

Answer 3

Maybe something gets mixed up here a bit: If package a defines an interface only than there is no code to test as interfaces in Go are implementation free.

So I assume the methods in your interface in package a have constraints. E.g. in

interface Walker {
    Walk(step int)
    Tired() bool
}

you contract assumes that Tired returns true if more than 500 steps have been Walk'ed (and false otherwise) and your test code checks these dependencies (or assumption, contracts, invariants whatever you name it).

If this is the case I would provide (in package a) an exported function

func TestWalkerContract(w Walker) error {
    w.Walk(100)
    if w.Tired() { return errors.New("Tired after 100 steps") }
    w.Walk(450)
    if !w.Tired() { return errors.New("Not tired after 100+450 steps") }
}

Which documents the contract properly and can be used by packages b and c with types implementing walker to test their implementations in b_test.go and c_test.go. IMHO it is perfectly okay that these function like TestWalkerContract are displayed by godoc.

P.S. More common than Walk and Tired might be an error state which is kept and reported until cleared/reseted.