How do you create tests for "make check" with GNU autotools

Question

I'm using GNU autotools for the build system on a particular project. I want to start writing automated tests for verifcation. I would like to just type "make check" to have it automatically run these. My project is in C++, although I am still curious about writing automated tests for other languages as well.

Is this compatible with pretty much every unit testing framework out there (I was thinking of using cppunit)? How do I hook these unit testing frameworks into make check? Can I make sure that I don't require the unit test software to be installed to be able to configure and build the rest of the project?

Answer 1

To make test run when you issue make check, you need to add them to the TESTS variable

Assuming you've already built the executable that runs the unit tests, you just add the name of the executable to the TESTS variable like this:

TESTS=my-test-executable

It should then be automatically run when you make check, and if the executable returns a non-zero value, it will report that as a test failure. If you have multiple unit test executables, just list them all in the TESTS variable:

TESTS=my-first-test my-second-test my-third-test

and they will all get run.

Answer 2

I'm using Check 0.9.10

    configure.ac
    Makefile.am
    src/Makefile.am
    src/foo.c
    tests/check_foo.c
    tests/Makefile.am

1. ./configure.ac

   PKG_CHECK_MODULES([CHECK], [check >= 0.9.10])

2. ./tests/Makefile.am for test codes

   TESTS = check_foo
   check_PROGRAMS = check_foo
   check_foo_SOURCES = check_foo.c $(top_builddir)/src/foo.h
   check_foo_CFLAGS = @CHECK_CFLAGS@
   

3. and write test code, ./tests/check_foo.c

   START_TEST (test_foo)
   {
       ck_assert( foo() == 0 );
       ck_assert_int_eq( foo(), 0);
   }
   END_TEST
   
   /// And there are some tcase_xxx codes to run this test
   

Using check you can use timeout and raise signal. it is very helpful.

Answer 3

You seem to be asking 2 questions in the first paragraph.

The first is about adding tests to the GNU autotools toolchain - but those tests, if I'm understanding you correctly, are for both validating that the environment necessary to build your application exists (dependent libraries and tools) as well as adapt the build to the environment (platform specific differences).

The second is about unit testing your C++ application and where to invoke those tests, you've proposed doing so from the autotools tool chain, presumably from the configure script. Doing that isn't conventional though - putting a 'test' target in your Makefile is a more conventional way of executing your test suite. The typical steps for building and installing an application with autotools (at least from a user's perspective, not from your, the developer, perspective) is to run the configure script, then run make, then optionally run make test and finally make install.

For the second issue, not wanting cppunit to be a dependency, why not just distribute it with your c++ application? Can you just put it right in what ever archive format you're using (be it tar.gz, tar.bz2 or .zip) along with your source code. I've used cppunit in the past and was happy with it, having used JUnit and other xUnit style frameworks.

Answer 4

Here is a method without dependencies:

#src/Makefile.am
check_PROGRAMS = test1 test2
test1_SOURCES = test/test1.c code_needed_to_test1.h code_needed_to_test1.c
test2_SOURCES = test/test2.c code_needed_to_test2.h code_needed_to_test2.c
TESTS = $(check_PROGRAMS)

The make check will naturally work and show formatted and summarized output:

$ make check
...
PASS: test1
PASS: test2
============================================================================
Testsuite summary for foo 1.0
============================================================================
# TOTAL: 2
# PASS:  2
# SKIP:  0
# XFAIL: 0
# FAIL:  0
# XPASS: 0
# ERROR: 0
============================================================================

1. When you do a make dist nothing from src/test/* will be in the tarball. Test code is not in the dist, only source will be.
2. When you do a make distcheck it will run make check and run your tests.