Is there any widely used C++ unit testing framework that supports test categories?

Question

I'm currently using CppUnit for testing, but am frustrated by the lack of support for the concept of Categories. Is there any widely used C++ unit testing framework that supports this idea?

Here's an example straight from NUnit documentation:

namespace NUnit.Tests
{
  using System;
  using NUnit.Framework;

  [TestFixture]
  [Category("LongRunning")]
  public class LongRunningTests
  {
    // ...
  }
}

The idea is to be able to group tests in different categories, and to execute tests in specified categories, or perhaps to exclude tests based on their category.

I have tried CppUnit, Boost.Test and Google Test, none of which directly support this idea.

For example, I have unit tests, integration tests, medium tests, and large tests. unit and integration tests run quickly, so I run them in every build configuration in the automated build: Release/Debug, x86/x64, linux/windows. medium tests take time to run, so I only run them in Release|x64|windows build. large tests exist as development aids: they are never run in the automated build. Then to add to the fun, I have features that only exist in x86|windows (it's complicated). I do all this with a complicated hierarchy, which CppUnit happily supports. However, it would be much nicer to do this with categories such as "release", "debug", "x86", etc.

In CppUnit, my tests are currently in Fixture classes. Ideally, I'd like to be able to tag those Fixtures with Categories, which I could then filter accordingly.

The one key thing that a test hierarchy doesn't do, which categories would do, is to have multiple categories on a single fixture.

Answer 1

I'm not certain I understand the question. Allow me to write some pseudo-code to see if I understand:

class TestCategory : public CPPUNIT_NS::TestCase {

    CPPUNIT_TEST_SUITE_BEGIN(TestCategory);
    CPPUNIT_TEST(myTestFunction);
    CPPUNITT_TEST_SUITE_END();

protected:

    void myTestFunction() {}

}

You can have n number of these classes. In the main:

// headers omitted

CPPUNT_TEST_SUITE_REGISTRATION(TestCategory)
// more registrations here
CPPUNIT_TEST_SUITE_REGISTRATION(TestCategory_n_)

int main(int argc, char** argv) {

   // test runner steps
}

Forgive me for being obtuse, but you're looking for a logical partition in the testing architecture, yes? If so, you can group your tests via classes and include them as necessary in the main function.

Help me understand. Thanks.

Answer 2

The --run_test option in Boost.Test supports wildcard matching for the test and or suite names. I'm not entirely sure how flexible the matching is, but if you were to put tag-like naming part into your test(-suite) names, you could perhaps run --run_test=suite_LongRunning*/* something like that.

---

I also just stumbled on xUnit++ which has Attributes that can be used to run different categories. (Since it's modeled after NUnit, this should be no wonder.)