Running Boost unit tests on different processes

Question

I want to do unit testing in a SystemC program. The idea is to have multiple test suites with several tests in each suite. Each one of the tests would require resetting the SystemC framework (e.g., by calling sc_simcontext::reset()), but that is actually not possible due to some bug that is apparently not going to be fixed anytime soon. Therefore, I decided to come up with a workaround.

I found out that if I run each test on a different process everything works fine. The following code snippet gives an overview of the scheme I used to make it work:

void test1() {
  // ...
  sc_start();
}

void test2() {
  // ...
  sc_start();
}

typedef std::function<void()> TestFunction;

void run_test(TestFunction test_function) {
  pid_t pid = fork();
  switch (pid) {
  case -1:
    throw std::runtime_error("Error forking process");
  case 0:
    test_function();
    exit(0);
  default:
    waitpid(pid, nullptr, 0);
    break;
  }
}

int main() {
  run_test(test1);
  run_test(test2);
}

Now I want to implement such a testing scheme with Boost Unit Test.

I have been studying the internals of Boost Unit Test library and I have found that unit_test_main seems to be the function that triggers the execution of all the tests. But I could not devise a non-intrusive way to interact with Boost Unit Test in order to run each test on a different process.

Does anyone know of a simple solution for running each test on a different process?

Answer 1

I am not 100% satisfied with the solution that I came up, but I will post it anyway. For convenience, I encapsulated everything into a namespace:

Header file:

namespace util {

typedef std::function<void()> TestFunction;

void run_test(TestFunction test_function);

} // namespace util

#define SYSTEMC_TEST_CASE(name)       \
  void name##_impl();                 \
  BOOST_AUTO_TEST_CASE(name) {        \
    util::run_test(name##_impl);      \
  }                                   \
  void name##_impl()

Source file:

namespace util {

void run_test(TestFunction test_function) {
  pid_t pid = fork();
  switch (pid) {
    case -1:
      throw std::runtime_error("Error forking process");
    case 0:
      try { test_function(); }
      catch (const std::exception& e) {
        std::cout << boost::format("Exception caught: %1%") % e.what() << std::endl;
        exit(1);
      }
      catch (...) { exit(1); }
      exit(0);
    default:
      waitpid(pid, nullptr, 0);
      break;
  }
}

} // namespace util

Usage example:

BOOST_AUTO_TEST_SUITE(suite)

SYSTEMC_TEST_CASE(test_case1) {
  // ...
}

SYSTEMC_TEST_CASE(test_case2) {
  // ...
}

BOOST_AUTO_TEST_SUITE_END()

main.cpp contains:

#define BOOST_TEST_MODULE TestModule
#define BOOST_TEST_DYN_LINK
#define BOOST_TEST_NO_MAIN
#include <boost/test/unit_test.hpp>

#include <systemc.h>

boost::unit_test::test_suite* init_unit_test_suite(int, char*[]) {
  using namespace ::boost::unit_test;
  assign_op(framework::master_test_suite().p_name.value,
      BOOST_TEST_STRINGIZE(BOOST_TEST_MODULE).trim("\""), 0);
  return 0;
}

int sc_main(int argc, char* argv[]) {
  return boost::unit_test::unit_test_main(&init_unit_test, argc, argv);
}

Each test case will now be executed on a different process. Therefore, SystemC runs multiple times during a single execution without any problem.

The only real issue of this solution is that for some reason it is not possible to use a file sink when outputting XML results. But I have found that everything works fine if the sink is stderr and the output is redirected to a file.