Use C++ catch framework to verify assert statement

Question

Is it possible to use the C++ CATCH framework to verify that an assert statement correctly identifies an invalid precondition?

// Source code
void loadDataFile(FILE* input) {
  assert(input != NULL);
  ...
}

// Test code
TEST_CASE("loadDataFile asserts out when passed NULL", "[loadDataFile]") {
   loadDataFile(NULL)
   // Now what do I look for?
}

Answer 1

Assuming that the first section of your example is the source code under test and the second part is the unit test, then you'll need to make a choice in how you handle this:

Some open-source frameworks like BDE and Boost have their own ASSERT macro which can be configured at application startup to behave different than C assert. For example, you can specify that a failed ASSERT throws an exception - and then you could use Catch's REQUIRE_THROWS() assertion to verify that your code enforced it's non-NULL FILE descriptor contract.

BDE example

#include <bsls_assert.h>

void loadDataFile(FILE* input) {
  BSLS_ASSERT_OPT(input != NULL);
  ...
}

TEST_CASE("loadDataFile asserts out when passed NULL", "[loadDataFile]") {
   // Opt-in to the 'throw exception on assert failure' handler
   // just for this test case.
   bsls::AssertFailureHandlerGuard guard(&bsls::Assert::failThrow);
   REQUIRE_THROWS_AS(loadDataFile(NULL), bsls::AssertFailedException);
}

Boost example

#include <boost/assert.hpp>

void loadDataFile(FILE* input) {
  BOOST_ASSERT(input != NULL);
  ...
}

namespace boost {
void assertion_failed(char const * expr, char const * function, char const * file, long line) {
    throw std::runtime_error("Assertion Failed"); // TODO: use expr, function, file, line
}
}

TEST_CASE("loadDataFile asserts out when passed NULL", "[loadDataFile]") {
   REQUIRE_THROWS(loadDataFile(NULL));
   // Now what do I look for?
}

You could roll your own assert() macro. This is re-inventing the wheel - see above examples instead.

You could change your code to throw std::invalid_argument() exception instead:

  void loadDataFile(FILE* input) {
    if (input == NULL) {
      throw std::invalid_argument("input file descriptor cannot be NULL");
    }
    ...
  }

You can test that your code enforces it's contract with:

REQUIRE_THROWS_AS(loadDataFile(NULL), std::invalid_argument);

This is introducing exceptions (and the need to handle them) into your code and this is a bigger change than you clients may be happy with - some companies have a no-exceptions rule, some platforms (e.g. embedded) do not support exceptions.

Finally, if you really wanted to, you could alter your code's interface to expose contract failure:

enum LoadDataFile_Result {
  LDF_Success,
  LDF_InputIsNull,
  ...
};

LoadDataFile_Result loadDataFile(FILE* input) {
  if (input == NULL) {
    // bail out early for contract failure
    return LDF_InputIsNull;
  }
  // input is non-NULL
  ...
  return LDF_Success;
}

...but this has the inherent risk that clients don't check the return value, the cause of many bugs, and feels like C all over again.

Answer 2

You may be interested in the google test framework. It has the ability to catch abnormal program termination with:

ASSERT_DEATH(statement, regex);
ASSERT_DEATH_IF_SUPPORTED(statement, regex);
ASSERT_EXIT(statement, predicate, regex);

EXPECT_DEATH(statement, regex);
EXPECT_DEATH_IF_SUPPORTED(statement, regex);
EXPECT_EXIT(statement, predicate, regex);

regex matches text on stderr predicate matches the program's exit code.

I suspect this works by forking the test program before the assertion.

documentation here:

https://github.com/google/googletest/blob/master/googletest/docs/advanced.md

Answer 3

If you can mock C functions, you could temporarily change the assertion failure to an exception.

For example, using Hippomocks your test case would look something like:

#ifndef NDEBUG
TEST_CASE("loadDataFile asserts out when passed NULL", "[loadDataFile]") {
    MockRepository mocks;
    mocks.OnCallFunc(__assert_fail).Throw(nullptr);

    CHECK_THROWS_AS(loadDataFile(NULL), std::nullptr_t);
}
#endif

Portability may be an issue, I've only tested this on Linux with glibc, and musl.