Create tests at run-time (google test)

Question

I have a Parser that has to be tested. There are lots of test input files to this Parser. The expected behaviour of the Parser is tested by comparing output of the Parser with corresponding pregenerated files.

Currently I'm processing YAML file in the test to get input files, expected files and their's description (in case of failure this description will be printed).

There are also parameters of the Parser that should be tested on the same input.

So, I need to form the following code in the test:

TEST(General, GeneralTestCase)
{
   test_parameters = yaml_conf.get_parameters("General", "GeneralTestCase");
   g_parser.parse(test_parameters);

   ASSERT_TRUE(g_env.parsed_as_expected()) << g_env.get_description("General", "GeneralTestCase");
}

TEST(Special, SpecialTestCase1)
{
   test_parameters = yaml_conf.get_parameters("Special", "SpecialTestCase1");
   g_parser.parse(test_parameters);

   ASSERT_TRUE(g_env.parsed_as_expected()) << g_env.get_description("Special", "SpecialTestCase1");
}

TEST(Special, SpecialTestCase2)
{
   test_parameters = yaml_conf.get_parameters("Special", "SpecialTestCase2");
   g_parser.parse(test_parameters);

   ASSERT_TRUE(g_env.parsed_as_expected()) << g_env.get_description("Special", "SpecialTestCase2");
}

It is easy to see the repetition of the code. So I feel there is a way to automate these tests.

Thanks in advance.

Answer 1

Use value-parameterized tests:

typedef std::pair<std::string, std::string> TestParam;

class ParserTest : public testing::TestWithParam<TestParam> {};

TEST_P(ParserTest, ParsesAsExpected) {
   test_parameters = yaml_conf.get_parameters(GetParam().first,
                                              GetParam().second);
   g_parser.parse(test_parameters);
   ASSERT_TRUE(g_env.parsed_as_expected());
}

INSTANTIATE_TEST_CASE_P(
    GeneralAndSpecial,
    ParserTest,
    testing::Values(
        TestParam("General", "GeneralTestCase")
        TestParam("Special", "SpecialTestCase1")
        TestParam("Special", "SpecialTestCase2")));

You can read the list of test cases from disk and return it as a vector:

std::vector<TestParam> ReadTestCasesFromDisk() { ... }

INSTANTIATE_TEST_CASE_P(
    GeneralAndSpecial,  // Instantiation name can be chosen arbitrarily.
    ParserTest,
    testing::ValuesIn(ReadTestCasesFromDisk()));

Answer 2

I have added two classes DynamicTestInfo & ScriptBasedTestInfo as well as RegisterDynamicTest function into google unit test. It requires C++17 at least (haven't analyzed backporting to C++11 or C++14) - it allow to create google unit tests dynamically / at run-time slightly simpler than current google API's.

For example usage could be something like this:

https://github.com/tapika/cppscriptcore/blob/f6823b76a3bbc0ed41b4f3cf05bc89fe32697277/SolutionProjectModel/cppexec/cppexec.cpp#L156

But this requires modified google test, see this file for example:

https://github.com/tapika/cppscriptcore/blob/f6823b76a3bbc0ed41b4f3cf05bc89fe32697277/SolutionProjectModel/logTesting/gtest/gtest.h#L819

I will try to merge changes to official google test repository.

I have changed also how tests are reported to user application (using <loc> tag) but that one requires modified google test adapter for Visual studio, for more information see following youtube video for more explanation how things works.

https://www.youtube.com/watch?v=-miGEb7M3V8

Using newer GTA you can get file system listing in test explorer, for example like this: