C++ High performance unit testing with Google Mock?

Question

I'm using Google Mock, and I'm struggling to mock out C++ system calls (specifically the C++11 chrono functions).

I'm know I'm supposed to make an interface, create a class to implement the interface for my actual implementation, and then mock out the interface on my tests. I'm trying to write an embedded application, so this level of indirection sounds too expensive for me.

What is the most efficient/performant way to incorporate system calls into Google Mock?

Answer 1

No, you don't need to resort to mocked static classes - that's one of many options.

If you're in an embedded environment where a virtual dispatch is too much overhead, or the compiler/linker optimizer for that architecture does a really bad job, then you can try the following 3 ways to mock the platform calls.

Assume for simplicity that you want to mock a function in the std::this_thread namespace, like sleep_for(std::milliseconds).

Example 0 - Untestable Baseline

Without mocking, let's assume your code looks like this:

class untestable_class
{
public:

    void some_function()
    {
        if (must_sleep())
        {
            auto sleep_duration = std::chrono::milliseconds(1000);
            std::this_thread::sleep_for(sleep_duration);
        }
    }
};

You'd use that class like this:

void use_untestable_class()
{
    untestable_class instance;
    instance.some_function();
}

Due to the dependency on the standard library sleep_for function, you have a platform dependency that makes some_function difficult to unit test without actually making an integration test out of it.

Example 1 - Testable Using Static Policy

By telling our class to use a specific thread policy using a class template, we can abstract away the platform dependency in unit tests. The policy can either be static or instance - they both remove the need for a virtual dispatch at runtime and they are pretty easy for a compiler/linker to optimize.

In the static policy case, we have one "real" policy that depends on the platform:

struct system_thread_policy1
{
    static void sleep_milliseconds(long milliseconds)
    {
        auto sleep_duration = std::chrono::milliseconds(milliseconds);
        std::this_thread::sleep_for(sleep_duration);
    }
};

We also have a "mock" policy that we can control in unit tests:

struct mock_thread_policy1
{
    // Mock attributes to verify interactions.
    static size_t sleep_milliseconds_count;
    static size_t sleep_milliseconds_arg1;

    // Resets the mock attributes before usage.
    static void sleep_milliseconds_reset()
    {
        sleep_milliseconds_count = 0;
        sleep_milliseconds_arg1 = 0;
    }

    static void sleep_milliseconds(size_t milliseconds)
    {
        sleep_milliseconds_count++;
        sleep_milliseconds_arg1 = milliseconds;
    }
};

// This is needed with MS compilers to keep all mock code in a header file.
__declspec(selectany) size_t mock_thread_policy1::sleep_milliseconds_count;
__declspec(selectany) size_t mock_thread_policy1::sleep_milliseconds_arg1;

The production class that uses the policy takes the policy type as a template parameter and calls into its sleep_milliseconds statically:

template <typename thread_policy>
class testable_class1
{
public:

    void some_function()
    {
        if (must_sleep())
        {
            thread_policy::sleep_milliseconds(sleep_duration_milliseconds);
        }
    }

private:

    enum { sleep_duration_milliseconds = 1000 };
};

In production code, testable_class1 is instantiated using the "real" policy:

void use_testable_class1()
{
    testable_class1<system_thread_policy1> instance;
    instance.some_function();
}

In the unit test, testable_class1 is instantiated using the "mock" policy:

void test_testable_class1()
{
    mock_thread_policy1::sleep_milliseconds_reset();
    testable_class1<mock_thread_policy1> instance;
    instance.some_function();

    assert(mock_thread_policy1::sleep_milliseconds_count == 1);
    assert(mock_thread_policy1::sleep_milliseconds_arg1 == 1000);
    //assert("some observable behavior on instance");
}

Upsides of this method:

• Features to test interactions, like the call count and argument checks above, can be added to the mock and be used to verify the class interactions unit tests.
• The static call makes it very easy for the optimizer to inline the "real" call to sleep_for.

Downsides of this method:

• The static state adds noise to the mock.
• The static state needs to be reset in each unit test where it's used, since different unit tests will change that sticky state.
• The static state makes it impossible to use the mock reliably if the test runner paralellizes unit tests, since different threads will fiddle with the same state, causing unpredictable behavior.

Example 2 - Testable Using Instance Policy

In the instance policy case, we have one "real" policy that depends on the platform:

struct system_thread_policy2
{
    void sleep_milliseconds(size_t milliseconds) const
    {
        auto sleep_duration = std::chrono::milliseconds(milliseconds);
        std::this_thread::sleep_for(sleep_duration);
    }
};

We also have a "mock" policy that we can control in unit tests:

struct mock_thread_policy2
{
    mutable size_t sleep_milliseconds_count;
    mutable size_t sleep_milliseconds_arg1;

    mock_thread_policy2()
        : sleep_milliseconds_count(0)
        , sleep_milliseconds_arg1(0)
    {
    }

    void sleep_milliseconds(size_t milliseconds) const
    {
        sleep_milliseconds_count++;
        sleep_milliseconds_arg1 = milliseconds;
    }
};

The production class that uses the policy takes the policy type as a template parameter, gets an instance of the policy injected in the contructor and calls into its sleep_milliseconds:

template <typename thread_policy>
class testable_class2
{
public:

    testable_class2(const thread_policy& policy = thread_policy()) : m_thread_policy(policy) { }

    void some_function() const
    {
        if (must_sleep())
        {
            m_thread_policy.sleep_milliseconds(sleep_duration_milliseconds);
        }
    }

private:

    // Needed since the thread policy is taken as a reference.
    testable_class2(const testable_class2&);
    testable_class2& operator=(const testable_class2&);

    enum { sleep_duration_milliseconds = 1000 };

    const thread_policy& m_thread_policy;
};

In production code, testable_class2 is instantiated using the "real" policy:

void use_testable_class2()
{
    const testable_class2<system_thread_policy2> instance;
    instance.some_function();
}

In the unit test, testable_class2 is instantiated using the "mock" policy:

void test_testable_class2()
{
    mock_thread_policy2 thread_policy;
    const testable_class2<mock_thread_policy2> instance(thread_policy);
    instance.some_function();

    assert(thread_policy.sleep_milliseconds_count == 1);
    assert(thread_policy.sleep_milliseconds_arg1 == 1000);
    //assert("some observable behavior on instance");
}

Upsides of this method:

• Features to test interactions, like the call count and argument checks above, can be added to the mock and be used to verify the class interactions unit tests.
• The instance call makes it very easy for the optimizer to inline the "real" call to sleep_for.
  • There's no static state in the mocks, which makes writing, reading and maintaining the unit tests easier.

Downsides of this method:

• The instance state adds mutable noise to the mock.
• The instance state adds noise to the client (testable_class2) - if the interactions don't need verifying, the policy can be passed by value in the constructor and most of the class goo goes away.

Example 3 - Testable Using Virtual Policy

This differs from the first 2 examples in the way that this relies on virtual dispatch, but leaves a likely possibility for the compiler/linker to optimize the virtual dispatch away if it can detect that the instance operated on is of base type.

First, we have the production base class that uses the "real" policy in a non-pure virtual function:

class testable_class3
{
public:

    void some_function()
    {
        if (must_sleep())
        {
            sleep_milliseconds(sleep_duration_milliseconds);
        }
    }

private:

    virtual void sleep_milliseconds(size_t milliseconds)
    {
        auto sleep_duration = std::chrono::milliseconds(milliseconds);
        std::this_thread::sleep_for(sleep_duration);
    }

    enum { sleep_duration_milliseconds = 1000 };
};

Second, we have the derived class that implements a "mock" policy in the virtual function (a kind of Template Method design pattern):

class mock_testable_class3 : public testable_class3
{
public:

    size_t sleep_milliseconds_count;
    size_t sleep_milliseconds_arg1;

    mock_testable_class3()
        : sleep_milliseconds_count(0)
        , sleep_milliseconds_arg1(0)
    {
    }

private:

    virtual void sleep_milliseconds(size_t milliseconds)
    {
        sleep_milliseconds_count++;
        sleep_milliseconds_arg1 = milliseconds;
    }
};

In production code, testable_class3 is just instantiated as itself:

void use_testable_class3()
{
    // Lots of opportunities to optimize away the virtual dispatch.
    testable_class3 instance;
    instance.some_function();
}

In the unit test, testable_class3 is instantiated using the "mock" derived class:

void test_testable_class3()
{
    mock_testable_class3 mock_instance;
    auto test_function = [](testable_class3& instance) { instance.some_function(); };
    test_function(mock_instance);

    assert(mock_instance.sleep_milliseconds_count == 1);
    assert(mock_instance.sleep_milliseconds_arg1 == 1000);
    //assert("some observable behavior on mock_instance");
}

Upsides of this method:

• Features to test interactions, like the call count and argument checks above, can be added to the mock and be used to verify the class interactions unit tests.
• The base class virtual call on "itself" makes it possible for the optimizer to inline the "real" call to sleep_for.
• There's no static state in the mocks, which makes writing, reading and maintaining the unit tests easier.

Downsides of this method:

• The base class cannot be marked final (C++11), since it must be allowed to be inherited from, and this might affect the rest of the class design if there is more complexity than the simple example above.
• The compiler/linker might be subpar or simply unable to optimize the virtual dispatch.

Test Run

All of the above can be tested with this:

int _tmain(int argc, _TCHAR* argv[])
{
    test_testable_class1();
    test_testable_class2();
    test_testable_class3();

    return 0;
}

and the complete runnable example is at http://pastebin.com/0qJaQVcD