Google Benchmark Custom Setup And Teardown Method

Question

I am using benchmark library to benchmark some codes. I want to call a setup method before calling the actual benchmark code one time and not to be repeated everytime, for multiple benchmark method calls.. For e.g:

static void BM_SomeFunction(benchmark::State& state) {
  // Perform setup here
  for (auto _ : state) {
    // This code gets timed
  }
}

As we can see the setup code will be called multiple times here, for the range I specify. I did have a look at the fixture tests. But my question is can it be done without using fixture tests. And if yes then how can we do it?

Answer 1

As far as I can remember, the function is called multiple times, since benchmark dynamically decides how many times your benchmark needs to be run in order to get reliable results. If you don't want to use fixtures, there are multiple workarounds. You can use a global or static class member bool to check if the setup function was already called (don't forget to set it after the setup routine has run). Another possibility is to use a Singleton that calls the setup method in its ctor:

class Setup
{
    Setup()
    {
        // call your setup function
        std::cout << "singleton ctor called only once in the whole program" << std::endl;
    }

public:
    static void PerformSetup()
    {
        static Setup setup;
    }
};

static void BM_SomeFunction(benchmark::State& state) {
  Setup::PerformSetup()
  for (auto _ : state) {
    // ...
  }
}

However, fixtures are quite simple to use and are made for such use-cases.

Define a fixture class which inherits from benchmark::Fixture:

class MyFixture : public benchmark::Fixture
{
public:
    // add members as needed

    MyFixture()
    {
        std::cout << "Ctor only called once per fixture testcase hat uses it" << std::endl;
        // call whatever setup functions you need in the fixtures ctor 
    }
};

Then use the BENCHMARK_F macro to use your fixture in the test.

BENCHMARK_F(MyFixture, TestcaseName)(benchmark::State& state)
{
    std::cout << "Benchmark function called more than once" << std::endl;
    for (auto _ : state)
    {
        //run your benchmark
    }
}

However, if you use the fixture in multiple benchmarks, the ctor will be called multiple times. If you really need a certain setup function to be called only once during the whole benchmark, you can use a Singleton or a static bool to work around this as described earlier. Maybe benchmark also has a built-in solution for that, but I don't know it.

---

Alternative to Singleton

If you don't like the singleton class, you can also use a global function like this:

void Setup()
{
    static bool callSetup = true;
    if (callSetup)
    {
        // Call your setup function
    }
    callSetup = false;
}

Greetings